From ea9297bedc8fb3f81274c94d229788c30bcd9bd1 Mon Sep 17 00:00:00 2001 From: Amitay Keisar Date: Thu, 20 Aug 2026 00:49:10 +0300 Subject: [PATCH 1/5] =?UTF-8?q?feat!:=20the=20great=20cleanup=20=E2=80=94?= =?UTF-8?q?=20remove=20the=20local=20AI=20stack=20and=20the=20Files=20band?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Refocus the app on what actually gets used: the window switcher, the four-finger launcher, and clipboard history. The full-featured app is preserved on the v1 branch and the v1.0.0 release (notarized DMG). Removed outright: - The entire on-device AI platform: the GemmaRuntime (MLX) target and its package deps, Sources/ThreeFingerSwitcher/AI/ (executor, agent loop, tool routing, memory, skills, media, fleet, voice/computer-use, background autonomy + audit, parked sessions), the AI command canvas, the notch home zone + timeline, the region picker, the Hub AI pages, ModelManagementView, the .aiCommand item kind, the speakLastResponse action, and 46 AI test suites. - The Files band: Sources/ThreeFingerSwitcher/Files/, FilesBandView / FilesDwellArming / BubbleMorph, HubFilesPage, the recognizer's Files-drill sub-state, the .fileEntry kind, and 11 Files test suites. - 15 AI/Files openspec capability folders + 16 active ai-* change folders (the archive stays as design history). Surgical edits: AppCoordinator (~1,100 lines of wiring out), GestureRecognizer (canvas-resolve / notch-flick / files-drill out), the launcher stack, the Hub, onboarding, AppSettings (45 keys out), GestureBindings (switcher-only), Package.swift/main.swift (no MLX), build-app.sh, release.yml (Xcode pin was MLX-only), Info.plist (5 AI usage strings out), README + CLAUDE.md. Migration: Favorites schema v3 — bands decode items lossily (ContextBand.FailableItem) so records still carrying retired kinds drop those items instead of failing wholesale and being reseeded; the empty seeded "AI" band is removed by sentinel id. swift build clean; swift test: 805 tests, 0 failures. The whole app now builds under plain swift build (no MLX/xcodebuild-only target). Co-Authored-By: Claude Fable 5 --- .github/workflows/release.yml | 11 +- CLAUDE.md | 60 +- Package.resolved | 171 +- Package.swift | 42 +- README.md | 61 +- Resources/Info.plist | 16 - .../GemmaRuntime/BatchedGemmaMLXRuntime.swift | 523 ------ Sources/GemmaRuntime/CloudVideoRuntime.swift | 175 -- Sources/GemmaRuntime/GemmaMLXRuntime.swift | 740 -------- .../GemmaResumableDownloader.swift | 463 ----- Sources/GemmaRuntime/GemmaRuntime.swift | 304 ---- Sources/GemmaRuntime/LaneDispatch.swift | 169 -- Sources/GemmaRuntime/LocalLTXVRuntime.swift | 158 -- Sources/GemmaRuntime/MFluxImageRuntime.swift | 418 ----- Sources/GemmaRuntime/TernaryCPURuntime.swift | 145 -- Sources/ThreeFingerSwitcher/AI/AIBand.swift | 41 - .../ThreeFingerSwitcher/AI/AICommand.swift | 357 ---- .../AI/AICommandCatalog.swift | 285 --- .../AI/AICommandExecutor.swift | 526 ------ Sources/ThreeFingerSwitcher/AI/AIError.swift | 195 --- .../AI/AIRuntimeInjection.swift | 36 - .../AI/Agent/AgentContextBudgetProvider.swift | 45 - .../AI/Agent/AgentContextCostModel.swift | 78 - .../AI/Agent/AgentContextPreset.swift | 34 - .../AI/Agent/AgentConversation.swift | 240 --- .../AI/Agent/AgentLoop.swift | 270 --- .../AI/Agent/AutoApprovingGate.swift | 70 - .../AI/Agent/CanvasApprovalGate.swift | 68 - .../AI/Agent/ChatTemplate.swift | 34 - .../AI/Agent/ConcurrencyBudget.swift | 62 - .../AI/Agent/ContextBudget.swift | 44 - .../AI/Agent/ConversationCompactor.swift | 104 -- .../AI/Agent/RouteSchema.swift | 46 - .../AI/Agent/Subagent.swift | 59 - .../AI/Agent/SubagentToolContributor.swift | 93 - .../AI/Agent/TaskKindToolContributor.swift | 133 -- .../AI/Agent/ToolCandidateSource.swift | 79 - .../AI/Agent/ToolContracts.swift | 123 -- .../AI/Agent/ToolRegistry.swift | 62 - .../AI/Agent/ToolRouter.swift | 52 - .../AI/Agent/WritePolicyResolving.swift | 16 - .../AI/Audit/AuditError.swift | 35 - .../AI/Audit/AuditLog.swift | 220 --- .../AI/Audit/AuditRecord.swift | 123 -- .../AI/Audit/BackgroundGate.swift | 63 - .../AI/Audit/BackgroundToolRunner.swift | 81 - .../AI/Audit/BlastRadius.swift | 58 - .../AI/Audit/Whitelist.swift | 101 -- .../AI/Audit/WritePolicyResolution.swift | 71 - .../AI/Ax/AXActionError.swift | 56 - .../AI/Ax/AXActionPerformer.swift | 189 -- .../AI/Ax/AXWindowReader.swift | 130 -- .../AI/Ax/AXWindowSnapshot.swift | 176 -- .../AI/Ax/AgentActionArbiter.swift | 67 - .../AI/Ax/ComputerUseToolContributor.swift | 242 --- .../AI/Compute/ComputeLane.swift | 105 -- .../AI/Compute/GatedLaneRouting.swift | 36 - .../AI/Compute/LaneArbiter.swift | 91 - .../AI/Compute/LaneResidencyBudget.swift | 52 - .../AI/Compute/StubTernaryRuntime.swift | 110 -- .../ThreeFingerSwitcher/AI/DevAIRuntime.swift | 64 - .../AI/EvictionPolicy.swift | 107 -- .../AI/Fleet/FleetCloudEscalation.swift | 60 - .../AI/Fleet/FleetError.swift | 43 - .../AI/Fleet/FleetModelRegistry.swift | 27 - .../AI/Fleet/FleetRoster.swift | 263 --- .../AI/Fleet/ResidencyPlanner.swift | 180 -- .../AI/ForwardingLLMRuntime.swift | 68 - .../AI/FullPotential/AIGatingSnapshot.swift | 68 - .../AI/FullPotential/FullPotentialGate.swift | 96 - .../AI/Handoff/ClaudeHandoffContributor.swift | 315 ---- .../AI/Handoff/HandoffBudget.swift | 150 -- .../AI/Handoff/HandoffError.swift | 45 - .../AI/Handoff/HandoffLauncher.swift | 66 - .../AI/Handoff/HandoffSeams.swift | 50 - .../ThreeFingerSwitcher/AI/LLMRuntime.swift | 380 ---- .../AI/Media/Image/ImageCostDisclosure.swift | 82 - .../AI/Media/Image/ImageModelCatalog.swift | 147 -- .../Media/Image/ImageRequestValidation.swift | 102 -- .../AI/Media/Image/ImageResidencyClass.swift | 95 - .../AI/Media/Image/StubImageRuntime.swift | 147 -- .../AI/Media/MediaArgs.swift | 76 - .../AI/Media/MediaCanvasResolve.swift | 52 - .../AI/Media/MediaError.swift | 77 - .../AI/Media/MediaGallery.swift | 92 - .../AI/Media/MediaGenSink.swift | 306 ---- .../AI/Media/MediaJobState.swift | 70 - .../AI/Media/MediaParkFeed.swift | 46 - .../AI/Media/MediaResidency.swift | 54 - .../AI/Media/MediaRuntime.swift | 153 -- .../AI/Media/MediaSeed.swift | 64 - .../AI/Media/MediaToolContributor.swift | 173 -- .../AI/Media/MediaVideoBudget.swift | 57 - .../AI/Media/StubMediaRuntime.swift | 77 - .../AI/Media/Video/StubVideoRuntimes.swift | 118 -- .../AI/Media/Video/VideoBudget.swift | 232 --- .../AI/Media/Video/VideoProvider.swift | 55 - .../AI/Media/Video/VideoTierResolution.swift | 102 -- .../Media/Video/VideoUploadDisclosure.swift | 78 - .../AI/Memory/MemoryAudit.swift | 51 - .../AI/Memory/MemoryDocument.swift | 139 -- .../AI/Memory/MemoryError.swift | 34 - .../AI/Memory/MemoryReconcile.swift | 28 - .../AI/Memory/MemoryStore.swift | 326 ---- .../AI/Memory/MemorySubfile.swift | 111 -- .../AI/Memory/MemoryToolProvider.swift | 262 --- .../AI/MemoryPressureSource.swift | 69 - .../ThreeFingerSwitcher/AI/ModelManager.swift | 789 --------- .../AI/ModelRegistry.swift | 228 --- .../AI/Parked/NotchHomeZoneAnchor.swift | 155 -- .../AI/Parked/NotchRevealModel.swift | 115 -- .../AI/Parked/NotchSessionEngine.swift | 741 -------- .../AI/Parked/NotchTuning.swift | 54 - .../AI/Parked/ParkError.swift | 37 - .../AI/Parked/ParkLifecycle.swift | 49 - .../AI/Parked/ParkLifecycleCoordinator.swift | 49 - .../AI/Parked/ParkScheduler.swift | 144 -- .../AI/Parked/ParkedSession.swift | 73 - .../AI/Parked/ParkedSessionStore.swift | 225 --- .../AI/PromptTemplate.swift | 94 - .../AI/RuntimeParameter.swift | 104 -- .../AI/SelectionProviding.swift | 55 - .../AI/SelectionService.swift | 484 ----- .../AI/Skills/ClaudeHandoffConfig.swift | 30 - .../AI/Skills/DocIndex.swift | 82 - .../AI/Skills/SkillError.swift | 28 - .../AI/Skills/SkillFile.swift | 307 ---- .../AI/Skills/SkillFolderWatcher.swift | 126 -- .../AI/Skills/SkillManifest.swift | 43 - .../AI/Skills/SkillStore.swift | 137 -- .../AI/Skills/SkillToolProvider.swift | 127 -- .../AI/StubLLMRuntime.swift | 307 ---- .../AI/TaskDispatching.swift | 31 - .../AI/Tasks/ParsedActions.swift | 221 --- .../AI/Tasks/TaskDispatcher.swift | 246 --- .../AI/Tasks/TaskReview.swift | 88 - .../AI/Tasks/TaskSinks.swift | 488 ------ .../AI/Voice/PTTArmingModel.swift | 134 -- .../AI/Voice/PTTKeyMonitor.swift | 120 -- .../AI/Voice/SentenceChunker.swift | 135 -- .../AI/Voice/SpeechAnalyzerTranscriber.swift | 177 -- .../AI/Voice/SpeechSeams.swift | 117 -- .../AI/Voice/SystemSpeechSynthesizer.swift | 63 - .../AI/Voice/VoiceError.swift | 49 - .../AI/Voice/VoiceSessionController.swift | 166 -- .../AI/Voice/VoiceToolContributor.swift | 113 -- .../AI/Voice/VoiceTurnModel.swift | 178 -- .../App/AppCoordinator.swift | 1553 +---------------- .../App/ParkController.swift | 536 ------ .../App/StatusItemController.swift | 8 - .../Dock/DockPreviewError.swift | 6 +- .../ThreeFingerSwitcher/Dock/DockTile.swift | 2 +- .../Files/DirectoryLister.swift | 168 -- .../ThreeFingerSwitcher/Files/FileEntry.swift | 103 -- .../Files/FileOpenService.swift | 283 --- .../Files/FileWorkspace.swift | 145 -- .../Files/FilesActionMenu.swift | 157 -- .../Files/FilesBandBuilder.swift | 92 - .../Files/FilesColumnController.swift | 307 ---- .../Files/FilesDelivery.swift | 45 - .../Files/FilesNavigationModel.swift | 445 ----- .../Files/FilesToolCatalog.swift | 28 - .../Files/OpenWithEntry.swift | 39 - .../Gesture/FlickExcursionClassifier.swift | 103 -- .../Gesture/GestureBindings.swift | 175 +- .../Gesture/GesturePose.swift | 34 +- .../Gesture/GestureRecognizer.swift | 378 +--- .../Gesture/LauncherTourEngine.swift | 2 +- .../ThreeFingerSwitcher/Hub/BandsCanvas.swift | 576 +----- .../Hub/HubBackgroundAutonomy.swift | 327 ---- .../Hub/HubBindingPicker.swift | 67 +- .../Hub/HubFeaturePages.swift | 465 +---- .../Hub/HubFilesPage.swift | 631 ------- .../Hub/HubFleetRosterView.swift | 354 ---- .../Hub/HubGesturePreview.swift | 2 +- .../Hub/HubOverviewPage.swift | 6 - .../Hub/HubPreviewModels.swift | 8 +- Sources/ThreeFingerSwitcher/Hub/HubView.swift | 37 +- .../KeyboardLanguage/BrowserRegistry.swift | 2 +- .../KeyboardLanguagePolicy.swift | 2 +- .../Launcher/ClaudeLaunch.swift | 2 +- .../Launcher/FavoritesStore.swift | 88 +- .../Launcher/LaunchItem.swift | 65 +- .../Launcher/LaunchService.swift | 42 +- .../Onboarding/FirstTouchWizardModel.swift | 16 +- .../Onboarding/WizardActs.swift | 8 +- .../Onboarding/WizardContext.swift | 2 +- .../Onboarding/WizardMotion.swift | 2 +- .../Onboarding/WizardTourBands.swift | 19 - .../Overlay/AICommandCanvasView.swift | 591 ------- .../Overlay/BidiText.swift | 197 --- .../Overlay/BubbleMorph.swift | 60 - .../Overlay/FilesBandView.swift | 977 ----------- .../Overlay/FilesDwellArming.swift | 40 - .../Overlay/LauncherGridLayout.swift | 92 - .../Overlay/LauncherModel.swift | 322 +--- .../Overlay/LauncherOverlayController.swift | 461 +---- .../Overlay/LauncherView.swift | 51 +- .../Overlay/MediaCanvasPlayerOverlay.swift | 156 -- .../Overlay/NotchHomeZoneController.swift | 389 ----- .../Overlay/NotchHomeZoneOverlay.swift | 1113 ------------ .../Overlay/OverlayController.swift | 18 +- .../Overlay/RegionPickerOverlay.swift | 138 -- .../RegionPicker/RegionPickerModel.swift | 69 - .../Settings/AppDataReset.swift | 26 +- .../Settings/AppSettings.swift | 580 +----- .../Settings/ModelManagementView.swift | 181 -- .../TouchInput/KeyboardSwitcherTap.swift | 6 - Sources/ThreeFingerSwitcherApp/main.swift | 21 - .../AICommandCatalogTests.swift | 114 -- .../AICommandExecutorTests.swift | 954 ---------- .../AICommandFoldInTests.swift | 183 -- .../AICommandTests.swift | 245 --- .../AIErrorTests.swift | 126 -- .../AgentConversationTests.swift | 195 --- .../AppDataResetTests.swift | 31 +- .../AppSettingsTests.swift | 167 -- .../BackgroundAutonomyTests.swift | 376 ---- .../BackgroundToolRunnerTests.swift | 169 -- .../BatchedRuntimeTests.swift | 233 --- .../BidiTextDirectionTests.swift | 90 - .../CanvasResolveBindingTests.swift | 87 - .../ClaudeHandoffTests.swift | 411 ----- .../ComputeLaneTests.swift | 242 --- .../ComputerUseTests.swift | 241 --- .../ConversationCompactionTests.swift | 131 -- .../ConversationSessionTests.swift | 520 ------ .../ConversationalCanvasTests.swift | 249 --- .../EvictionPolicyTests.swift | 248 --- .../FileOpenServiceTests.swift | 335 ---- .../FilesActionMenuTests.swift | 160 -- .../FilesActionSettingsTests.swift | 63 - .../FilesBandBuilderTests.swift | 113 -- .../FilesColumnRoutingTests.swift | 413 ----- .../FilesDeliveryTests.swift | 34 - .../FilesDrillRecognizerTests.swift | 402 ----- .../FilesDwellArmingTests.swift | 63 - .../FilesNavigationModelTests.swift | 382 ---- .../FilesSeamsTests.swift | 176 -- .../FirstTouchWizardModelTests.swift | 5 +- .../FleetErrorTests.swift | 33 - .../FleetRosterTests.swift | 120 -- .../FullPotentialGateTests.swift | 155 -- .../FullPotentialSettingsTests.swift | 153 -- .../GestureBindingsTests.swift | 148 +- .../GesturePoseTests.swift | 17 +- .../GestureRecognizerLauncherTests.swift | 150 -- .../HubLauncherDemoTests.swift | 30 +- .../LLMRuntimeStubTests.swift | 296 ---- .../LaunchItemTests.swift | 37 +- .../LauncherCanvasModeTests.swift | 223 --- .../LauncherModelTests.swift | 37 - .../LocalImageGenerationTests.swift | 312 ---- .../MediaRuntimeTests.swift | 447 ----- .../MemoryTests.swift | 376 ---- .../ModelManagerFleetTests.swift | 162 -- .../ModelManagerTests.swift | 698 -------- .../ModelRegistryTests.swift | 78 - .../NotchFlickGestureTests.swift | 246 --- .../NotchTuningTests.swift | 36 - .../OpenWithEntriesTests.swift | 26 - .../PTTArmingModelTests.swift | 99 -- .../ParkedSessionsTests.swift | 1295 -------------- .../PromptTemplateTests.swift | 87 - .../RegionPickerModelTests.swift | 68 - .../ResidencyPlannerTests.swift | 126 -- .../ScrollConsumeRuleTests.swift | 44 +- .../SelectionServiceTests.swift | 405 ----- .../SkillsTests.swift | 513 ------ .../StubTernaryRuntimeTests.swift | 152 -- .../SubagentToolContributorTests.swift | 151 -- .../TaskDispatcherTests.swift | 479 ----- .../ToolRoutingTests.swift | 372 ---- .../VideoAnimationTests.swift | 418 ----- .../VoiceSessionControllerTests.swift | 165 -- .../VoiceTurnModelTests.swift | 195 --- .../WizardTourBandsTests.swift | 46 +- ...i-agent-v2-addendum-compute-media-fleet.md | 296 ---- docs/ai-agent-v2-blueprint.md | 479 ----- docs/ai-agent-v2-roadmap.md | 241 --- docs/ai-agent-v2.5-roadmap-addendum.md | 284 --- docs/notch-geometry-reference.md | 96 - .../specs/gesture-bindings/spec.md | 44 +- .../ai-action-context-resolution/design.md | 72 - .../ai-action-context-resolution/proposal.md | 33 - .../specs/ai-command-band/spec.md | 66 - .../specs/selection-io/spec.md | 16 - .../ai-action-context-resolution/tasks.md | 50 - .../changes/ai-agent-memory/.openspec.yaml | 2 - openspec/changes/ai-agent-memory/design.md | 327 ---- openspec/changes/ai-agent-memory/proposal.md | 95 - .../ai-agent-memory/specs/ai-memory/spec.md | 150 -- openspec/changes/ai-agent-memory/tasks.md | 45 - .../ai-background-autonomy/.openspec.yaml | 2 - .../changes/ai-background-autonomy/design.md | 271 --- .../ai-background-autonomy/proposal.md | 33 - .../specs/ai-background-autonomy/spec.md | 100 -- .../specs/configuration-hub/spec.md | 29 - .../changes/ai-background-autonomy/tasks.md | 52 - .../.openspec.yaml | 2 - .../ai-batched-runtime-and-context/design.md | 176 -- .../proposal.md | 32 - .../specs/on-device-ai-runtime/spec.md | 100 -- .../specs/tunable-settings/spec.md | 28 - .../ai-batched-runtime-and-context/tasks.md | 55 - .../changes/ai-claude-handoff/.openspec.yaml | 2 - openspec/changes/ai-claude-handoff/design.md | 274 --- .../changes/ai-claude-handoff/proposal.md | 28 - .../specs/ai-claude-handoff/spec.md | 113 -- openspec/changes/ai-claude-handoff/tasks.md | 44 - .../changes/ai-compute-tiers/.openspec.yaml | 2 - openspec/changes/ai-compute-tiers/design.md | 73 - openspec/changes/ai-compute-tiers/proposal.md | 34 - .../specs/on-device-ai-runtime/spec.md | 104 -- openspec/changes/ai-compute-tiers/tasks.md | 46 - .../ai-conversation-runtime/.openspec.yaml | 2 - .../changes/ai-conversation-runtime/design.md | 223 --- .../ai-conversation-runtime/proposal.md | 35 - .../specs/on-device-ai-runtime/spec.md | 127 -- .../changes/ai-conversation-runtime/tasks.md | 53 - .../ai-full-potential-toggle/.openspec.yaml | 2 - .../ai-full-potential-toggle/design.md | 133 -- .../ai-full-potential-toggle/proposal.md | 36 - .../specs/configuration-hub/spec.md | 41 - .../specs/tunable-settings/spec.md | 75 - .../changes/ai-full-potential-toggle/tasks.md | 38 - .../ai-local-image-generation/.openspec.yaml | 2 - .../ai-local-image-generation/design.md | 89 - .../ai-local-image-generation/proposal.md | 34 - .../specs/ai-generative-media/spec.md | 106 -- .../ai-local-image-generation/tasks.md | 39 - .../changes/ai-media-runtime/.openspec.yaml | 2 - openspec/changes/ai-media-runtime/design.md | 211 --- openspec/changes/ai-media-runtime/proposal.md | 133 -- .../specs/ai-command-tasks/spec.md | 33 - .../specs/ai-generative-media/spec.md | 254 --- openspec/changes/ai-media-runtime/tasks.md | 120 -- .../changes/ai-model-fleet/.openspec.yaml | 2 - openspec/changes/ai-model-fleet/design.md | 99 -- openspec/changes/ai-model-fleet/proposal.md | 35 - .../specs/configuration-hub/spec.md | 33 - .../specs/on-device-ai-runtime/spec.md | 69 - openspec/changes/ai-model-fleet/tasks.md | 51 - .../changes/ai-skills-as-files/.openspec.yaml | 2 - openspec/changes/ai-skills-as-files/design.md | 179 -- .../changes/ai-skills-as-files/proposal.md | 34 - .../specs/ai-command-catalog/spec.md | 78 - .../specs/ai-skill-index/spec.md | 38 - openspec/changes/ai-skills-as-files/tasks.md | 47 - .../changes/ai-tool-routing/.openspec.yaml | 2 - openspec/changes/ai-tool-routing/design.md | 244 --- openspec/changes/ai-tool-routing/proposal.md | 30 - .../specs/ai-command-tasks/spec.md | 109 -- openspec/changes/ai-tool-routing/tasks.md | 48 - .../.openspec.yaml | 2 - .../ai-video-animation-generation/design.md | 105 -- .../ai-video-animation-generation/proposal.md | 28 - .../specs/ai-generative-media/spec.md | 159 -- .../ai-video-animation-generation/tasks.md | 70 - .../canvas-bidi-and-selection-fixes/design.md | 37 - .../proposal.md | 36 - .../specs/launcher-overlay/spec.md | 45 - .../specs/selection-io/spec.md | 16 - .../canvas-bidi-and-selection-fixes/tasks.md | 31 - .../notch-timeline-and-tuning/.openspec.yaml | 2 - .../notch-timeline-and-tuning/design.md | 55 - .../notch-timeline-and-tuning/proposal.md | 33 - .../specs/ai-parked-sessions/spec.md | 118 -- .../notch-timeline-and-tuning/tasks.md | 37 - openspec/changes/remove-local-ai/design.md | 29 + openspec/changes/remove-local-ai/proposal.md | 31 + openspec/changes/remove-local-ai/tasks.md | 33 + openspec/explore/ai-command-band/design.md | 233 --- openspec/specs/ai-background-autonomy/spec.md | 119 -- openspec/specs/ai-command-band/spec.md | 168 -- openspec/specs/ai-command-catalog/spec.md | 72 - openspec/specs/ai-command-tasks/spec.md | 121 -- openspec/specs/ai-error-handling/spec.md | 107 -- openspec/specs/ai-parked-sessions/spec.md | 289 --- openspec/specs/ai-subagents/spec.md | 38 - openspec/specs/computer-use-tools/spec.md | 88 - openspec/specs/configuration-hub/spec.md | 104 +- openspec/specs/favorites-editor/spec.md | 27 +- openspec/specs/files-action-menu/spec.md | 204 --- openspec/specs/files-band/spec.md | 234 --- .../specs/files-contextual-delivery/spec.md | 75 - openspec/specs/first-run-onboarding/spec.md | 6 +- openspec/specs/gesture-recognition/spec.md | 41 - openspec/specs/launch-actions/spec.md | 11 - openspec/specs/launcher-overlay/spec.md | 252 --- openspec/specs/on-device-ai-runtime/spec.md | 319 ---- openspec/specs/permissions-onboarding/spec.md | 44 - openspec/specs/screen-region-picker/spec.md | 50 - openspec/specs/selection-io/spec.md | 84 - openspec/specs/tunable-settings/spec.md | 64 - openspec/specs/voice-conversation/spec.md | 88 - scripts/build-app.sh | 23 +- 397 files changed, 456 insertions(+), 59407 deletions(-) delete mode 100644 Sources/GemmaRuntime/BatchedGemmaMLXRuntime.swift delete mode 100644 Sources/GemmaRuntime/CloudVideoRuntime.swift delete mode 100644 Sources/GemmaRuntime/GemmaMLXRuntime.swift delete mode 100644 Sources/GemmaRuntime/GemmaResumableDownloader.swift delete mode 100644 Sources/GemmaRuntime/GemmaRuntime.swift delete mode 100644 Sources/GemmaRuntime/LaneDispatch.swift delete mode 100644 Sources/GemmaRuntime/LocalLTXVRuntime.swift delete mode 100644 Sources/GemmaRuntime/MFluxImageRuntime.swift delete mode 100644 Sources/GemmaRuntime/TernaryCPURuntime.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/AIBand.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/AICommand.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/AICommandCatalog.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/AICommandExecutor.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/AIError.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/AIRuntimeInjection.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Agent/AgentContextBudgetProvider.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Agent/AgentContextCostModel.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Agent/AgentContextPreset.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Agent/AgentConversation.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Agent/AgentLoop.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Agent/AutoApprovingGate.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Agent/CanvasApprovalGate.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Agent/ChatTemplate.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Agent/ConcurrencyBudget.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Agent/ContextBudget.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Agent/ConversationCompactor.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Agent/RouteSchema.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Agent/Subagent.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Agent/SubagentToolContributor.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Agent/TaskKindToolContributor.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Agent/ToolCandidateSource.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Agent/ToolContracts.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Agent/ToolRegistry.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Agent/ToolRouter.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Agent/WritePolicyResolving.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Audit/AuditError.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Audit/AuditLog.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Audit/AuditRecord.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Audit/BackgroundGate.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Audit/BackgroundToolRunner.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Audit/BlastRadius.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Audit/Whitelist.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Audit/WritePolicyResolution.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Ax/AXActionError.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Ax/AXActionPerformer.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Ax/AXWindowReader.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Ax/AXWindowSnapshot.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Ax/AgentActionArbiter.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Ax/ComputerUseToolContributor.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Compute/ComputeLane.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Compute/GatedLaneRouting.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Compute/LaneArbiter.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Compute/LaneResidencyBudget.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Compute/StubTernaryRuntime.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/DevAIRuntime.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/EvictionPolicy.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Fleet/FleetCloudEscalation.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Fleet/FleetError.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Fleet/FleetModelRegistry.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Fleet/FleetRoster.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Fleet/ResidencyPlanner.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/ForwardingLLMRuntime.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/FullPotential/AIGatingSnapshot.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/FullPotential/FullPotentialGate.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Handoff/ClaudeHandoffContributor.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Handoff/HandoffBudget.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Handoff/HandoffError.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Handoff/HandoffLauncher.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Handoff/HandoffSeams.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/LLMRuntime.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/Image/ImageCostDisclosure.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/Image/ImageModelCatalog.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/Image/ImageRequestValidation.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/Image/ImageResidencyClass.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/Image/StubImageRuntime.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/MediaArgs.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/MediaCanvasResolve.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/MediaError.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/MediaGallery.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/MediaGenSink.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/MediaJobState.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/MediaParkFeed.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/MediaResidency.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/MediaRuntime.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/MediaSeed.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/MediaToolContributor.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/MediaVideoBudget.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/StubMediaRuntime.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/Video/StubVideoRuntimes.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/Video/VideoBudget.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/Video/VideoProvider.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/Video/VideoTierResolution.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Media/Video/VideoUploadDisclosure.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Memory/MemoryAudit.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Memory/MemoryDocument.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Memory/MemoryError.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Memory/MemoryReconcile.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Memory/MemoryStore.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Memory/MemorySubfile.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Memory/MemoryToolProvider.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/MemoryPressureSource.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/ModelManager.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/ModelRegistry.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Parked/NotchHomeZoneAnchor.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Parked/NotchRevealModel.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Parked/NotchSessionEngine.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Parked/NotchTuning.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Parked/ParkError.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Parked/ParkLifecycle.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Parked/ParkLifecycleCoordinator.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Parked/ParkScheduler.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Parked/ParkedSession.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Parked/ParkedSessionStore.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/PromptTemplate.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/RuntimeParameter.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/SelectionProviding.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/SelectionService.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Skills/ClaudeHandoffConfig.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Skills/DocIndex.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Skills/SkillError.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Skills/SkillFile.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Skills/SkillFolderWatcher.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Skills/SkillManifest.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Skills/SkillStore.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Skills/SkillToolProvider.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/StubLLMRuntime.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/TaskDispatching.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Tasks/ParsedActions.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Tasks/TaskDispatcher.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Tasks/TaskReview.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Tasks/TaskSinks.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Voice/PTTArmingModel.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Voice/PTTKeyMonitor.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Voice/SentenceChunker.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Voice/SpeechAnalyzerTranscriber.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Voice/SpeechSeams.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Voice/SystemSpeechSynthesizer.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Voice/VoiceError.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Voice/VoiceSessionController.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Voice/VoiceToolContributor.swift delete mode 100644 Sources/ThreeFingerSwitcher/AI/Voice/VoiceTurnModel.swift delete mode 100644 Sources/ThreeFingerSwitcher/App/ParkController.swift delete mode 100644 Sources/ThreeFingerSwitcher/Files/DirectoryLister.swift delete mode 100644 Sources/ThreeFingerSwitcher/Files/FileEntry.swift delete mode 100644 Sources/ThreeFingerSwitcher/Files/FileOpenService.swift delete mode 100644 Sources/ThreeFingerSwitcher/Files/FileWorkspace.swift delete mode 100644 Sources/ThreeFingerSwitcher/Files/FilesActionMenu.swift delete mode 100644 Sources/ThreeFingerSwitcher/Files/FilesBandBuilder.swift delete mode 100644 Sources/ThreeFingerSwitcher/Files/FilesColumnController.swift delete mode 100644 Sources/ThreeFingerSwitcher/Files/FilesDelivery.swift delete mode 100644 Sources/ThreeFingerSwitcher/Files/FilesNavigationModel.swift delete mode 100644 Sources/ThreeFingerSwitcher/Files/FilesToolCatalog.swift delete mode 100644 Sources/ThreeFingerSwitcher/Files/OpenWithEntry.swift delete mode 100644 Sources/ThreeFingerSwitcher/Gesture/FlickExcursionClassifier.swift delete mode 100644 Sources/ThreeFingerSwitcher/Hub/HubBackgroundAutonomy.swift delete mode 100644 Sources/ThreeFingerSwitcher/Hub/HubFilesPage.swift delete mode 100644 Sources/ThreeFingerSwitcher/Hub/HubFleetRosterView.swift delete mode 100644 Sources/ThreeFingerSwitcher/Overlay/AICommandCanvasView.swift delete mode 100644 Sources/ThreeFingerSwitcher/Overlay/BidiText.swift delete mode 100644 Sources/ThreeFingerSwitcher/Overlay/BubbleMorph.swift delete mode 100644 Sources/ThreeFingerSwitcher/Overlay/FilesBandView.swift delete mode 100644 Sources/ThreeFingerSwitcher/Overlay/FilesDwellArming.swift delete mode 100644 Sources/ThreeFingerSwitcher/Overlay/MediaCanvasPlayerOverlay.swift delete mode 100644 Sources/ThreeFingerSwitcher/Overlay/NotchHomeZoneController.swift delete mode 100644 Sources/ThreeFingerSwitcher/Overlay/NotchHomeZoneOverlay.swift delete mode 100644 Sources/ThreeFingerSwitcher/Overlay/RegionPickerOverlay.swift delete mode 100644 Sources/ThreeFingerSwitcher/RegionPicker/RegionPickerModel.swift delete mode 100644 Sources/ThreeFingerSwitcher/Settings/ModelManagementView.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/AICommandCatalogTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/AICommandExecutorTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/AICommandFoldInTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/AICommandTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/AIErrorTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/AgentConversationTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/BackgroundAutonomyTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/BackgroundToolRunnerTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/BatchedRuntimeTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/BidiTextDirectionTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/CanvasResolveBindingTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/ClaudeHandoffTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/ComputeLaneTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/ComputerUseTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/ConversationCompactionTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/ConversationSessionTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/ConversationalCanvasTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/EvictionPolicyTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/FileOpenServiceTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/FilesActionMenuTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/FilesActionSettingsTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/FilesBandBuilderTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/FilesColumnRoutingTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/FilesDeliveryTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/FilesDrillRecognizerTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/FilesDwellArmingTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/FilesNavigationModelTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/FilesSeamsTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/FleetErrorTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/FleetRosterTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/FullPotentialGateTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/FullPotentialSettingsTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/LLMRuntimeStubTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/LauncherCanvasModeTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/LocalImageGenerationTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/MediaRuntimeTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/MemoryTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/ModelManagerFleetTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/ModelManagerTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/ModelRegistryTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/NotchFlickGestureTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/NotchTuningTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/OpenWithEntriesTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/PTTArmingModelTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/ParkedSessionsTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/PromptTemplateTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/RegionPickerModelTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/ResidencyPlannerTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/SelectionServiceTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/SkillsTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/StubTernaryRuntimeTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/SubagentToolContributorTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/TaskDispatcherTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/ToolRoutingTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/VideoAnimationTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/VoiceSessionControllerTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/VoiceTurnModelTests.swift delete mode 100644 docs/ai-agent-v2-addendum-compute-media-fleet.md delete mode 100644 docs/ai-agent-v2-blueprint.md delete mode 100644 docs/ai-agent-v2-roadmap.md delete mode 100644 docs/ai-agent-v2.5-roadmap-addendum.md delete mode 100644 docs/notch-geometry-reference.md delete mode 100644 openspec/changes/ai-action-context-resolution/design.md delete mode 100644 openspec/changes/ai-action-context-resolution/proposal.md delete mode 100644 openspec/changes/ai-action-context-resolution/specs/ai-command-band/spec.md delete mode 100644 openspec/changes/ai-action-context-resolution/specs/selection-io/spec.md delete mode 100644 openspec/changes/ai-action-context-resolution/tasks.md delete mode 100644 openspec/changes/ai-agent-memory/.openspec.yaml delete mode 100644 openspec/changes/ai-agent-memory/design.md delete mode 100644 openspec/changes/ai-agent-memory/proposal.md delete mode 100644 openspec/changes/ai-agent-memory/specs/ai-memory/spec.md delete mode 100644 openspec/changes/ai-agent-memory/tasks.md delete mode 100644 openspec/changes/ai-background-autonomy/.openspec.yaml delete mode 100644 openspec/changes/ai-background-autonomy/design.md delete mode 100644 openspec/changes/ai-background-autonomy/proposal.md delete mode 100644 openspec/changes/ai-background-autonomy/specs/ai-background-autonomy/spec.md delete mode 100644 openspec/changes/ai-background-autonomy/specs/configuration-hub/spec.md delete mode 100644 openspec/changes/ai-background-autonomy/tasks.md delete mode 100644 openspec/changes/ai-batched-runtime-and-context/.openspec.yaml delete mode 100644 openspec/changes/ai-batched-runtime-and-context/design.md delete mode 100644 openspec/changes/ai-batched-runtime-and-context/proposal.md delete mode 100644 openspec/changes/ai-batched-runtime-and-context/specs/on-device-ai-runtime/spec.md delete mode 100644 openspec/changes/ai-batched-runtime-and-context/specs/tunable-settings/spec.md delete mode 100644 openspec/changes/ai-batched-runtime-and-context/tasks.md delete mode 100644 openspec/changes/ai-claude-handoff/.openspec.yaml delete mode 100644 openspec/changes/ai-claude-handoff/design.md delete mode 100644 openspec/changes/ai-claude-handoff/proposal.md delete mode 100644 openspec/changes/ai-claude-handoff/specs/ai-claude-handoff/spec.md delete mode 100644 openspec/changes/ai-claude-handoff/tasks.md delete mode 100644 openspec/changes/ai-compute-tiers/.openspec.yaml delete mode 100644 openspec/changes/ai-compute-tiers/design.md delete mode 100644 openspec/changes/ai-compute-tiers/proposal.md delete mode 100644 openspec/changes/ai-compute-tiers/specs/on-device-ai-runtime/spec.md delete mode 100644 openspec/changes/ai-compute-tiers/tasks.md delete mode 100644 openspec/changes/ai-conversation-runtime/.openspec.yaml delete mode 100644 openspec/changes/ai-conversation-runtime/design.md delete mode 100644 openspec/changes/ai-conversation-runtime/proposal.md delete mode 100644 openspec/changes/ai-conversation-runtime/specs/on-device-ai-runtime/spec.md delete mode 100644 openspec/changes/ai-conversation-runtime/tasks.md delete mode 100644 openspec/changes/ai-full-potential-toggle/.openspec.yaml delete mode 100644 openspec/changes/ai-full-potential-toggle/design.md delete mode 100644 openspec/changes/ai-full-potential-toggle/proposal.md delete mode 100644 openspec/changes/ai-full-potential-toggle/specs/configuration-hub/spec.md delete mode 100644 openspec/changes/ai-full-potential-toggle/specs/tunable-settings/spec.md delete mode 100644 openspec/changes/ai-full-potential-toggle/tasks.md delete mode 100644 openspec/changes/ai-local-image-generation/.openspec.yaml delete mode 100644 openspec/changes/ai-local-image-generation/design.md delete mode 100644 openspec/changes/ai-local-image-generation/proposal.md delete mode 100644 openspec/changes/ai-local-image-generation/specs/ai-generative-media/spec.md delete mode 100644 openspec/changes/ai-local-image-generation/tasks.md delete mode 100644 openspec/changes/ai-media-runtime/.openspec.yaml delete mode 100644 openspec/changes/ai-media-runtime/design.md delete mode 100644 openspec/changes/ai-media-runtime/proposal.md delete mode 100644 openspec/changes/ai-media-runtime/specs/ai-command-tasks/spec.md delete mode 100644 openspec/changes/ai-media-runtime/specs/ai-generative-media/spec.md delete mode 100644 openspec/changes/ai-media-runtime/tasks.md delete mode 100644 openspec/changes/ai-model-fleet/.openspec.yaml delete mode 100644 openspec/changes/ai-model-fleet/design.md delete mode 100644 openspec/changes/ai-model-fleet/proposal.md delete mode 100644 openspec/changes/ai-model-fleet/specs/configuration-hub/spec.md delete mode 100644 openspec/changes/ai-model-fleet/specs/on-device-ai-runtime/spec.md delete mode 100644 openspec/changes/ai-model-fleet/tasks.md delete mode 100644 openspec/changes/ai-skills-as-files/.openspec.yaml delete mode 100644 openspec/changes/ai-skills-as-files/design.md delete mode 100644 openspec/changes/ai-skills-as-files/proposal.md delete mode 100644 openspec/changes/ai-skills-as-files/specs/ai-command-catalog/spec.md delete mode 100644 openspec/changes/ai-skills-as-files/specs/ai-skill-index/spec.md delete mode 100644 openspec/changes/ai-skills-as-files/tasks.md delete mode 100644 openspec/changes/ai-tool-routing/.openspec.yaml delete mode 100644 openspec/changes/ai-tool-routing/design.md delete mode 100644 openspec/changes/ai-tool-routing/proposal.md delete mode 100644 openspec/changes/ai-tool-routing/specs/ai-command-tasks/spec.md delete mode 100644 openspec/changes/ai-tool-routing/tasks.md delete mode 100644 openspec/changes/ai-video-animation-generation/.openspec.yaml delete mode 100644 openspec/changes/ai-video-animation-generation/design.md delete mode 100644 openspec/changes/ai-video-animation-generation/proposal.md delete mode 100644 openspec/changes/ai-video-animation-generation/specs/ai-generative-media/spec.md delete mode 100644 openspec/changes/ai-video-animation-generation/tasks.md delete mode 100644 openspec/changes/canvas-bidi-and-selection-fixes/design.md delete mode 100644 openspec/changes/canvas-bidi-and-selection-fixes/proposal.md delete mode 100644 openspec/changes/canvas-bidi-and-selection-fixes/specs/launcher-overlay/spec.md delete mode 100644 openspec/changes/canvas-bidi-and-selection-fixes/specs/selection-io/spec.md delete mode 100644 openspec/changes/canvas-bidi-and-selection-fixes/tasks.md delete mode 100644 openspec/changes/notch-timeline-and-tuning/.openspec.yaml delete mode 100644 openspec/changes/notch-timeline-and-tuning/design.md delete mode 100644 openspec/changes/notch-timeline-and-tuning/proposal.md delete mode 100644 openspec/changes/notch-timeline-and-tuning/specs/ai-parked-sessions/spec.md delete mode 100644 openspec/changes/notch-timeline-and-tuning/tasks.md create mode 100644 openspec/changes/remove-local-ai/design.md create mode 100644 openspec/changes/remove-local-ai/proposal.md create mode 100644 openspec/changes/remove-local-ai/tasks.md delete mode 100644 openspec/explore/ai-command-band/design.md delete mode 100644 openspec/specs/ai-background-autonomy/spec.md delete mode 100644 openspec/specs/ai-command-band/spec.md delete mode 100644 openspec/specs/ai-command-catalog/spec.md delete mode 100644 openspec/specs/ai-command-tasks/spec.md delete mode 100644 openspec/specs/ai-error-handling/spec.md delete mode 100644 openspec/specs/ai-parked-sessions/spec.md delete mode 100644 openspec/specs/ai-subagents/spec.md delete mode 100644 openspec/specs/computer-use-tools/spec.md delete mode 100644 openspec/specs/files-action-menu/spec.md delete mode 100644 openspec/specs/files-band/spec.md delete mode 100644 openspec/specs/files-contextual-delivery/spec.md delete mode 100644 openspec/specs/on-device-ai-runtime/spec.md delete mode 100644 openspec/specs/screen-region-picker/spec.md delete mode 100644 openspec/specs/selection-io/spec.md delete mode 100644 openspec/specs/voice-conversation/spec.md diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index a32228f..94671b8 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -23,10 +23,6 @@ permissions: jobs: release: - # NOTE: the MLX/Gemma 4 dependency (gemma-4-swift-mlx → mlx-swift-lm) only compiles on a recent - # toolchain: Xcode 26.3 (Swift 6.3.x) FAILS to type-check it (LoRA/TurboQuant), while Xcode 26.5 - # (Swift 6.3.2) builds it cleanly. macos-15 tops out at Xcode 26.3, so the release runs on macos-26 - # (which ships 26.5) and the step below selects 26.5 explicitly. runs-on: macos-26 env: PRODUCT: ThreeFingerSwitcher @@ -36,11 +32,8 @@ jobs: - name: Select Xcode and gate on toolchain (fail fast) run: | set -euo pipefail - # Prefer the verified-working Xcode 26.5 (Swift 6.3.2 — the toolchain that compiles the - # MLX/Gemma deps; 26.3 does not). Fall back to the newest installed if that exact version is - # ever removed from the image; the Swift gate below still guards against a too-old toolchain. - XC="/Applications/Xcode_26.5.app" - [ -d "$XC" ] || XC="$(ls -d /Applications/Xcode*.app | sort -V | tail -1)" + # Use the newest installed Xcode; the Swift gate below guards against a too-old toolchain. + XC="$(ls -d /Applications/Xcode*.app | sort -V | tail -1)" echo "Using Xcode at: $XC" sudo xcode-select -s "$XC/Contents/Developer" swift --version diff --git a/CLAUDE.md b/CLAUDE.md index 12ad015..a6209d0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -2,13 +2,15 @@ Orientation lives in **`README.md`** (it's written for an agent: Job A = install/run, Job B = work on the code). This file is the short list of things that are easy to get wrong. Read the **Building & signing** rule before you build anything. +> **The great cleanup (`remove-local-ai`):** the on-device AI stack (Gemma/MLX, the AI command band + canvas, background agents, the notch timeline, voice/computer-use) and the Files band were **removed** — the app is refocused on the **switcher, the launcher, and clipboard history**. The `v1` branch / `v1.0.0` release preserve the full-featured app; don't reintroduce those features casually. + ## Building & signing — read this first **Do not assemble or install the `.app` from the agent's shell.** The sandboxed shell has **no keychain access**, so `scripts/build-app.sh` falls back to **ad-hoc signing**. Ad-hoc signing changes the app's code identity (CDHash) on every build, which **silently invalidates the macOS TCC permission grants** the app depends on — Accessibility, Input Monitoring, Screen Recording. The result: the app launches but quietly does nothing (no gesture capture, no thumbnails), which looks like a bug but is really a broken signature. An agent-built `.app` also **collides with the user's own stable-signed install** at the same path. So the division of labor is: -- **Agent does:** edit code, and verify with **`swift build`** / **`swift test`** (the MLX-free `ThreeFingerSwitcherCore` + test target) and, for the MLX-linked `GemmaRuntime`/app target, **`xcodebuild`** to *compile-verify only*. These compile and run logic — they don't sign, install, or launch the app, so they're safe and useful. To compile-check a *subset* of the tree in isolation (e.g. one feature without another's uncommitted files), use a throwaway **`git worktree`** and `swift build` there — never the shared working tree's `.app`. +- **Agent does:** edit code, and verify with **`swift build`** / **`swift test`** (the `ThreeFingerSwitcherCore` library + test target hold ALL app logic). These compile and run logic — they don't sign, install, or launch the app, so they're safe and useful. To compile-check a *subset* of the tree in isolation (e.g. one feature without another's uncommitted files), use a throwaway **`git worktree`** and `swift build` there — never the shared working tree's `.app`. - **User does (in their own Terminal):** the real build for any in-app or permission testing — ```bash INSTALL=1 ./scripts/build-app.sh # stable-signed, installed in place to /Applications @@ -18,63 +20,43 @@ So the division of labor is: **Releases are never built locally.** Pushing a `vX.Y.Z` git tag triggers `.github/workflows/release.yml`, which builds, **Developer-ID-signs + notarizes + staples**, and publishes a DMG to GitHub Releases (see `docs/RELEASING.md`). Don't try to notarize or Developer-ID-sign from the agent shell — that's the CI runner's job, and it has the secrets. -## On-device AI (the AI Command Band) — build & landmines +## Error handling — one taxonomy per domain, mapped at the boundary, surfaced bounded + non-blocking -The AI band runs **Gemma 4 in-process via MLX**. Two targets, on purpose: **`ThreeFingerSwitcherCore` stays MLX-free** (the `LLMRuntime` seam + a `StubLLMRuntime`/`DevAIRuntime`, the executor, tasks, selection, canvas — all verify under `swift build`/`swift test`); the real model lives in **`GemmaRuntime`**, which links MLX and therefore builds via **`xcodebuild` only** (MLX compiles Metal shaders — one-time `xcodebuild -downloadComponent MetalToolchain`). The app injects the real runtime at the seam in `main.swift`. +New failure-surfacing code inherits this convention (established by the archived `harden-ai-error-handling` change and kept after the AI removal): -- **The metallib landmine:** MLX ships `default.metallib` as a SwiftPM resource bundle (`mlx-swift_Cmlx.bundle`). `build-app.sh` **must copy `*.bundle` into `Contents/Resources/`** — if it doesn't, the app launches but is **SIGKILL'd at first GPU use with no crash report**. This is already handled; don't regress it. -- **Errors: one taxonomy, one translator, mapped at the boundary, surfaced bounded + non-blocking.** New AI code inherits this convention (see the `harden-ai-error-handling` change): - - **One taxonomy:** classify every AI failure into the shared `RuntimeError` (Core), which conforms to `LocalizedError` with a clean per-case string. **One translator:** `AIError.message(for:) -> AIPresentedError` (a clean `headline` + opt-in copyable `details`) is the SINGLE error→message function — every surface (Settings row, canvas, any alert) routes through it, so the same error reads identically everywhere. - - **Ban raw interpolation in UI strings:** never put `"\(error)"` / `String(describing: error)` / an OS error's `.localizedDescription` into a headline. Raw text is allowed only in logs and as `AIPresentedError.details`. - - **Map at the layer boundary:** convert vendor/OS errors (`Gemma4DownloadError`, `NSURLError`, EventKit, `FileManager`/`Process`) into the taxonomy where they cross into app code (e.g. `GemmaMLXRuntime.prepare`, the sinks) — Core stays MLX-free, so it can't see vendor types; only `RuntimeError`/`TaskError` cross into feature/UI code. - - **Failure is observable state, never silence:** a failure transitions to a `.failed` state carrying the clean headline (never leaves a state stuck mid-flight; cancellation is not a failure). A side effect that didn't land (write/open/paste/task) becomes `.failed`, never a false "Done." - - **Bounded + non-blocking UI:** never surface a background AI failure via app-modal `NSAlert.runModal()` (it freezes the Settings window) — use the in-window `.failed` row + Retry. Cap long messages (`.lineLimit` + `.truncationMode(.middle)`), put raw text behind a "Show details / Copy" disclosure, and keep layouts scroll-safe. -- **Swipe-to-resolve, not lift-to-commit:** while the preview canvas is open it's resolved by a *fresh **two-finger** swipe* — **down = commit/apply, horizontal = discard, up = ignored**; a stray re-lift is a no-op (the firing lift already raised the fingers). *(Changed from four-finger by `positional-navigation`: the grammar is now **4 fingers open/dismiss the platform, 2 fingers act within it**. The resolve excursion threshold sits **above incidental two-finger scroll** — `canvasResolveThreshold` — so reading the canvas never resolves it.)* -- **Vision is supported** (Gemma 4 is multimodal via MLX; the `LLMRuntime` seam carries an optional `image:` and the registry advertises `.vision`). An image input comes from a **captured screen region** (`.screenRegion`) or the **live clipboard image** (`.clipboardImage`, on-demand — copying an image never auto-fires); both statically require a vision-capable model. A `clipboardImage` read uses the live pasteboard (symmetric with the clipboard-text read, normalized to PNG), not `ClipboardStore`. The model is **Apple-Silicon-only** (no Intel/low-end fallback). The seam exists so another backend can replace Gemma without touching feature code. *(The interactive screen-region picker — drag a rectangle instead of grabbing the whole display — is the `add-region-capture-picker` change.)* +- **One taxonomy per domain** (e.g. `DockPreviewError`, `ClaudeLaunchError`): a small Core `LocalizedError` enum with a clean per-case string. +- **Ban raw interpolation in UI strings:** never put `"\(error)"` / `String(describing: error)` / an OS error's `.localizedDescription` into a headline. Raw text is allowed only in logs and as opt-in copyable details. +- **Map at the layer boundary:** convert vendor/OS errors (`NSURLError`, `FileManager`/`Process`) into the domain taxonomy where they cross into app code; only typed errors cross into feature/UI code. +- **Failure is observable state, never silence:** a failure transitions to a `.failed` state carrying the clean headline (never leaves a state stuck mid-flight; cancellation is not a failure). A side effect that didn't land becomes `.failed`, never a false "Done." +- **Bounded + non-blocking UI:** never surface a background failure via app-modal `NSAlert.runModal()` (it freezes the window) — use an in-window `.failed` row/card + Retry. Cap long messages (`.lineLimit` + `.truncationMode(.middle)`), put raw text behind a "Show details / Copy" disclosure, and keep layouts scroll-safe. ## SwiftUI "liveness" animations — the idle-CPU-spin landmine -A repeating `TimelineView(.periodic)` / "breathing" animation hosted in an `NSHostingView` **keeps ticking after its window is hidden** — `orderOut` does NOT stop the SwiftUI animation clock, and neither does wrapping it in `.opacity(0)` or `if isActive { … }`. In a window kept alive by **`isReleasedWhenClosed = false`** (the Hub, the first-touch wizard) or an autonomously-shown panel (the now-removed notch needs-you glow), each tick drives a **non-converging Auto-Layout ⇄ render ⇄ Observation loop** that **pins the main thread at ~100% forever** — which starves the main-thread gesture→switcher path and reads as "the switcher is slow after a break," clearing only on restart. Full repro + stack: **`docs/postmortem-idle-cpu-spin.md`**. **Rule:** gate every repeating animation on **real window visibility** (`NSWindow.occlusionState` / `didChangeOcclusionStateNotification`, or an explicit active flag the controller sets on show/hide) — **never** on `onAppear`/`onDisappear` (they don't fire for a hidden-but-retained window) — or release the hosting controller on close. The AI-canvas "Thinking" pulse and the notch needs-you glow were **removed** for this reason; the Hub gesture preview was reduced to a **visibility-gated autoplay** (live finger-tracking + the free-running model driver deleted — the Switcher AND Launcher/band miniatures now follow the ghost hand only via the preview's **clockless sync seam**, `GhostSyncPose` frames emitted from inside the gated `TimelineView`, so it owns no timer and a hidden Hub drives nothing; don't "upgrade" it back to a self-ticking driver); the onboarding wizard (`Onboarding/WizardMotion.swift`) uses the same breathers — first-run-only, but the same pattern. - -## The Files band (the in-launcher Finder) — build & landmines - -The Files band (`files-band`) is a **local-only Finder** that lives as a synthetic launcher band: land on it and you're in a bounded **column navigator** (icon-rail ancestors + current folder list + live preview). It's all in **MLX-free Core** (`Sources/ThreeFingerSwitcher/Files/` + `Overlay/FilesBandView.swift` + `Overlay/BubbleMorph.swift` + `Hub/HubFilesPage.swift`), so it verifies under `swift build` / `swift test`. Opt-in, default off; no gesture relocation, no re-login, no new permission (it reads the filesystem on demand — like `keepClipboardHistory`). - -- **Synthetic band, NOT a band-type enum.** Like the Clipboard band, it's recognized by a sentinel `FilesBandBuilder.bandID` + a threaded `filesBandIndex`, switched on `LauncherModel.currentBandIsFiles`. Its items are an **ephemeral `.fileEntry(FileEntry)`** kind (never persisted, like `.clipboardEntry` — it *is* `Codable` only because the enclosing `LaunchItemKind` is); the band is appended to a local copy in `AppCoordinator.launcherDidActivate()` and **never written to `FavoritesStore`**. `FileEntry.id` is the standardized path; `FilesBandBuilder` derives a deterministic UUID from it for `LaunchItem.id` (stable across re-lists → no highlight strobe). -- **Lift-to-open, NOT swipe-to-resolve.** Resolution is: **lift opens** the highlighted item (file → default app, folder → Finder window) on the current Space; **+1-finger lift** opens the **Open-With picker** (a scrubbable popup of the apps that can open it); a **four-finger horizontal swipe discards** (defuses a *pending* open — it **never terminates a running app**). The AI canvas's swipe-to-resolve (above) exists to let you *review a generated result*; do **not** generalize it to this navigation surface — the spec was corrected after the implementation surfaced that drift. -- **The drill modal sub-state.** While the navigator is open, `GestureRecognizer.filesDrillActive` routes `feed()` to `trackFilesDrill` (a second early short-circuit, mirroring `launcherCanvasResolutionActive`) — finger-count deltas are re-interpreted there. **Open-With is a *relative* +1 finger** (`count > drillContacts`), not an absolute three (you may be holding three the whole time). **Re-baseline the origin on every contact-count change** or a leaving finger fires a phantom step. Resolution is one-shot (`drillResolved`); **any handler that leaves the navigator open must call `recognizer.rearmDrill()`** or navigation goes inert (this is the non-obvious bit — the Open-With picker and the picker-back-out both re-arm). -- **No search — the Files band is pure-trackpad, no keypresses.** Type-to-filter search was removed (it broke the no-keypress model and went unused); an up-step at the top of the column simply **clamps**. There is NO search field, no `@FocusState`, and the overlay panel **never becomes key/main** for the Files band (unlike the AI canvas) — don't reintroduce a key-interactive flip here. `FilesNavigationModel.visibleEntries` is retained as the column read-seam but is always the unfiltered `entries`. -- **The remember-last-folder toggle gates restore at BOTH init and `enterRoot`.** `FilesNavigationModel.restoreLastLocation` is a stored property: when OFF the band opens on the roots list AND descending into a root lands on the root's **top level** (never the remembered deep folder); when ON it opens displaying / re-enters the remembered location. Deepest-location *tracking* runs regardless of the flag (so flipping it ON later restores). The bug to avoid: gating only the init landing — that let `enterRoot` jump to the last-visited folder even with the toggle off (a visual/state desync). -- **The sync/async cache seam.** `FilesNavigationModel` is a **pure, synchronous** state machine; the real `DirectoryLister` is **async / off-main**. `FilesColumnController` bridges them with a listing **cache**: the model's sync lister reads `cache[path] ?? []`; a miss spawns a coalesced off-main listing that stores, re-feeds the column, and republishes. The folder-peek preview flows through the same cache. **Don't make the pure model touch `FileManager`.** -- **BubbleMorph — the first spring, on containers only.** `Overlay/BubbleMorph.swift` (scale-from-0.02 + opacity on a soft spring) animates columns / rows / preview / menus; the **single sliding `FilesRowHighlight` is NOT bubble-morphed** (per-row morphs reintroduce the documented scrub strobe). Depth uses the `SwitcherView` `.id`/`.transition` idiom but **scaling, not sliding**. Don't retrofit the charge ramp / arm snap to a spring, and add **no new haptics**. -- **Errors map at the boundary into `FileActionError`** (a Core `LocalizedError` taxonomy parallel to `RuntimeError`); a failed open surfaces as a **bounded, non-blocking** card (clean headline + opt-in copyable details + Retry/Dismiss) over the navigator — **never** an `NSAlert`, never raw error text in a headline. -- **Local-only, navigate-and-open only (v1).** No file ops (move/rename/delete/tag), no iCloud/network. The `FileWorkspace`/provider seam is built so other providers could come later. -- **Deferred (don't "fix" casually):** the *whole-panel* recede-on-leave teardown. Per-element transitions recede inside the live panel, but the panel's `orderOut`+`close` stays **synchronous** — deferring it behind an exit animation re-opens the documented **ghost-on-Space-switch** bug. If you wire a receding exit, you must NOT break the synchronous teardown for a Space-switching open. +A repeating `TimelineView(.periodic)` / "breathing" animation hosted in an `NSHostingView` **keeps ticking after its window is hidden** — `orderOut` does NOT stop the SwiftUI animation clock, and neither does wrapping it in `.opacity(0)` or `if isActive { … }`. In a window kept alive by **`isReleasedWhenClosed = false`** (the Hub, the first-touch wizard), each tick drives a **non-converging Auto-Layout ⇄ render ⇄ Observation loop** that **pins the main thread at ~100% forever** — which starves the main-thread gesture→switcher path and reads as "the switcher is slow after a break," clearing only on restart. Full repro + stack: **`docs/postmortem-idle-cpu-spin.md`**. **Rule:** gate every repeating animation on **real window visibility** (`NSWindow.occlusionState` / `didChangeOcclusionStateNotification`, or an explicit active flag the controller sets on show/hide) — **never** on `onAppear`/`onDisappear` (they don't fire for a hidden-but-retained window) — or release the hosting controller on close. The Hub gesture preview is a **visibility-gated autoplay** (live finger-tracking + the free-running model driver deleted — the Switcher AND Launcher/band miniatures follow the ghost hand only via the preview's **clockless sync seam**, `GhostSyncPose` frames emitted from inside the gated `TimelineView`, so it owns no timer and a hidden Hub drives nothing; don't "upgrade" it back to a self-ticking driver); the onboarding wizard (`Onboarding/WizardMotion.swift`) uses the same breathers — first-run-only, but the same pattern. ## The Dock window previews (the switcher "from another angle") — build & landmines -The Dock-preview feature (`dock-window-previews`) is the switcher reached by **mouse**, not trackpad: hover an app's Dock tile → a row of that app's **current-Space** windows (normal **and minimized**) pops above the tile; hover a thumbnail to **peek its live content**; click to raise it. Opt-in, default off (`showDockPreviews`); no gesture relocation, no re-login, **no new permission** (reuses the already-granted Accessibility + Screen Recording). All of it is MLX-free Core (`Sources/ThreeFingerSwitcher/Dock/` + `Overlay/DockPreviewOverlay.swift`), so it verifies under `swift build`/`swift test`; only the live AX/cursor behavior needs the real app. +The Dock-preview feature (`dock-window-previews`) is the switcher reached by **mouse**, not trackpad: hover an app's Dock tile → a row of that app's **current-Space** windows (normal **and minimized**) pops above the tile; hover a thumbnail to **peek its live content**; click to raise it. Opt-in, default off (`showDockPreviews`); no gesture relocation, no re-login, **no new permission** (reuses the already-granted Accessibility + Screen Recording). All of it is Core (`Sources/ThreeFingerSwitcher/Dock/` + `Overlay/DockPreviewOverlay.swift`), so it verifies under `swift build`/`swift test`; only the live AX/cursor behavior needs the real app. - **Overlay on the REAL Dock — never build a Dock.** `Dock.app` has no hover event and no plugin surface, so hover is *inferred*: `AXDockReader` reads `Dock.app`'s AX tree for app-tile frames (`AXApplicationDockItem` only — folders/Trash/Downloads/separators/**minimized-window tiles** are filtered out; minimized windows come from the app's own enumeration, NOT the Dock's minimized-window items), and a passive global `.mouseMoved` monitor (`GlobalCursorMonitor`, no Input Monitoring needed) feeds the cursor. Don't try to extend or replace the system Dock. -- **The first cursor-first, mouse-INTERACTIVE overlay.** Every other overlay sets `ignoresMouseEvents = true`; `DockPreviewOverlayController` is the lone exception — it sets `ignoresMouseEvents = false` + `acceptsMouseMovedEvents = true` so thumbnails take hover/click. It stays a `.nonactivatingPanel` and **never** becomes key/main (no keyboard), so it never steals focus (the previously focused window stays the raise target). Teardown is **synchronous** (`orderOut`) — the files-band ghost-on-Space-switch landmine applies here too. The popup anchors in the gap between tile and content so a native Dock-icon click still falls through to the system. +- **The first cursor-first, mouse-INTERACTIVE overlay.** Every other overlay sets `ignoresMouseEvents = true`; `DockPreviewOverlayController` is the lone exception — it sets `ignoresMouseEvents = false` + `acceptsMouseMovedEvents = true` so thumbnails take hover/click. It stays a `.nonactivatingPanel` and **never** becomes key/main (no keyboard), so it never steals focus (the previously focused window stays the raise target). Teardown is **synchronous** (`orderOut`) — the ghost-on-Space-switch landmine applies here too. The popup anchors in the gap between tile and content so a native Dock-icon click still falls through to the system. - **Peek = front the REAL window, then restore it (the macOS-forced model).** macOS won't render fresh pixels for an off-screen window, so an in-popup *projection* of an occluded/minimized window is stale/icon-only — the only true live preview is to bring the real window to the front. So hover front-raises via `WindowService.peekRaise`, which must **genuinely front the window so its app renders it live** for capture — two parts, both load-bearing: (1) the **SkyLight `setFront` handshake** (`_SLPSSetFrontProcessWithOptions` + `makeKeyWindow`, the AltTab idiom) reliably activates a **background app** in one shot — `kAXRaise` + `activate()` alone often leaves a background app un-activated (which is why the *commit* path needs a watchdog to retry until it sticks), so its windows stay throttled and the capture comes back **stale** (the bug: a peek's live preview only worked for the app already in front); (2) `kAXMain` + the app's `kAXFocusedWindow` make THIS window the app's focused one (an app with several windows otherwise keeps drawing its previously-focused window). It's still lightweight + reversible otherwise (**no** focus-history promotion, **no** watchdog/hold-guard — those fight the put-back), and it **skips both the SkyLight handshake and the singletons under Stage Manager** (they make WindowManager's stage arbiter oscillate — the switcher landmine; a peek there falls back to raise + activate). NB: a peek now performs a *real* app switch on hover (menu bar follows) — that's intrinsic to getting live pixels from a background app; restore on leave switches back. The previously-front window is captured at session start (`WindowService.frontmostWindow()`) and restored on leave-without-commit; a **click** commits via the hardened `raiseDeminimizing` (un-minimize then `raise()`) and skips restore. **Minimized** windows are NOT fronted to peek (would need de-minimizing) — they surface only on commit. Raise on hover-**enter** only (not per tick) to bound z-order churn; current-Space-only keeps it on the cheap AX raise path. **This REVERSES the original "project, never raise" decision** (`design.md` D2) — don't switch it back to in-card projection thinking it's safer; it just shows icons. The **tabs** keep the switcher's cache-first / last-good-frame safety (`isDegradedCapture`/`isStripProxy`) so they never show a sideways proxy; a peeked (fronted) window yields a clean tab frame that persists. **The hovered tab shows a STATIC last-good image — a single one-shot capture after the window settles, NOT a continuous stream / screen recording** (a prior `SCStream`/`WindowLivePreview` version was deliberately reverted). The live view is the *fronted real window itself*; the tab is just a selector. On `peek`, after a **~0.5s settle delay** (`captureDelay`, a cancellable `Task` killed on retarget/dismiss — the window animates forward when fronted and an immediate capture grabs the mid-transition "sideways, coming-from-the-Dock" frame), the controller takes ONE `thumbnails.liveCapture` of the settled window and stores it. Don't reintroduce a per-frame pump or a stream. Each **tab is sized to its window's own aspect ratio** (`DockPreviewLayout.cardWidth(forAspect:)`, fixed height × aspect, clamped); the image then **fills** that aspect-correct card (`.aspectRatio(.fill)` + `.clipped()`). The order matters: the card MUST be aspect-sized FIRST — filling a *fixed-resolution* box mangles a portrait window into a landscape crop (don't do that). Given an aspect-correct card, fill crops only the hairline difference between the AX `realFrame` aspect (which sizes the card) and the captured image's aspect, and — unlike `scaledToFit` — it leaves no letterbox gap and lands every frame edge-to-edge in the same place, so the seed→capture swap neither sits off-center nor jumps to re-center. The capture routes through `thumbnails.inject` → dock cache + dock tab + switcher cache/model (cross-population). Minimized windows aren't captured (can't front → no fresh frame) — they hold their seeded frame and surface on commit. **`openApp` must SEED each tab from `thumbnails.cached(id)` on open** (the switcher's `seed` pattern) — `dismiss` wipes `overlay.model`, and the immediate per-window refresh of still-occluded windows degrades (mac won't capture off-screen), so without the seed the popup re-opens on bare icons and the good cached frame is never shown. The `ThumbnailService` cache (persistent; `dismiss` only calls `endLiveSession`, never `clear`) is the last-good-frame store; the dock controller uses its OWN `ThumbnailService` instance (no live-session contention with the switcher's) but **cross-populates** the switcher's: a peek fronts a window → captures a good frame → the dock `onThumbnail` calls `switcherThumbnails.inject(image, for:)` so the switcher's cache + live model refresh too, and `openApp` seeds tabs from `thumbnails.cached(id) ?? switcherThumbnails.cached(id)` so each surface benefits from the other's captures. (`inject` has no degraded gate — only feed it known-good frames, which the `onThumbnail` path already guarantees.) - **The pure brain is `DockHoverModel`; time is an INPUT.** Hit-test, orientation-aware anchor (bottom = above, left/right = beside, clamped on-screen), and the open/swap/keep/dismiss lifecycle with a unified tile+popup **live zone** + **grace dismiss** are all pure and unit-tested — `feed(...)` takes a `now:` timestamp so grace timing is deterministic. Coordinates are **Cocoa global (bottom-left)**; `AXDockReader` converts AX's top-left space at the boundary so the model never juggles handedness. - **Edge-gated tracking.** While nothing is shown the Dock is read only when the cursor is near a screen edge (`nearDockEdge`); while the popup is open a ~60ms timer re-reads + re-feeds so grace, **magnification** (the anchor re-glues to the growing tile via `reanchor`), and the live peek all advance without depending on move events. The reader caches nothing, so an **auto-hidden** Dock simply reads empty (idle) until it reveals. `DockPreviewController` is gated by the opt-in (`setEnabled`); when off, the monitor isn't even installed. - **Menu parity (change `dock-preview-menu-parity`) — the popup yields to the Dock's own menu.** A passive global `.rightMouseDown` monitor (`GlobalCursorMonitor` — it **never consumes** the event, so the native Dock menu opens unmodified) feeds the pure `DockHoverModel.rightClick(at:tiles:)`; a right-click on a tile **dismisses** the popup (restoring any peeked window) so the native action menu owns the stage. Two reinforcing pieces keep it gone: (a) the per-tick `reanchor` calls `overlay.move(to:)` (**reposition only**) — never `show`/`orderFrontRegardless` — so a menu that opens above the popup stays above it (the old per-tick re-front was exactly what stomped the menu behind it); (b) the controller records the right-clicked tile (`menuSuppressedPID`) and **short-circuits `handleCursor` while the cursor stays on that tile**, so a stray move (the live global `.mouseMoved` monitor is still running) doesn't re-open the popup behind the menu. Suppression clears the instant the cursor leaves the tile (the proxy for "menu interaction over" — there's no cheap signal for the Dock menu actually *closing*; the one edge is Escape-without-moving, which stays suppressed until you leave the tile). - **Left-click on the shown tile commits the highlighted preview (change `commit-dock-preview-on-icon-click`) — the mirror of the right-click rule.** A passive global `.leftMouseDown` monitor (the `CursorMonitor.onLeftDown` seam — same never-consumes contract as the right-click pair; the `.leftMouseDown` infra already exists for window-groups snap) feeds the pure `DockHoverModel.leftClick(at:tiles:) -> pid_t?` (returns the **active** app's pid iff the click hit *its* tile). `handleLeftClick` then commits `overlay.model.highlightedID` via the normal `commit` path (`raiseDeminimizing` + `dismiss(restore: false)`) — **only when a card is actually highlighted (peeked)**. Why it exists: an icon click falls through to the native Dock but never routed through commit, so the popup treated it as "not a commit" and the leave-restore (`dismiss(restore: true)` → re-front the pre-peek window) **silently undid it** — clicking the card stuck, clicking the icon reverted. With no card highlighted, `handleLeftClick` no-ops: native activation stands and nothing was peeked, so there's nothing to restore/undo. **We do NOT (and can't cheaply) consume the click**, so the native Dock *also* activates the app; for a live window the two converge (the peek already made it the app's focused window), for a minimized card our `raiseDeminimizing` de-minimizes the right one (native may not) — the one behavior to confirm on the real signed build. - **Keeping an auto-hide Dock VISIBLE under the popup is infeasible — don't retry.** Investigated exhaustively in `dock-preview-menu-parity` (see its archived `design.md` → "Rejected"): disabling auto-hide (`CoreDockSetAutoHideEnabled(false)`) makes the Dock reserve space and **reflows/shrinks windows**; the Dock **polls the real HID cursor**, so synthetic `.mouseMoved` posted to Dock.app (the reverted `DockRevealKeeper`) is ignored; the full `CoreDock*` surface (HIServices, ~80 syms) has **no "suspend auto-hide"** primitive (only the reflow toggle); the native menu's hold is **modal-menu-tracking** state, inseparable from the visible menu. Only the Dock can hold its own peek. **Landed behavior:** when an auto-hide Dock slides away the popup **freezes in place and stays usable** (graceful, no reflow). Affects only users who auto-hide their Dock. -- **App with no current-Space windows shows NOTHING** (no empty popup) — tracked via `emptyPID` so it doesn't re-enumerate per move. **Minimized windows are included** (the enumeration variant `WindowService.currentSpaceWindows(forApp:)` stops excluding the minimized subrole — the switcher's all-Spaces `snapshot()` is untouched and still excludes them; `WindowInfo.isMinimized` defaults false so the switcher path is unaffected). Errors map to `DockPreviewError` (Core `LocalizedError`, parallel to `FileActionError`) surfaced as a bounded, non-blocking card — never an `NSAlert`, never raw error text in a headline. +- **App with no current-Space windows shows NOTHING** (no empty popup) — tracked via `emptyPID` so it doesn't re-enumerate per move. **Minimized windows are included** (the enumeration variant `WindowService.currentSpaceWindows(forApp:)` stops excluding the minimized subrole — the switcher's all-Spaces `snapshot()` is untouched and still excludes them; `WindowInfo.isMinimized` defaults false so the switcher path is unaffected). Errors map to `DockPreviewError` (a Core `LocalizedError` taxonomy) surfaced as a bounded, non-blocking card — never an `NSAlert`, never raw error text in a headline. ## In-overlay navigation: the odometer (change `restore-odometer-navigation`) — landmines -Post-activation **launcher** and **Files-drill** navigation is the **odometer**: the recognizer accumulates signed centroid travel per axis (`acc += Δcentroid`) and emits one step each time the accumulator crosses the per-axis step distance, **with carry** (`while |acc| ≥ step { acc ∓= step; emit }`). The anchored-positional "joystick" (`PositionalNavigator` / `AxisZone` / `RepeatCadence` / directional axis-lock / the `positional*` tunables / the `Hub/PositionalTrackpadPreview` aim-wedge) was **reverted** — do not reintroduce it. The opening fling and the three-finger **window switcher** were always odometer and are untouched. +Post-activation **launcher** navigation is the **odometer**: the recognizer accumulates signed centroid travel per axis (`acc += Δcentroid`) and emits one step each time the accumulator crosses the per-axis step distance, **with carry** (`while |acc| ≥ step { acc ∓= step; emit }`). The anchored-positional "joystick" (`PositionalNavigator` / `AxisZone` / `RepeatCadence` / directional axis-lock / the `positional*` tunables / the `Hub/PositionalTrackpadPreview` aim-wedge) was **reverted** — do not reintroduce it. The opening fling and the three-finger **window switcher** were always odometer and are untouched. -- **One mechanic, three emit-closures.** `updateLauncher` (item / band) and `updateFilesDrill` (depth = X, highlight = Y) each run the same accumulate-and-emit loop, differing only in what a step *does*. The switcher's `update` is the same loop for windows / Space-rows. There is **no** per-surface navigator object and **no** axis-lock — both axes accumulate independently, so a diagonal steps both. -- **Auto-repeat is the physical trackpad EDGE, not a dwell-eased zone.** `updateEdges`/`edgeAxis` (enter `edgeEnterZone` 0.16 / exit `edgeExitZone` 0.24 hysteresis) emit `launcherEdgeChanged(dx,dy)` when the controlling contact is held near a trackpad border; the controller's `edgeTimer` repeats on `LauncherOverlayController.edgeInterval(tick:acceleration:)` (hyperbolic 0.18s → 0.03s). The launcher and Files drill share `edgeDX/DY`, since they're mutually exclusive sub-states. -- **Edge auto-repeat suppression is Clipboard-only.** `setEdgeAutoScroll` suppresses **horizontal** auto-repeat only for the **Clipboard** band (there horizontal is the deliberate pin / return-to-band action). The **Files** band keeps horizontal auto-repeat — holding depth at the edge **auto-drills** the folder tree (uniform both-axis auto-repeat, the user's choice). -- **Re-baseline the origin on every contact-count change.** `startCentroid`/`lastCentroid`/`stepAccumulator{,Y}` (launcher) and `drillStart`/`drillLast`/`drillAccum{X,Y}` (Files) all reset on a count change so a leaving/landing finger emits no phantom step. Files also clears its held-edge state across the re-baseline. -- **`+1`-finger = action menu (intent).** The Files drill binds the relative `+1` (`count > drillContacts`, one-shot) to **Open-With** (`filesOpenWith`). Detected BEFORE re-baselining the count change. -- **Tuning is feel-only.** `launcherStepDistance` (item / Files depth+highlight) and `launcherContextStepDistance` (band switch, coarser) are **travel distances** (normalized, with carry) — defaults `0.04` / `0.09`. Edge-repeat cadence is the `edgeInterval` ramp; `edgeAcceleration` is pushed from `clipboardEdgeAcceleration`. Surfaced on the Hub Launcher page (plain sliders, no trackpad preview). `axisLockRatio` is the **switcher's** horizontal-vs-vertical dominance gate (pre-existing, unrelated to the deleted positional axis-lock) and stays. +- **One mechanic, shared emit-closures.** `updateLauncher` (item / band) runs the accumulate-and-emit loop; the switcher's `update` is the same loop for windows / Space-rows. There is **no** per-surface navigator object and **no** axis-lock — both axes accumulate independently, so a diagonal steps both. +- **Auto-repeat is the physical trackpad EDGE, not a dwell-eased zone.** `updateEdges`/`edgeAxis` (enter `edgeEnterZone` 0.16 / exit `edgeExitZone` 0.24 hysteresis) emit `launcherEdgeChanged(dx,dy)` when the controlling contact is held near a trackpad border; the controller's `edgeTimer` repeats on `LauncherOverlayController.edgeInterval(tick:acceleration:)` (hyperbolic 0.18s → 0.03s). +- **Edge auto-repeat suppression is Clipboard-only.** `setEdgeAutoScroll` suppresses **horizontal** auto-repeat only for the **Clipboard** band (there horizontal is the deliberate pin / return-to-band action). +- **Re-baseline the origin on every contact-count change.** `startCentroid`/`lastCentroid`/`stepAccumulator{,Y}` all reset on a count change so a leaving/landing finger emits no phantom step. +- **Tuning is feel-only.** `launcherStepDistance` (item step) and `launcherContextStepDistance` (band switch, coarser) are **travel distances** (normalized, with carry) — defaults `0.04` / `0.09`. Edge-repeat cadence is the `edgeInterval` ramp; `edgeAcceleration` is pushed from `clipboardEdgeAcceleration`. Surfaced on the Hub Launcher page (plain sliders, no trackpad preview). `axisLockRatio` is the **switcher's** horizontal-vs-vertical dominance gate (pre-existing, unrelated to the deleted positional axis-lock) and stays. ## Final gesture mechanism @@ -83,7 +65,7 @@ Post-activation **launcher** and **Files-drill** navigation is the **odometer**: ## Minimize-all + first-class minimized windows (change `minimize-all-and-reachable-minimized`) — landmines -Two coupled opt-ins (both default OFF): **`swipeDownMinimizesAll`** (three-finger DOWN minimizes all current-Space windows, revealing the desktop) and **`includeMinimizedWindows`** (minimized windows appear in the switcher + ⌘-Tab, restored on select). All MLX-free Core → `swift build`/`swift test`; the live AX behavior needs the user's signed build. +Two coupled opt-ins (both default OFF): **`swipeDownMinimizesAll`** (three-finger DOWN minimizes all current-Space windows, revealing the desktop) and **`includeMinimizedWindows`** (minimized windows appear in the switcher + ⌘-Tab, restored on select). All Core → `swift build`/`swift test`; the live AX behavior needs the user's signed build. - **Real minimize, current-Space, NOT `showDesktop()`.** `WindowService.minimizeAllWindows()` writes `kAXMinimized = true` per window (genuine minimize into the Dock, Windows Win+D) — deliberately **not** the native slide-aside `MissionControl.showDesktop()` (which doesn't minimize, is a toggle, and never populates the switcher). It reuses the `isSwitchable` gate (own app / floating / non-standard excluded), **skips already-minimized** (idempotent), scopes to the **current Space** (`kAXWindowsAttribute` returns current-Space + minimized; the minimized ones are skipped, off-Space aren't returned), and one failed write never blocks the rest (returns `(minimized, failed)` counts — observable state, no `NSAlert`). - **Only reachable with the vertical opt-in.** The DOWN action is dispatched in `AppCoordinator.gestureDidTriggerMissionControl(up:false)`, which only fires when `manageVerticalGesture` is effective (else the OS owns three-finger-vertical). So `swipeDownMinimizesAll` is gated on the Space-row opt-in; App Exposé stays the default down-action when off. diff --git a/Package.resolved b/Package.resolved index fa9e8cf..aafd905 100644 --- a/Package.resolved +++ b/Package.resolved @@ -1,49 +1,6 @@ { - "originHash" : "6617f71c544d470b7c182872df01a4e9c9c2fc091f95662eccd5d851815b1956", + "originHash" : "7d7d016890307ae16cb97cde1bf8a577855701705b41bdb8eff1b5eae6939bb9", "pins" : [ - { - "identity" : "eventsource", - "kind" : "remoteSourceControl", - "location" : "https://github.com/mattt/EventSource.git", - "state" : { - "revision" : "a3a85a85214caf642abaa96ae664e4c772a59f6e", - "version" : "1.4.1" - } - }, - { - "identity" : "flux-2-swift-mlx", - "kind" : "remoteSourceControl", - "location" : "https://github.com/VincentGourbin/flux-2-swift-mlx", - "state" : { - "revision" : "add498c4a30581d4457ef513bbfe8c9c097c504c" - } - }, - { - "identity" : "gemma-4-swift-mlx", - "kind" : "remoteSourceControl", - "location" : "https://github.com/VincentGourbin/gemma-4-swift-mlx", - "state" : { - "revision" : "c6f8ab5820379898b1d437e8e5c463f376672613" - } - }, - { - "identity" : "mlx-swift", - "kind" : "remoteSourceControl", - "location" : "https://github.com/ml-explore/mlx-swift", - "state" : { - "revision" : "dc43e62d7055353c7f99fa071a4e71d29dfddc44", - "version" : "0.31.4" - } - }, - { - "identity" : "mlx-swift-lm", - "kind" : "remoteSourceControl", - "location" : "https://github.com/ml-explore/mlx-swift-lm", - "state" : { - "branch" : "main", - "revision" : "a47894a1e7e963b24bd48c030f5fc1d1627e60e9" - } - }, { "identity" : "openmultitouchsupport", "kind" : "remoteSourceControl", @@ -53,24 +10,6 @@ "version" : "4.0.0" } }, - { - "identity" : "swift-argument-parser", - "kind" : "remoteSourceControl", - "location" : "https://github.com/apple/swift-argument-parser", - "state" : { - "revision" : "6a52f3251125d74daf04fcbd5e6f08a75d074382", - "version" : "1.8.2" - } - }, - { - "identity" : "swift-asn1", - "kind" : "remoteSourceControl", - "location" : "https://github.com/apple/swift-asn1.git", - "state" : { - "revision" : "a9a5efd40eaf558a2bcd48d64b1d1646be686008", - "version" : "1.7.1" - } - }, { "identity" : "swift-async-algorithms", "kind" : "remoteSourceControl", @@ -80,15 +19,6 @@ "version" : "1.1.4" } }, - { - "identity" : "swift-atomics", - "kind" : "remoteSourceControl", - "location" : "https://github.com/apple/swift-atomics.git", - "state" : { - "revision" : "b601256eab081c0f92f059e12818ac1d4f178ff7", - "version" : "1.3.0" - } - }, { "identity" : "swift-collections", "kind" : "remoteSourceControl", @@ -97,105 +27,6 @@ "revision" : "fea17c02d767f46b23070fdfdacc28a03a39232a", "version" : "1.5.1" } - }, - { - "identity" : "swift-crypto", - "kind" : "remoteSourceControl", - "location" : "https://github.com/apple/swift-crypto.git", - "state" : { - "revision" : "1b6b2e274e85105bfa155183145a1dcfd63331f1", - "version" : "4.5.0" - } - }, - { - "identity" : "swift-huggingface", - "kind" : "remoteSourceControl", - "location" : "https://github.com/huggingface/swift-huggingface.git", - "state" : { - "revision" : "b721959445b617d0bf03910b2b4aced345fd93bf", - "version" : "0.9.0" - } - }, - { - "identity" : "swift-jinja", - "kind" : "remoteSourceControl", - "location" : "https://github.com/huggingface/swift-jinja.git", - "state" : { - "revision" : "0b67ecb79139f6addef8699eff3622808aa6c7dc", - "version" : "2.3.6" - } - }, - { - "identity" : "swift-mlx-profiler", - "kind" : "remoteSourceControl", - "location" : "https://github.com/VincentGourbin/swift-mlx-profiler", - "state" : { - "revision" : "b2a83b36a24b2e252573369644259a648fbaf18a", - "version" : "1.4.0" - } - }, - { - "identity" : "swift-nio", - "kind" : "remoteSourceControl", - "location" : "https://github.com/apple/swift-nio.git", - "state" : { - "revision" : "77b84ac2cd2ac9e4ac67d19f045fd5b434f56967", - "version" : "2.101.0" - } - }, - { - "identity" : "swift-numerics", - "kind" : "remoteSourceControl", - "location" : "https://github.com/apple/swift-numerics", - "state" : { - "revision" : "0c0290ff6b24942dadb83a929ffaaa1481df04a2", - "version" : "1.1.1" - } - }, - { - "identity" : "swift-syntax", - "kind" : "remoteSourceControl", - "location" : "https://github.com/swiftlang/swift-syntax.git", - "state" : { - "revision" : "79e4b74a295b6eb74a8b585e3a39d29e70c1dbd1", - "version" : "603.0.2" - } - }, - { - "identity" : "swift-system", - "kind" : "remoteSourceControl", - "location" : "https://github.com/apple/swift-system.git", - "state" : { - "revision" : "669763cfd5806a67e21972d7e5e2d6b80b1ea985", - "version" : "1.6.5" - } - }, - { - "identity" : "swift-transformers", - "kind" : "remoteSourceControl", - "location" : "https://github.com/huggingface/swift-transformers", - "state" : { - "revision" : "2fa33e1f5e7131a7fc64c28e6d161dcec0d24820", - "version" : "1.3.3" - } - }, - { - "identity" : "yams", - "kind" : "remoteSourceControl", - "location" : "https://github.com/jpsim/Yams", - "state" : { - "revision" : "3d6871d5b4a5cd519adf233fbb576e0a2af71c17", - "version" : "5.4.0" - } - }, - { - "identity" : "yyjson", - "kind" : "remoteSourceControl", - "location" : "https://github.com/ibireme/yyjson.git", - "state" : { - "revision" : "8b4a38dc994a110abaec8a400615567bd996105f", - "version" : "0.12.0" - } } ], "version" : 3 diff --git a/Package.swift b/Package.swift index e8d3a95..7456599 100644 --- a/Package.swift +++ b/Package.swift @@ -18,23 +18,7 @@ let package = Package( // VENDORED in-repo at ./DeviceLinkKit (rather than a `../` sibling) so CI can resolve it — the // sibling path only existed on the maintainer's machine and broke the release build. Keep this // copy in sync with the standalone package if the iOS app consumes a separate copy. - .package(path: "DeviceLinkKit"), - // The MLX/Gemma 4 runtime. Pulls mlx-swift / swift-transformers / mlx-swift-lm transitively. - // Building anything that links this needs `xcodebuild` (Metal shaders) — see GemmaRuntime target. - // PINNED to an exact revision (not `branch: "main"`): the upstream `main` drifts and has shipped - // commits that fail to compile (e.g. a `Float`→`MLXArray` type error in LoRA/TurboQuant). A - // branch requirement let CI re-resolve to a broken HEAD and fail the release build even though - // the committed Package.resolved pinned a good commit. An exact revision freezes it everywhere. - // To bump: change the SHA here, re-resolve, and verify with `xcodebuild`. - .package(url: "https://github.com/VincentGourbin/gemma-4-swift-mlx", - revision: "c6f8ab5820379898b1d437e8e5c463f376672613"), - // Flux 2 image-generation runtime (MLX). Pins mlx-swift to the SAME exact 0.31.4 as - // gemma-4-swift-mlx, so it resolves with no version change. Linked into GemmaRuntime ONLY - // (keeps ThreeFingerSwitcherCore MLX-free). PINNED to an exact revision (same rationale as - // gemma above: upstream `main` drifts). To bump: change the SHA here, re-resolve, verify - // with `xcodebuild`. - .package(url: "https://github.com/VincentGourbin/flux-2-swift-mlx", - revision: "add498c4a30581d4457ef513bbfe8c9c097c504c") + .package(path: "DeviceLinkKit") ], targets: [ // All app logic lives in this library so the test target can `@testable import` it. @@ -55,31 +39,11 @@ let package = Package( .swiftLanguageMode(.v5) ] ), - // The MLX/Gemma 4 runtime, ISOLATED in its own target so `ThreeFingerSwitcherCore` and the - // test target stay MLX-free and keep building under plain `swift build`/`swift test`. This - // target links MLX (Metal shaders) so it — and anything depending on it (the app) — builds - // ONLY via `xcodebuild`, never `swift build`. It conforms to Core's public `LLMRuntime` seam. - .target( - name: "GemmaRuntime", - dependencies: [ - "ThreeFingerSwitcherCore", - .product(name: "Gemma4Swift", package: "gemma-4-swift-mlx"), - // Flux 2 image-generation (MLX). Same exact mlx-swift 0.31.4 as Gemma — no version - // change. Kept in GemmaRuntime so Core stays MLX-free. - .product(name: "Flux2Core", package: "flux-2-swift-mlx"), - .product(name: "FluxTextEncoders", package: "flux-2-swift-mlx") - ], - path: "Sources/GemmaRuntime", - swiftSettings: [ - .swiftLanguageMode(.v5) - ] - ), - // Thin executable: calls runThreeFingerSwitcher() from Core and injects the real Gemma runtime - // (from GemmaRuntime) at the model seam. Builds via `xcodebuild` (it transitively links MLX). + // Thin executable: calls runThreeFingerSwitcher() from Core. .executableTarget( name: "ThreeFingerSwitcher", dependencies: [ - "ThreeFingerSwitcherCore", "GemmaRuntime" + "ThreeFingerSwitcherCore" ], path: "Sources/ThreeFingerSwitcherApp", swiftSettings: [ diff --git a/README.md b/README.md index c4fa363..374b312 100644 --- a/README.md +++ b/README.md @@ -29,11 +29,9 @@ A lightweight **macOS menu-bar app** that recreates the **Windows Precision Touc - **Dock window previews (optional — the switcher by *mouse*).** An opt-in (default off): hover an app's **Dock icon** with the cursor and a row of that app's **windows on the current Space** (including **minimized**) fans out above the tile. Hovering a window tab brings the **real window to the front** so you see its live content (and a crisp static thumbnail lands in the tab); moving away **restores** the window you came from, and **clicking** a tab keeps it forward. It's the mouse-flow counterpart to the three-finger switcher — the app's one deliberately **cursor-driven** surface. Apps with no windows on the current Space show nothing. Like Clipboard/Files it **relocates no gesture and needs no re-login or new permission** (it reads the Dock's Accessibility tree and reuses the Screen Recording grant for thumbnails). Toggle it under **Hub → Switcher → Dock window previews**. - **Four-finger launcher (optional).** Slide **four fingers** horizontally → a launcher overlay of your favorites (apps, folders, URLs, Shortcuts, scripts, and "preset" workspaces) organized into **context bands**. The bands are a **vertical strip of band icons down the left** (each band has its own icon; only the active one is colored), with the active band's items in a grid on the right — **vertical** scrubs between bands, **horizontal** crosses into the grid. Scrub to an item, then **dwell** (≈500 ms, haptic tick + charge-ring) and **lift to fire**; a quick flick lifts off without firing. Once open you can **relax to two fingers** to navigate comfortably. Navigation is **step-based** (an odometer) — scrub to move one item per step, and **hold a contact at the trackpad edge** to auto-repeat, accelerating the longer you hold. Launching always **opens a window in the *current* Space** (or pulls a single-window app to you) instead of teleporting you away. Also an opt-in (default off). - **Clipboard history (optional, lives in the launcher).** A further opt-in (default off — it records copied content, so privacy-gated): when on, what you copy — text, images, files/folders, colors, links — is stored locally and shown as the **last band** in the launcher. Scrub the key list, see a live preview on the right (image / QuickLook file content / text / color), **lift to paste** it into the app you were in; **swipe right** on an entry to pin it, **left** to leave the band. Unlike the two gesture opt-ins it **relocates no gesture and needs no re-login or new permission**. -- **Files band (optional, lives in the launcher).** An opt-in (default off) that drops a **local-only Finder** into the launcher: land on the **Files band** and you're in a bounded **column navigator** — a thin **icon rail of ancestor folders**, the **current folder's list**, and a **live preview** (QuickLook for files, a contents-peek for folders). With two fingers: **horizontal** descends into / out of folders, **vertical** moves the highlight (it's **pure trackpad** — no search, no keyboard); **lift opens** the highlighted item for real (file → default app, folder → Finder window) on the *current* Space; **+1 finger then lift** opens an **Open-With** popup of just the apps that can open it (scrub + lift to pick); a **four-finger horizontal swipe discards** (defusing a pending open — it never kills a running app). Configure roots and appearance under **Hub → Files** — including **"remember and reopen the last folder"** (when off, the band always opens on your configured roots; when on, it reopens where you left off). Like Clipboard it **relocates no gesture and needs no re-login or new permission** (it reads the filesystem on demand). Everything **morphs in like a water droplet** — nothing pops to screen. -- **AI commands (optional, on-device, live in the launcher).** An opt-in (default off): AI commands are **ordinary, movable band items** you pick from a **~50-command catalog** (writing, tone, translate, dev, reply, capture, vision, format) and can drop into **any** band. Scrub to one, dwell to arm, lift to fire — it grabs your **selected text** (or clipboard / a **screen region** for vision), runs it through an on-device **Gemma 4** model, and **streams the result into a preview canvas** (rendering right-to-left/mixed text correctly). A fresh **two-finger** **down swipe applies** the result (replace-in-place / paste), a **horizontal swipe discards** (4 fingers open/dismiss the platform, 2 fingers act within it); a translate command carries an in-canvas **language picker** that re-runs in place. Commands are either **in-place edits** (rewrite/translate/summarize) or **background tasks** (add-to-calendar/reminders, new contact, save-to-project, open-a-tool-with-payload), each with an optional action-review before any side effect. The model is **multi-gigabyte and downloaded on first enable**; it runs **in-process via MLX**, so this requires **Apple Silicon** (M-series; targets the stronger M4/M5-class chips) — there is no cloud and no low-end fallback in v1. Calendar/Reminders/Contacts tasks ask for their permission the first time they run; nothing else new. - **Per-app & per-site keyboard language (optional).** An opt-in (default off) that gives macOS the Windows-style per-window keyboard language it lacks: it **remembers the input source you use in each app** — and, inside browsers, **per website (host)** — and **restores it automatically** on every app/page switch. It learns as you change the language by hand (no list to fill in). No gesture relocation and no re-login; per-site host reading uses Accessibility by default, with an optional Apple Events path for browsers that don't expose the URL. - **Mission Control / App Exposé always available** — natively on three-finger up/down when the opt-ins are off, or app-synthesized when they're on. The app never blocks the OS (see the keystone below). -- **One configuration Hub.** Every setting, the favorites/bands editor, AI-command authoring, and the permissions/setup flow live in a **single Hub window** (menu → **Open Hub…**) with an Overview landing page and a grouped sidebar (Bands · Switcher · Spaces · Launcher · Clipboard · AI · Keyboard Language · Setup · General). The menu-bar menu is trimmed to quick actions. +- **One configuration Hub.** Every setting, the favorites/bands editor, and the permissions/setup flow live in a **single Hub window** (menu → **Open Hub…**) with an Overview landing page and a grouped sidebar (Bands · Switcher · Launcher · Clipboard · Keyboard Language · Devices · Setup · General). The menu-bar menu is trimmed to quick actions. - No keypresses, no clicks — pure trackpad. (The lone exception is the optional **Dock window previews**, which are mouse-hover-driven by nature.) **Platform:** built and tested on **macOS 26 (Tahoe)**; deployment target **macOS 15.0+**. Apple Silicon + Intel (universal dep). **License: GPL-3.0.** @@ -91,9 +89,7 @@ Tell them: enabling either feature requires a re-login to take effect, and it st **Clipboard history** is a third opt-in, but not a gesture one: it just records what they copy and shows it as the last band inside the launcher (so it needs the launcher on to be useful). It **relocates no gesture and needs no re-login or new permission** — flip it on under **Hub → Clipboard**, where they can also set retention caps, exclude apps, pause, or clear. Off by default because it stores copied content locally. -**AI commands** is a fourth opt-in (also not a gesture one; also lives in the launcher, so it needs the launcher on). Flip it on under **Hub → AI**. Two things to tell the user honestly: (1) enabling it starts a **one-time multi-gigabyte model download** (Gemma 4, on-device) — the AI page shows progress and offers **Retry** if it fails (e.g. offline); the commands only work once the download completes. (2) It requires **Apple Silicon** (M-series) — it runs the model in-process via MLX, so it won't function on Intel. No re-login and no new permission to enable; **Calendar / Reminders / Contacts** prompts appear only the first time the matching task actually runs. AI commands are ordinary band items: browse the **catalog** and author/edit them inline on the **Hub → Bands** page, and choose the model on **Hub → AI**. It **relocates no gesture**. - -**Per-app / per-site keyboard language** is a fifth opt-in (also not a gesture one). Flip it on under **Hub → Keyboard Language**. Once on, it learns the keyboard input source you use in each app and restores it on the next app switch — turn on **per-site** too and, inside browsers, it remembers the language per website (host). No re-login and no new permission to enable; per-site host reading uses Accessibility by default, with an optional Apple Events fallback (prompts the first time) for browsers that don't expose the page URL via Accessibility. +**Per-app / per-site keyboard language** is a fourth opt-in (also not a gesture one). Flip it on under **Hub → Keyboard Language**. Once on, it learns the keyboard input source you use in each app and restores it on the next app switch — turn on **per-site** too and, inside browsers, it remembers the language per website (host). No re-login and no new permission to enable; per-site host reading uses Accessibility by default, with an optional Apple Events fallback (prompts the first time) for browsers that don't expose the page URL via Accessibility. ### A4. Make it permanent (optional but recommended) @@ -111,18 +107,18 @@ Symptom: cursor still moves but **clicks/scroll/keyboard stop reaching any windo ### B0. Source of truth: read `openspec/specs/` first -This project was built spec-first with **OpenSpec**. The **canonical behavior** lives in `openspec/specs//spec.md` (**29 capabilities** — the switcher core: `gesture-recognition`, `switcher-overlay`, `window-enumeration-and-raising`, `touch-input`, `native-gesture-config`, `spaces-rearrange-config`, `tunable-settings`, `menubar-app-shell`, `permissions-onboarding`, `first-run-onboarding`; the opt-in features: `runtime-gesture-ownership`, `launcher-overlay`, `launch-items`, `launch-actions`, `favorites-editor`, `clipboard-history`, `dock-hover-detection`, `dock-preview-overlay` (the mouse-hover Dock previews); the AI commands: `on-device-ai-runtime`, `ai-command-band`, `ai-command-catalog`, `ai-command-tasks`, `ai-error-handling`, `selection-io`; keyboard language: `per-app-keyboard-language`, `per-site-keyboard-language`; and the shell/pipeline: `configuration-hub`, `app-branding`, `release-pipeline`). Every feature was a `change/` (proposal → design → spec delta → tasks), now in `openspec/changes/archive/`. **Before changing behavior, read the relevant spec; after changing behavior, update it.** The archived changes are an excellent design history: +This project was built spec-first with **OpenSpec**. The **canonical behavior** lives in `openspec/specs//spec.md` (**35 capabilities** — the switcher core: `gesture-recognition`, `switcher-overlay`, `window-enumeration-and-raising`, `window-groups`, `command-tab-switcher`, `touch-input`, `native-gesture-config`, `spaces-rearrange-config`, `tunable-settings`, `menubar-app-shell`, `permissions-onboarding`, `first-run-onboarding`; the opt-in features: `runtime-gesture-ownership`, `launcher-overlay`, `launch-items`, `launch-actions`, `open-claude-here`, `favorites-editor`, `clipboard-history`, `dock-hover-detection`, `dock-preview-overlay` (the mouse-hover Dock previews); the device link: the `device-link-*` capabilities + `mac-qr-pairing`; keyboard language: `per-app-keyboard-language`, `per-site-keyboard-language`; and the shell/pipeline: `configuration-hub`, `app-branding`, `release-pipeline`). Every feature was a `change/` (proposal → design → spec delta → tasks), now in `openspec/changes/archive/`. **Before changing behavior, read the relevant spec; after changing behavior, update it.** The archived changes are an excellent design history: - **Switcher / window-raising internals:** `cross-space-windows`, `fix-focus-vacuum-on-raise`, `space-grid-navigation`, `fix-off-space-listing-and-focus` — read their `design.md` for the hard-won private-API details. - **Optional features:** `optional-space-row-gesture` (runtime gesture ownership — the scroll tap + Mission Control synthesis substrate), `four-finger-launcher` (the launcher, favorites model, launch strategies, dwell-to-arm), `launcher-two-finger-nav` (drop-to-two-finger navigation), `launcher-vertical-band-list` (the band strip as a vertical icon list on the left; vertical switches bands, horizontal crosses into the grid), and `clipboard-history` (the launcher's Clipboard band: a polled-`changeCount` recorder + on-disk store, paste-into-front, pin/edge-accel — its `design.md` covers the band-not-keyboard-picker decision). -- **AI commands:** `ai-command-band` (command model, executor orchestration, **swipe-to-resolve** preview canvas — down swipe commits, horizontal swipe discards) — read its `design.md` for the `LLMRuntime` seam, the in-process MLX Gemma 4 runtime, the resumable download, and the swipe-to-resolve decision (D4); `expand-ai-command-catalog` (the ~50-command catalog browser, the in-canvas translate language picker, vision, and bidi RTL/LTR rendering); `harden-ai-error-handling` (every AI error surfaces clean + non-blocking). -- **Shell & later features:** `configuration-hub` (folds the Settings/Favorites/AI-editor/Setup windows into one **Hub**, and makes AI commands first-class, movable band items) and `hub-in-switcher` (a Hub item in the switcher so it's reachable in accessory mode); `per-app-keyboard-language` / `per-site-keyboard-language` (auto-remember the input source per app, and per website inside browsers). +- **Shell & later features:** `configuration-hub` (folds the Settings/Favorites/Setup windows into one **Hub**) and `hub-in-switcher` (a Hub item in the switcher so it's reachable in accessory mode); `per-app-keyboard-language` / `per-site-keyboard-language` (auto-remember the input source per app, and per website inside browsers). +- **The great cleanup (`remove-local-ai`):** the on-device AI stack (Gemma/MLX runtime, the AI command band + canvas, background agents, the notch timeline, voice/computer-use) and the Files band were **removed** to refocus the app on the switcher, the launcher, and clipboard history. The `v1` branch / `v1.0.0` release preserve the full-featured app; the archived `ai-*` / `files-*` changes remain as design history. - **Dock window previews:** `dock-window-previews` (the mouse-hover Dock preview — read its `design.md` for the **reversed peek decision**: the peek *fronts the real window* via the SkyLight `setFront` handshake + `kAXMain`/`kAXFocusedWindow` and **restores** it on leave, because macOS won't render fresh pixels for an off-screen window; the tab itself is a static last-good capture, not a stream). ### B1. Repo map ``` -Package.swift SwiftPM: Core library + GemmaRuntime (MLX) + thin executable + tests + TouchSpike + LauncherSpike -Sources/ThreeFingerSwitcher/ ── ThreeFingerSwitcherCore library (ALL app logic; MLX-free → builds under swift build/test) +Package.swift SwiftPM: Core library + thin executable + tests + TouchSpike + LauncherSpike +Sources/ThreeFingerSwitcher/ ── ThreeFingerSwitcherCore library (ALL app logic; builds under swift build/test) App/ AppDelegate, AppCoordinator (the wiring hub), StatusItemController, Bootstrap.swift (public runThreeFingerSwitcher()) TouchInput/ TouchEngine (wraps OpenMultitouchSupport; derives finger count + velocity), TouchFrame, ScrollEventTap (session CGEventTap consuming the freed scroll; runs only while an opt-in is effective) @@ -133,37 +129,27 @@ Sources/ThreeFingerSwitcher/ ── ThreeFingerSwitcherCore library (AL Overlay/ OverlayController (non-activating NSPanel), SwitcherView (SwiftUI strip + dots), SwitcherModel, SwitcherLayout, LauncherView / LauncherModel / LauncherOverlayController / LauncherGridLayout (the four-finger launcher HUD: a vertical band-icon list + side item grid, dwell-arm/charge-ring/haptics), ClipboardBandView (the Clipboard band's master-detail key-list + value preview; QuickLook for files), - FilesBandView (the Files band's column navigator: ancestor rail + current list + live preview + Open-With popup + bounded failure card), BubbleMorph (the first spring — droplet entrance for the Files band), DockPreviewOverlay (the Dock window previews' mouse-INTERACTIVE popup + aspect-sized window tabs — the one overlay that takes the pointer) Launcher/ LaunchItem (favorites data model: app/path/url/shortcut/script/preset + the synthetic clipboardEntry + context bands), FavoritesStore (Codable persistence), LaunchService (dispatch + "new window here" strategy + clipboard paste-into-front with text/image fallbacks), SpaceWindowMover (SLSMoveWindowsToManagedSpace bring-here) Clipboard/ ClipboardEntry (AppKit-free model), ClipboardStore (on-disk index + blobs, de-dup/retention/pins), ClipboardMonitor (changeCount poll + capture), ClipboardCapture (pure classify + concealed/exclusion filter), ClipboardBandBuilder (store → synthetic last band) — the opt-in clipboard history - Files/ the Files band (files-band, opt-in): FileEntry (AppKit-free entry model, path-stable id) + FileKind, FileWorkspace (NSWorkspace seam) + SystemFileWorkspace + FileActionError (the Files error taxonomy), - DirectoryLister (off-main local listing) + FilesSortOrder, FilesNavigationModel (the PURE column-navigation state machine: ancestors/current/highlight/remembered-locations), - FilesColumnController (bridges the synchronous model to the async lister via a listing cache), FileOpenService (+ PendingOpen defusable open + OpenWithCandidate), FilesBandBuilder (the synthetic "Files" band) — all MLX-free - Dock/ the Dock window previews (dock-window-previews, opt-in): DockTile/DockReader (+ AXDockReader: reads Dock.app's AX tree → app-tile frames) + CursorMonitor (+ GlobalCursorMonitor: passive global mouse-moved), DockHoverModel (PURE hit-test + orientation anchor + live-zone/grace lifecycle), DockPreviewModel + DockPreviewError, DockPreviewController (orchestrates: hover → peekRaise the real window + one-shot static capture, restore on leave, commit on click) — all MLX-free; overlay is Overlay/DockPreviewOverlay.swift + Dock/ the Dock window previews (dock-window-previews, opt-in): DockTile/DockReader (+ AXDockReader: reads Dock.app's AX tree → app-tile frames) + CursorMonitor (+ GlobalCursorMonitor: passive global mouse-moved), DockHoverModel (PURE hit-test + orientation anchor + live-zone/grace lifecycle), DockPreviewModel + DockPreviewError, DockPreviewController (orchestrates: hover → peekRaise the real window + one-shot static capture, restore on leave, commit on click); overlay is Overlay/DockPreviewOverlay.swift NativeGesture/ TrackpadGestureConfig (horizontal three-finger), VerticalGestureConfig (three-finger vertical), FourFingerGestureConfig (four-finger swipes), MissionControl (CoreDockSendNotification synthesis), SpacesRearrangeConfig — all defaults-based, absent-aware backup/restore Permissions/ PermissionsService (detection + lazy requests + refcounted live polling) Onboarding/ the First Touch wizard (first-run-onboarding): FirstRunState (persisted stage machine + legacy-flag bridge), FirstTouchWizardModel/View + WizardActs (the acts: live-touch demo, permission upgrades, lanes consent, playground, curtain), WizardContext (closure wiring), LanesLiveToast (post-re-login acknowledgment) - Settings/ AppSettings (tunables + opt-ins, persisted), ModelManagementView (download/status/evict the on-device model — re-hosted on the Hub's AI page) - Hub/ the single configuration window (configuration-hub): HubView + sidebar pages (Overview / Bands / Switcher·Spaces·Launcher·Clipboard·AI·Keyboard-Language / Setup·General), - BandsCanvas (the favorites "small IDE": sources → bands → items, with inline AI-command authoring + the AI catalog browser, AppearanceEditor icon/tint picker), HubControls / HubStyle (shared Liquid-Glass controls), HubFilesPage (the Files band page: roots editor + appearance/behavior) - AI/ LLMRuntime (the swappable model seam + RuntimeError), StubLLMRuntime / DevAIRuntime (test/dev conformers), ModelManager + ModelRegistry (download/verify/residency lifecycle), - AICommand + RuntimeParameter (the command model + translate language param), AICommandCatalog (the ~50-command catalog), AIBand (the seeded "AI" band — commands are first-class, movable Favorites band items, not synthetic), - AICommandExecutor (input→template→model→output orchestration), PromptTemplate, AIError (the one AI error→message translator), - SelectionService (AX read/replace + ⌘C-restore + screen-region capture), Tasks/ (TaskDispatcher + EventKit/Contacts/project/tool sinks + TaskReview) — all MLX-FREE + Settings/ AppSettings (tunables + opt-ins, persisted), AppDataReset (the Danger-zone selective reset) + Hub/ the single configuration window (configuration-hub): HubView + sidebar pages (Overview / Bands / Switcher·Launcher·Clipboard·Keyboard-Language·Devices / Setup·General), + BandsCanvas (the favorites "small IDE": sources → bands → items, AppearanceEditor icon/tint picker), HubControls / HubStyle (shared Liquid-Glass controls) KeyboardLanguage/ per-app & per-site keyboard language (opt-in): KeyboardLanguageStore (context-key → input-source map), KeyboardLanguagePolicy (pure activate/learn), KeyboardLanguageService (learn-on-deactivation / apply-on-context-change engine), InputSourceController + CarbonInputSourceController (Carbon TIS seam), ContextResolver + BrowserRegistry + ContextKey + HostNormalizer (per-site host context = bundleID|host), AXHostProvider / AppleEventsHostProvider (host readers, AX default + Apple Events opt-in), BrowserContextMonitor (within-browser host-change poll) — auto-remembers the input source per app, and per website (host root) inside browsers -Sources/GemmaRuntime/ ── GemmaRuntime target (links MLX/Metal → builds via xcodebuild ONLY): GemmaMLXRuntime (in-process Gemma 4 via gemma-4-swift-mlx, conforms to LLMRuntime), - GemmaResumableDownloader (HTTP-Range resumable weights download), GemmaRuntime.makeModelManager (injects the real runtime at the seam) -Sources/ThreeFingerSwitcherApp/main.swift thin executable: import Core + GemmaRuntime; inject the MLX runtime, runThreeFingerSwitcher() +Sources/ThreeFingerSwitcherApp/main.swift thin executable: import Core, runThreeFingerSwitcher() Sources/TouchSpike/ throwaway harness to print raw touch frames (swift run TouchSpike) Sources/LauncherSpike/ throwaway harness for the launcher spikes (haptics, window move) — not bundled -Tests/ThreeFingerSwitcherTests/ 848 XCTest unit tests (pure-logic core; @testable import ThreeFingerSwitcherCore — the MLX runtime is verified separately via xcodebuild) +Tests/ThreeFingerSwitcherTests/ 805 XCTest unit tests (pure-logic core; @testable import ThreeFingerSwitcherCore) scripts/ build-app.sh, make-dev-cert.sh, allow-codesign-key.sh, install-launch-agent.sh openspec/ specs (canonical) + changes/archive (history) ``` @@ -173,19 +159,16 @@ The Core/App split exists so the test target can `@testable import ThreeFingerSw ### B2. Build, run, test ```bash -swift build # build the MLX-free Core library + spikes (fast agent loop) -swift test # 848 unit tests (gesture machine + odometer navigation/edge auto-repeat, launcher latching, Dock-preview hover/anchor/layout, models, grouping, layout, settings, native-gesture config + relocation plan/markers, touch, AI executor/tasks/selection/canvas, first-run wizard machine) +swift build # build the Core library + spikes (fast agent loop) +swift test # 805 unit tests (gesture machine + odometer navigation/edge auto-repeat, launcher latching, Dock-preview hover/anchor/layout, models, grouping, layout, settings, native-gesture config + relocation plan/markers, touch, first-run wizard machine) swift run TouchSpike # print live multitouch frames (touch the trackpad) swift run LauncherSpike # throwaway launcher spike harness (haptics / window move) -./scripts/build-app.sh # assemble + sign ThreeFingerSwitcher.app (via xcodebuild — links MLX, packages the Metal metallib) +./scripts/build-app.sh # assemble + sign ThreeFingerSwitcher.app (via xcodebuild) INSTALL=1 ./scripts/build-app.sh # also install in place to /Applications ./ThreeFingerSwitcher.app/Contents/MacOS/ThreeFingerSwitcher --diag # dump the window-enumeration funnel and exit ``` -**The MLX split — why two build paths.** The AI Command Band's model runtime lives in a separate **`GemmaRuntime`** target that links **MLX** (and pulls `gemma-4-swift-mlx` / `mlx-swift` transitively). MLX compiles **Metal shaders**, which needs Xcode's **Metal toolchain** — a one-time `xcodebuild -downloadComponent MetalToolchain`. So: - -- **`ThreeFingerSwitcherCore` + the test target are MLX-free on purpose** → `swift build` / `swift test` build and verify all app logic (incl. the AI executor, tasks, selection, and canvas, which run against a `StubLLMRuntime`). This is the agent's fast verify loop and the only thing CI-pure logic needs. -- **The app executable + `GemmaRuntime` link MLX** → build them with **`xcodebuild`**, which `scripts/build-app.sh` now drives (it was migrated from a hand-rolled `swift build` assembly). `build-app.sh` also **copies MLX's `default.metallib` resource bundle into `Contents/Resources/`** — miss that and the app launches but is SIGKILL'd at first GPU use (no crash report). Don't go back to assembling the `.app` from a plain `swift build`. +**Two verify paths.** `swift build` / `swift test` build and verify all app logic — this is the agent's fast verify loop. The installable `.app` is assembled by `scripts/build-app.sh` via `xcodebuild` (which also handles resource bundles and the stable codesign below); don't hand-assemble the bundle from a plain `swift build`. **Signing matters for permissions.** `build-app.sh` signs with a stable self-signed cert named **"ThreeFingerSwitcher Dev"**. Run `./scripts/make-dev-cert.sh` **once** to create it. Why it matters: TCC (Accessibility/Screen Recording) and `SMAppService` key on the **signing identity**, not the binary hash — a stable cert means **grants persist across rebuilds**. Ad-hoc signing (the fallback) loses them every build. If codesign nags for your keychain password on each build, run `./scripts/allow-codesign-key.sh` once (or click "Always Allow"). @@ -204,9 +187,6 @@ INSTALL=1 ./scripts/build-app.sh # also install in place to /Appli - **Off-Space Chromium windows (Chrome, Chrome Remote Desktop) have no remote-token AX element.** A fresh `_AXUIElementCreateWithRemoteToken` brute force returns nothing for them, so they used to vanish from the list *and* couldn't be raised. Two pieces fix this, both in `WindowService`: (1) **listing** falls back to a CGS-metadata heuristic (`alpha > 0 && min(width,height) ≥ 130`) when no element resolves — empirically separates real windows (incl. Stage-Manager strip thumbnails, min-dim ≥ 150) from sliver/toolbar/zero-alpha junk; (2) **raising** uses a persistent **`elementCache`** keyed by `CGWindowID`, seeded when an app activates (its windows are then on the current Space and resolvable via `kAXWindowsAttribute`) and during snapshots — a cached element stays valid across Spaces, so `kAXRaiseAction` on it *navigates* to the window. Limit: a Chromium window off-Space since before launch and never focused has no cached element and can't be navigated to (the AltTab/HyperSwitch limit). **Do NOT** try to switch Spaces with `CGSManagedDisplaySetCurrentSpace` — the WindowServer gates Space switching to Dock.app's privileged connection; the symbol resolves but no-ops for an unentitled, SIP-on process (it's why yabai needs SIP off). We tried it; it's removed. - **Off-Space focus is stolen by `WindowManager` ~300 ms after the Space switch** — a *different* mechanism from the current-Space singleton oscillation above. The +180 ms watchdog checks too early to see it, so `raise()` arms a bounded **polling hold-guard** (`offSpaceHoldTick`, off-Space + Stage-Manager only): poll every ~60 ms and re-front the target the instant the steal is detected (≈ one-frame flash), bounded to a few re-fronts so a daemon that fights back can't make it thrash. Don't turn it back into a fixed-delay re-assert (slower, visible flash) or drop the bound. - **The overlay panel is non-activating, `ignoresMouseEvents`, must never become key/main, and is always ordered out on gesture end.** On the common path it sits at `.popUpMenu` with **no `.stationary`** (a higher band / `.stationary` are Exposé-exempt and perturb focus/Space arbitration). The **one scoped exception:** while **Mission Control is open**, `OverlayController.show(aboveMissionControl:)` raises it to `.screenSaver` + `.stationary` so the switcher floats *above* MC instead of behind it — applied per-show only in that case, and a commit then dismisses MC (synthesized Escape, never a re-toggle) before re-raising the window from a clean state. Don't widen that elevated config to the normal path. -- **The AI model runtime is isolated in `GemmaRuntime` so Core stays MLX-free; don't collapse it.** All AI *logic* (the `LLMRuntime` seam, `AICommandExecutor`, tasks, selection, canvas) lives in `ThreeFingerSwitcherCore` and runs against a `StubLLMRuntime` under `swift test`; only `GemmaRuntime` links MLX, and the app injects the real runtime at the seam in `main.swift`. Don't move MLX into Core (it would make `swift test` need the Metal toolchain) and don't add model calls to feature code directly — go through `LLMRuntime` so a future backend can replace Gemma. -- **`build-app.sh` must bundle MLX's `default.metallib`, and the app builds via `xcodebuild`, never a plain `swift build` assembly.** MLX ships its Metal shaders as a SwiftPM resource bundle (`mlx-swift_Cmlx.bundle`); if `build-app.sh` doesn't copy `*.bundle` into `Contents/Resources/`, the app launches but is **SIGKILL'd at first GPU use with no crash report** — looks like a hang/crash, is really a missing metallib. This cost a real debugging session. (See **B2** for the MLX/`xcodebuild` split.) -- **The AI preview canvas is resolved by a fresh four-finger *swipe*, not by lifting.** After the firing lift opens the canvas the fingers are already up, so a re-lift is a deliberate **no-op**; a fresh **down swipe commits/applies**, a **horizontal swipe discards**, an **up swipe is ignored**. The recognizer enters a one-shot *canvas-resolution mode* (`launcherCanvasResolutionActive`) that bypasses the normal launcher/switcher latch. Don't "fix" it back to lift-to-commit (that was the original design and it felt dead). - **The Dock window previews are mouse-driven and the peek *fronts the real window*.** This is the app's **only cursor-interactive surface** — `DockPreviewOverlay` sets `ignoresMouseEvents = false` (every other overlay is pass-through, non-key). Hovering a tab genuinely **activates** that window — SkyLight `setFront` handshake + `kAXMain`/the app's `kAXFocusedWindow` (the same reliable front the commit uses) — and **restores** the previously-front window on leave, because **macOS won't render fresh pixels for an off-screen window**, so there's no way to preview live content without bringing it forward (an `SCStream` version was tried and reverted). The tab thumbnail itself is a **single static capture taken ~0.5s after the window settles** (skipping the front-transition frame), not a continuous stream. Both the SkyLight handshake and the focus singletons are **skipped under Stage Manager** (they oscillate WindowManager — same landmine as the switcher raise). Full detail in CLAUDE.md's "Dock window previews" section. ### B4. How a gesture flows (mental model) @@ -215,8 +195,6 @@ INSTALL=1 ./scripts/build-app.sh # also install in place to /Appli **Launcher path (four-finger, when enabled):** the same `TouchEngine` → `GestureRecognizer` (latched to launcher mode) emits launcher intents (activate / item-step / context-step / end) → `AppCoordinator` drives `LauncherOverlayController` (the grid HUD + dwell-to-arm timer + charge-ring/haptics) over the `FavoritesStore` model; **lift below two contacts** ends the gesture and the controller fires the armed item (or dismisses) via `LaunchService` (which opens a new window in the current Space, runs the shortcut/script, opens the path/URL, or composes a preset). Meanwhile `ScrollEventTap` swallows the freed scroll while the overlay is open. -**AI command path (the AI band inside the launcher, when enabled):** scrubbing to an armed `aiCommand` item and lifting does **not** dismiss — `LauncherOverlayController` opens the **streaming preview canvas** (the overlay stays visible, never key) and `AICommandExecutor` runs the pipeline: `SelectionService` acquires the input (AX selected text → ⌘C-with-restore → clipboard → screen region for vision), `PromptTemplate` resolves the tokens, `ModelManager` hands back the resident `LLMRuntime` (the real one is `GemmaMLXRuntime`; tests use `StubLLMRuntime`), and tokens **stream into the canvas** live. Resolution is a fresh four-finger swipe (see the B3 canvas landmine): a **down swipe commits** — routing per the command's output (`replaceSelection`/paste for in-place edits, or `TaskDispatcher` → EventKit/project/tool/adapter sink for a background task, with an optional action-review first) — and a **horizontal swipe discards** (cancelling generation). The captured front app stays frontmost throughout. - ### B5. Tunables All in `AppSettings` (persisted, live-applied, editable on the Hub's feature pages): @@ -225,8 +203,7 @@ All in `AppSettings` (persisted, live-applied, editable on the Hub's feature pag - **Opt-ins (each gates a native-gesture relocation + its feature):** `manageVerticalGesture` (Space-row switching), `enableLauncher` (four-finger launcher), plus `manageSpacesRearrange` (keep Spaces in a fixed order). Each has an `is…Effective` gate in `AppCoordinator` — the feature only goes live once the relocation has actually taken runtime effect (post re-login), never merely when the flag is set this session. **Exception:** `keepClipboardHistory` is an opt-in with **no** gesture relocation and **no** `is…Effective` gate — it just starts/stops `ClipboardMonitor` (and gates injecting the Clipboard band), so it takes effect immediately with no re-login. - **Launcher tunables:** `launcherActivationThreshold`, `launcherStepDistance` (item step), `launcherContextStepDistance` (context-band step), and `dwellToArmDuration` (the dwell-to-fire delay). The launcher reuses the switcher's direction-inversion settings. - **Clipboard tunables:** `clipboardRecentWindow` (entries shown in the band), retention caps (`clipboardMaxCount` / `clipboardMaxBytes` / `clipboardMaxAgeDays`, pinned-exempt), `clipboardPollInterval`, `clipboardEdgeAcceleration` (edge auto-repeat ramp), `clipboardPinDistance` (deliberate pin/leave flick), `clipboardExcludedApps`, and `clipboardPaused`. -- **AI commands:** `aiCommandsEnabled` is a fourth opt-in (no gesture relocation, no `is…Effective` gate — like clipboard it takes effect immediately) that gates AI commands and unlocks the model download. AI commands persist as **first-class items inside `FavoritesStore` bands** (any band, movable — `AICommandStore`/`AICommandBandBuilder` are gone), chosen from `AICommandCatalog`; the per-command model selector resolves against `ModelRegistry`; `confirmBeforeRun` (per command) gates the action-review before a side-effecting task fires. AI commands reuse the launcher's activation/step/direction tunables; the swipe-to-resolve discard threshold reuses `clipboardPinDistance`. -- **Keyboard language:** `keyboardLanguageEnabled` is a fifth opt-in (no gesture, no `is…Effective` gate); `keyboardLanguagePerSiteEnabled` extends the learn/apply engine to per-host inside browsers, and `keyboardLanguageAllowBrowserControl` opts into the Apple Events host reader (Accessibility is the default). The learned context-key → input-source map lives in `KeyboardLanguageStore`, with `keyboardLanguageDefaultSourceID` as the fallback. +- **Keyboard language:** `keyboardLanguageEnabled` is a fourth opt-in (no gesture, no `is…Effective` gate); `keyboardLanguagePerSiteEnabled` extends the learn/apply engine to per-host inside browsers, and `keyboardLanguageAllowBrowserControl` opts into the Apple Events host reader (Accessibility is the default). The learned context-key → input-source map lives in `KeyboardLanguageStore`, with `keyboardLanguageDefaultSourceID` as the fallback. --- @@ -235,5 +212,5 @@ All in `AppSettings` (persisted, live-applied, editable on the Hub's feature pag - **GPL-3.0** — see `LICENSE` and `NOTICE`. The window-raising/Space technique (the private `_AXUIElementGetWindow`, the remote-token brute force, the SkyLight front/key byte protocol, the CGS Space enumeration) is adapted from **[AltTab](https://github.com/lwouis/alt-tab-macos)** (GPL-3), which is why this project is GPL-3. - Raw multitouch via **[OpenMultitouchSupport](https://github.com/Kyome22/OpenMultitouchSupport)** (Kyome, MIT), wrapping the private `MultitouchSupport.framework`. -If you (Claude) end up extending this, keep the spec in `openspec/specs/` honest, keep the 848 tests green (`swift test` for Core; `xcodebuild` for the MLX app target), and respect the landmines in **B3** — they each cost a real debugging session to learn. +If you (Claude) end up extending this, keep the spec in `openspec/specs/` honest, keep the 805 tests green (`swift test`), and respect the landmines in **B3** — they each cost a real debugging session to learn. diff --git a/Resources/Info.plist b/Resources/Info.plist index 298a105..f88caf3 100644 --- a/Resources/Info.plist +++ b/Resources/Info.plist @@ -28,22 +28,6 @@ NSInputMonitoringUsageDescription Reads raw trackpad touches to detect the three-finger swipe. - - NSCalendarsFullAccessUsageDescription - Creates calendar events from text when you run an "add to calendar" AI command. - - NSRemindersFullAccessUsageDescription - Creates reminders from text when you run an "add to reminders" AI command. - - NSContactsUsageDescription - Creates a contact card from text when you run a "new contact" AI command. - - NSMicrophoneUsageDescription - Hears you only while you hold the push-to-talk key, to talk with the on-device assistant. - - NSSpeechRecognitionUsageDescription - Transcribes your push-to-talk speech on this Mac. Audio never leaves the device. NSLocalNetworkUsageDescription Connects directly to your paired iPhone on your local network to move clipboard items and files between your devices. diff --git a/Sources/GemmaRuntime/BatchedGemmaMLXRuntime.swift b/Sources/GemmaRuntime/BatchedGemmaMLXRuntime.swift deleted file mode 100644 index e12055d..0000000 --- a/Sources/GemmaRuntime/BatchedGemmaMLXRuntime.swift +++ /dev/null @@ -1,523 +0,0 @@ -// BatchedGemmaMLXRuntime — the multi-stream (continuous-batching) Gemma 4 conformer. -// -// `ai-batched-runtime-and-context`, design D1/D2/D3/D9. It wraps the proven single-session -// `GemmaMLXRuntime` (composition — `GemmaMLXRuntime` is `final`) for the single-session -// `generate`/`structured`/`chat` paths (vision, reasoning channels, structured output behave exactly -// as today) and adds the `BatchedLLMRuntime` surface: `maxConcurrentStreams` (RAM-derived via Core's -// `ConcurrencyBudget`) and `batchStep` (multiplex K sessions over ONE resident weight graph, de-mux -// tokens by `AgentSessionID`). It is the new resident runtime the provisioner returns. -// -// MLX-linked → `xcodebuild` compile-verify ONLY. The agent never builds/signs/installs the `.app`. -// Live correctness (no cross-stream KV bleed, masking, prefill interleave) is USER-RUN-VERIFY in a -// stable-signed build (task 8.3); the agent compiles the genuine kernel — NOT a serialized stand-in. -// -// THE "ONE WEIGHT READ" MODEL (design D2). The vendored `Gemma4Pipeline` exposes a per-stream graph -// (`context.model(input, cache:)` takes ONE per-stream KV cache). Decode is memory-bandwidth-bound: -// the ~17 GB weight set is a SINGLE resident set of `MLXArray`s inside the one resident `context.model`. -// We hold ONE resident model and step EVERY active stream's own KV cache against it inside ONE -// `container.perform` block per decode round — so the weights are materialized once for the round and -// every stream's per-step matmuls read those same already-resident arrays (MLX does not re-load weights -// per call; the per-stream cost is the per-token activation/KV work, not a fresh weight read). This is the -// continuous-batching win on this vendored surface: one resident graph, K interleaved per-stream caches, -// mid-flight admit/free, chunked prefill. (A single fused K-row forward pass would need a batched -// model entry the vendored `Gemma4LLMModel` does not export; the round-robin-over-one-resident-graph form -// is the faithful realization of D2 against the available surface and is the documented user-run-verify -// target for the fused variant.) - -import CoreGraphics -import Foundation -import ImageIO -import os -import ThreeFingerSwitcherCore -import Gemma4Swift -import MLX -import MLXRandom -import MLXLMCommon - -public final class BatchedGemmaMLXRuntime: LLMRuntime, BatchedLLMRuntime, @unchecked Sendable { - - static let log = Logger(subsystem: "ThreeFingerSwitcher", category: "BatchedGemmaMLXRuntime") - - private let base: GemmaMLXRuntime - /// Resident weight size (read once) — from the selected `ModelDescriptor.sizeBytes`. - private let weightBytes: Int64 - /// The model's architectural max context — the clamp ceiling for the budget's per-stream KV math. - private let modelMaxContextTokens: Int - /// The current context budget the K computation uses (set from the user's `agentContextTokens`; - /// defaults to a comfortable Balanced value). Configured before use (the `@unchecked Sendable` - /// contract); read synchronously by the nonisolated `maxConcurrentStreams` requirement. - public var contextTokens: Int - /// 8-bit KV halves the per-token KV cost (the compact-KV toggle). Configured before use. - public var compactKV: Bool - - /// The resident multimodal container that ALSO serves text-only decode (no image → pure text). It is - /// the single resident weight graph all batched streams step against. Built lazily on the first - /// `batchStep`, cached, and reused — so the weights load exactly once for the whole batch lifetime. - @MainActor private var container: ModelContainer? - - /// Prefix/prompt cache (design D3): the shared system+skills prefix's KV is computed once and reused - /// across turns/sessions, keyed by a hash of the prefix text. Invalidated when the hash changes. - @MainActor private var prefixCache: PrefixKVCache? - - public var capabilities: Set { base.capabilities } - - @MainActor - public convenience init(weightBytes: Int64, maxContextTokens: Int) { - self.init(base: GemmaMLXRuntime(), weightBytes: weightBytes, maxContextTokens: maxContextTokens) - } - - @MainActor - public init(base: GemmaMLXRuntime, weightBytes: Int64, maxContextTokens: Int) { - self.base = base - self.weightBytes = weightBytes - self.modelMaxContextTokens = maxContextTokens - self.contextTokens = min(8_192, max(1, maxContextTokens)) - self.compactKV = false - } - - // MARK: - Preparation (forwarded) - - @MainActor - public func prepare(model: Gemma4Pipeline.Model, progress: @escaping @Sendable (Double) -> Void) async throws { - try await base.prepare(model: model, progress: progress) - } - - // MARK: - Single-session paths (forwarded to the proven runtime) - - public func generate(_ request: LLMRequest) -> AsyncThrowingStream { - base.generate(request) - } - - public func structured( - _ request: LLMRequest, schema: StructuredSchema, as type: T.Type - ) async throws -> StructuredOutcome { - try await base.structured(request, schema: schema, as: type) - } - - public func generateText(_ request: LLMRequest) async throws -> String { - try await base.generateText(request) // response-only override on the base - } - - // MARK: - Budget / K - - /// K — derived from free unified memory at the current context length + KV-quant bits (design D4), - /// clamped ≥ 1 (the foreground session always fits). Recomputed each read so it tracks the live - /// context setting and memory. - public var maxConcurrentStreams: Int { - currentBudget().maxStreams(contextTokens: contextTokens) - } - - /// Build the pure `ConcurrencyBudget` from the live memory probe + the model's weight size + a - /// Gemma-class KV cost model. The KV constants are ESTIMATES (roadmap open question — ground them in - /// the real `config.json` at run-verify); the math itself is unit-tested in Core. The compact-KV - /// toggle halves the per-token KV cost (8-bit) so the budget honestly fits MORE streams when on. - private func currentBudget() -> ConcurrencyBudget { - let physical = Int64(ProcessInfo.processInfo.physicalMemory) - // bf16 KV ≈ 2 bytes/elem × head_dim × num_kv_heads; compact-KV (8-bit) halves it. The exact value - // belongs to the model config — these are representative Gemma-class numbers for the math. - let bf16PerTokenPerLayer = 2_048.0 // ~ 2 bytes × 128 head_dim × 8 kv-heads - let perTokenPerLayer = compactKV ? bf16PerTokenPerLayer / 2 : bf16PerTokenPerLayer - let kv = KVCacheCost( - slidingLayers: 40, // Gemma interleaves ~5 local : 1 global - globalLayers: 8, - slidingWindow: 1_024, - kvBytesPerTokenPerLayer: perTokenPerLayer) - return ConcurrencyBudget( - unifiedMemoryBytes: physical, - weightBytes: weightBytes, - reservedBytes: 6 * 1_000_000_000, // OS + app + graph-activation headroom - kv: kv) - } - - // MARK: - Batched surface (the continuous-batching decode loop, design D2/D3) - - /// Advance K sessions over ONE resident weight graph, de-multiplexing each emitted token to its - /// `AgentSessionID`. New requests are admitted mid-flight; a finished stream frees its slot for a - /// queued one on the very next round with the weights still resident (continuous, not static). A - /// stream that errors ends quietly without aborting the batch (per-stream failure isolation, D8). - public func batchStep(_ requests: [AgentSessionID: LLMChatRequest]) - -> AsyncThrowingStream<(AgentSessionID, Token), Error> { - AsyncThrowingStream { continuation in - let task = Task { @MainActor in - await self.runBatch(requests, continuation: continuation) - continuation.finish() - } - continuation.onTermination = { _ in task.cancel() } - } - } - - /// The continuous-batching engine. Holds one resident container, builds a per-stream `StreamState` - /// (own KV cache + classifier + budgets), and round-robins a single decode step across all active - /// streams inside one `perform` block per round so the resident weights stay materialized for the - /// whole round. Streams that hit EOS / their visible budget free their slot; nothing here re-reads the - /// weights between streams. - @MainActor - private func runBatch(_ requests: [AgentSessionID: LLMChatRequest], - continuation: AsyncThrowingStream<(AgentSessionID, Token), Error>.Continuation) async { - guard !requests.isEmpty else { return } - let container: ModelContainer - do { - container = try await ensureContainer() - } catch { - // Container load failed for the whole batch (no resident model at all) — every requested - // stream is terminal with the same mapped error; the rest of the batch has nothing to keep. - let mapped = Self.mapError(error) - for id in requests.keys { - continuation.yield((id, Token(Self.headline(mapped), isFinal: true))) - } - return - } - - // Build per-stream state (tokenize prompt, allocate the per-stream KV cache for the selected - // quant/window). A stream that fails to build is terminal but never aborts the others (D8). - var streams: [StreamState] = [] - for (id, request) in requests { - do { - let state = try await makeStream(id: id, request: request, container: container) - streams.append(state) - } catch { - continuation.yield((id, Token("", isFinal: true))) - } - } - guard !streams.isEmpty else { return } - - let eosTokenIds = Gemma4Processor.eosTokenIds - - // One round = one decode step for every still-running stream, inside one resident `perform` block. - // Prefill of a freshly admitted stream is chunked and interleaved (its first round prefills its - // prompt then samples its first token); thereafter it joins the single-token decode batch. - roundLoop: while !streams.isEmpty { - if Task.isCancelled { return } - // De-mux buffer for this round: (id, Token) pairs produced by every stream this step. - var emitted: [(AgentSessionID, Token)] = [] - // Mutate per-stream state inside the resident block; capture by index so we write back. - nonisolated(unsafe) let streamsRef = streams - nonisolated(unsafe) var producedFinal: [Int: Bool] = [:] - nonisolated(unsafe) var roundEmitted: [(AgentSessionID, Token)] = [] - - await container.perform { context in - for i in streamsRef.indices { - let s = streamsRef[i] - if s.finished { continue } - if Task.isCancelled { return } - - // Admit (prefill) this stream if it has not run yet: feed its prompt suffix through - // the resident graph to populate its KV cache, then take argmax of the last position. - if !s.prefilled { - s.prefill(context: context) - s.prefilled = true - } - - let nextToken = s.nextToken - if eosTokenIds.contains(nextToken) || s.visibleResponseEmitted >= s.maxVisible || s.steps >= s.hardCap { - s.finished = true - producedFinal[i] = true - continue - } - - // Classify + emit this stream's current token, then decode one step to advance it. - let decoded = context.tokenizer.decode(tokenIds: [Int(nextToken)]) - let (visible, channel) = s.classifier.classify(tokenId: nextToken, text: decoded) - if !visible.isEmpty { - roundEmitted.append((s.id, Token(visible, channel: channel))) - if channel == .response { s.visibleResponseEmitted += 1 } - } - s.advance(context: context) - } - } - - emitted = roundEmitted - for (id, token) in emitted { - continuation.yield((id, token)) - } - // Emit per-stream terminal markers for streams that finished this round and drop them. - for i in streams.indices where producedFinal[i] == true { - continuation.yield((streams[i].id, Token("", isFinal: true))) - } - streams.removeAll { $0.finished } - if streams.isEmpty { break roundLoop } - } - } - - /// Lazily build (and cache) the resident multimodal container from the already-downloaded files — the - /// SINGLE resident weight graph every batched stream steps against. Mirrors the base runtime's loader. - @MainActor - private func ensureContainer() async throws -> ModelContainer { - if let container { return container } - let c = try await base.ensureMultimodalContainerForBatching() - container = c - return c - } - - /// Build one stream's decode state: tokenize its conversation into the model's chat-templated prompt, - /// allocate its per-stream KV cache (quantized / rotating per the settings), reuse the shared prefix - /// KV when the prefix hash matches, and run an initial prefill lazily on the first decode round. - @MainActor - private func makeStream(id: AgentSessionID, request: LLMChatRequest, container: ModelContainer) async throws -> StreamState { - let reasoning = request.reasoning - // This turn's images (design D2: a turn may carry MULTIPLE). The full array is forwarded; each gets - // its own `<|image|>` placeholder prepended to the latest user turn so the model splices each - // image's features at its own run of image tokens (real multi-image inference is run-verify). - let turnImages = request.effectiveImages - - // Assemble the chat-templated token ids for this turn (multi-turn messages flattened to roles). For - // a vision turn, prepend one "<|image|>" placeholder per image to the LAST user message's content. - let lastUserIndex = request.messages.lastIndex(where: { $0.role == .user }) - let messages: [[String: any Sendable]] = request.messages.enumerated().map { idx, msg in - var content = msg.text - if idx == lastUserIndex, !turnImages.isEmpty { - content = String(repeating: "<|image|>\n", count: turnImages.count) + content - } - return ["role": Self.templateRole(msg.role), "content": content] - } - let additionalContext: [String: any Sendable] = ["enable_thinking": reasoning] - let baseTokenIds: [Int] = try await container.perform { context in - try context.tokenizer.applyChatTemplate(messages: messages, tools: nil, additionalContext: additionalContext) - } - - // Expand each "<|image|>" placeholder to boi + image_token×N + eoi, and preprocess the images into a - // stacked [N, C, H, W] pixel array, mirroring the single-session base runtime (design D2). - let tokenIds: [Int] - let pixelValues: MLXArray? - if turnImages.isEmpty { - tokenIds = baseTokenIds - pixelValues = nil - } else { - tokenIds = Self.expandImagePlaceholders(baseTokenIds, numImageTokens: GemmaMLXRuntime.numImageTokens) - pixelValues = try Self.preprocessImages(turnImages, maxSoftTokens: GemmaMLXRuntime.numImageTokens) - } - - // Per-stream KV cache selection (design D3/D6): - // • compact-KV ON → 8-bit QuantizedKVCache (longer context, same RAM, near-lossless). - // • background unbounded thread → rotating fixed-window (4-bit) — bounded GPU cache. - // • else → the model's native bf16 cache. - let cacheKind = self.cacheKind(for: request) - let cache: [KVCache] = await container.perform { context in - Self.makeCache(kind: cacheKind, model: context.model) - } - - // Prefix/prompt caching (design D3): the shared system prefix (the leading `.system` messages — - // the system preamble + the skills/memory TOC) is identical across turns and often across - // sessions. Compute its hash; when it matches the cached prefix the turn's prefill can skip the - // shared prefix and only prefill the turn-specific suffix. The hash key makes invalidation - // deterministic — a changed prefix (a skill toggled mid-session) re-prefills once. - let prefixText = request.messages.prefix(while: { $0.role == .system }).map(\.text).joined(separator: "\n") - let prefixHash = prefixText.isEmpty ? nil : prefixText.hashValue - let reusePrefix = prefixHash != nil && prefixHash == prefixCache?.hash - if let prefixHash, !reusePrefix { - // Record the (newly seen) shared prefix so subsequent turns/sessions reuse it. The prefix KV - // itself is populated by the first stream's prefill against the resident graph. - prefixCache = PrefixKVCache(hash: prefixHash, cache: cache, length: prefixText.count) - } - - let maxVisible = max(1, request.parameters.maxTokens) - return StreamState( - id: id, - tokenIds: tokenIds, - cache: cache, - temperature: Float(request.parameters.temperature), - dropThinking: !reasoning, - maxVisible: maxVisible, - hardCap: maxVisible * 3, - reusesCachedPrefix: reusePrefix, - pixelValues: pixelValues) - } - - /// Expand each `<|image|>` placeholder token into boi + image_token×N + eoi (design D2), mirroring the - /// single-session base runtime so the batched vision prompt matches the model's expected layout. - private static func expandImagePlaceholders(_ baseTokenIds: [Int], numImageTokens: Int) -> [Int] { - let imageTokenId = Int(Gemma4Processor.imageTokenId) - let boiTokenId = Int(Gemma4Processor.boiTokenId) - let eoiTokenId = Int(Gemma4Processor.eoiTokenId) - var expanded: [Int] = [] - expanded.reserveCapacity(baseTokenIds.count + numImageTokens + 2) - for tid in baseTokenIds { - if tid == imageTokenId { - expanded.append(boiTokenId) - for _ in 0 ..< numImageTokens { expanded.append(imageTokenId) } - expanded.append(eoiTokenId) - } else { - expanded.append(tid) - } - } - return expanded - } - - /// Preprocess a turn's images into a stacked `[N, C, H, W]` pixel array (design D2). Each PNG `Data` is - /// decoded + processed into `[1, C, H, W]`, then concatenated along the batch axis. A decode/preprocess - /// failure maps to a clean `RuntimeError` (never raw OS text). Real multi-image inference is run-verify. - private static func preprocessImages(_ images: [Data], maxSoftTokens: Int) throws -> MLXArray? { - guard !images.isEmpty else { return nil } - var perImage: [MLXArray] = [] - perImage.reserveCapacity(images.count) - for image in images { - guard let source = CGImageSourceCreateWithData(image as CFData, nil), - let cgImage = CGImageSourceCreateImageAtIndex(source, 0, nil) else { - throw RuntimeError.modelLoadFailed(detail: "Could not decode the captured image (PNG).") - } - do { - perImage.append(try Gemma4ImageProcessor.processImage(cgImage, maxSoftTokens: maxSoftTokens)) - } catch { - throw RuntimeError.modelLoadFailed(detail: String(describing: error)) - } - } - return perImage.count == 1 ? perImage[0] : concatenated(perImage, axis: 0) - } - - /// Which per-stream KV cache to allocate. A background (parked-thread) request opts into the rotating - /// fixed-window 4-bit cache so an unbounded thread can never blow the KV budget (design D3); a - /// compact-KV foreground request uses 8-bit; otherwise bf16. - private func cacheKind(for request: LLMChatRequest) -> CacheKind { - if request.parameters.maxTokens > contextTokens { return .rotatingWindow(maxSize: contextTokens, bits: 4) } - if compactKV { return .quantized(bits: 8) } - return .standard - } - - private enum CacheKind { case standard; case quantized(bits: Int); case rotatingWindow(maxSize: Int, bits: Int) } - - /// Allocate the per-stream KV cache array of the chosen kind against the resident model's layers. - /// Called inside the container's `perform` block (not main-actor isolated). - private static func makeCache(kind: CacheKind, model: any LanguageModel) -> [KVCache] { - let base = model.newCache(parameters: nil) - switch kind { - case .standard: - return base - case let .quantized(bits): - // 8-bit (or 4-bit) quantized KV per layer — halves/quarters the KV bytes (design D3). - return base.map { layer in - if let simple = layer as? KVCacheSimple { - return simple.toQuantized(groupSize: 64, bits: bits) as KVCache - } - return layer - } - case let .rotatingWindow(maxSize, _): - // Fixed-window ring buffer per layer — bounded KV regardless of thread length (design D3). The - // window is the rotating backstop distinct from conversation-runtime compaction. - return base.map { _ in RotatingKVCache(maxSize: max(64, maxSize), keep: 4) as KVCache } - } - } - - /// Map an `AgentRole` to the chat-template role string. - private static func templateRole(_ role: AgentRole) -> String { - switch role { - case .system: return "system" - case .user: return "user" - case .assistant: return "assistant" - case .tool: return "tool" - } - } - - // MARK: - Prefix / prompt caching (design D3) - - /// The shared system+skills prefix's cached KV, keyed by a hash of the prefix text. A turn whose prompt - /// begins with this exact prefix reuses the cached KV and only prefills the turn-specific suffix; a - /// changed prefix (different hash) invalidates and re-prefills once. Stored alongside the resident - /// container. (Engaged by the decode loop when a stream's tokens share the cached prefix; the hash key - /// makes invalidation deterministic.) - final class PrefixKVCache { - let hash: Int - let cache: [KVCache] - let length: Int - init(hash: Int, cache: [KVCache], length: Int) { - self.hash = hash - self.cache = cache - self.length = length - } - } - - // MARK: - Error mapping (design D8) - - /// Map an MLX/OS/OOM failure to the shared `RuntimeError` taxonomy at the conformer boundary. A stream - /// error is per-stream terminal — surfaced via the clean headline only (raw text → logs / details). - static func mapError(_ error: Error) -> RuntimeError { - if let runtime = error as? RuntimeError { return runtime } - if error is CancellationError { return .cancelled } - Self.log.error("batch: stream error: \(String(describing: error), privacy: .public)") - return .modelLoadFailed(detail: String(describing: error)) - } - - /// The clean, user-facing headline for a mapped error (never raw error text in a headline). - static func headline(_ error: RuntimeError) -> String { - error.errorDescription ?? "The model could not complete this request." - } -} - -// MARK: - Per-stream decode state - -/// One batched stream's mutable decode state: its tokenized prompt, its OWN KV cache (the per-stream -/// memory cost — quantized / rotating / bf16 per the settings), its channel classifier, and its decode -/// budgets/cursor. A reference type so the resident `perform` block can mutate it in place and write the -/// advance back without copying the (non-Sendable) MLX cache out. -final class StreamState { - let id: AgentSessionID - let tokenIds: [Int] - nonisolated(unsafe) let cache: [KVCache] - let temperature: Float - let maxVisible: Int - let hardCap: Int - /// This turn's preprocessed image pixels (design D2) — `[N, C, H, W]` stacked across the turn's - /// images, or nil for a text-only stream. Injected onto the multimodal model right before the prefill - /// forward pass so its features scatter at this stream's `<|image|>` token positions. - nonisolated(unsafe) let pixelValues: MLXArray? - /// Whether this stream shares the runtime's cached system+skills prefix (design D3) — a diagnostic - /// signal that the turn could reuse the prefix KV rather than re-prefilling it. - let reusesCachedPrefix: Bool - - var prefilled = false - var finished = false - var steps = 0 - var visibleResponseEmitted = 0 - var nextToken: Int32 = 0 - var classifier: GemmaMLXRuntime.ChannelClassifier - - init(id: AgentSessionID, tokenIds: [Int], cache: [KVCache], temperature: Float, - dropThinking: Bool, maxVisible: Int, hardCap: Int, reusesCachedPrefix: Bool = false, - pixelValues: MLXArray? = nil) { - self.id = id - self.tokenIds = tokenIds - self.cache = cache - self.temperature = temperature - self.maxVisible = maxVisible - self.hardCap = hardCap - self.reusesCachedPrefix = reusesCachedPrefix - self.pixelValues = pixelValues - self.classifier = GemmaMLXRuntime.ChannelClassifier(dropThinking: dropThinking) - } - - /// Prefill the full prompt into the per-stream KV cache and sample the first generated token. Run once - /// on the stream's first decode round (chunked-prefill seam — the prompt feeds in one block here). - /// Runs inside the resident container's `perform` block (not main-actor isolated). For a vision turn, - /// the stream's pixels are injected onto the model immediately before the forward pass so they scatter - /// at this prompt's `<|image|>` token positions (design D2; real multi-image inference is run-verify). - func prefill(context: ModelContext) { - if reusesCachedPrefix { - BatchedGemmaMLXRuntime.log.debug("batch: stream reuses cached system+skills prefix (D3)") - } - if let pixelValues, let model = context.model as? Gemma4MultimodalLLMModel { - model.pendingPixelValues = pixelValues - } - let input = MLXArray(tokenIds.map { Int32($0) }).reshaped(1, -1) - let logits = context.model(input, cache: cache) - nextToken = sample(logits[0..., logits.dim(1) - 1, 0...]) - } - - /// Decode one step: feed the current `nextToken` through the resident graph against THIS stream's - /// cache, sample the next token, advance the cursor. The resident weights are already materialized for - /// the round — this is the per-token activation/KV work, not a fresh weight read. - func advance(context: ModelContext) { - let input = MLXArray([nextToken]).reshaped(1, 1) - let logits = context.model(input, cache: cache) - nextToken = sample(logits[0..., 0, 0...]) - steps += 1 - } - - /// Greedy at temperature≈0, categorical-sampled otherwise (mirrors the base runtime's manual loop). - private func sample(_ row: MLXArray) -> Int32 { - if temperature <= 0.01 { - return argMax(row, axis: -1).item(Int32.self) - } - let scaled = row / temperature - let probs = softmax(scaled, axis: -1) - return MLXRandom.categorical(MLX.log(probs)).item(Int32.self) - } -} diff --git a/Sources/GemmaRuntime/CloudVideoRuntime.swift b/Sources/GemmaRuntime/CloudVideoRuntime.swift deleted file mode 100644 index 871b625..0000000 --- a/Sources/GemmaRuntime/CloudVideoRuntime.swift +++ /dev/null @@ -1,175 +0,0 @@ -// CloudVideoRuntime — the concrete CLOUD-escalation VIDEO `MediaRuntime` over a hosted video API -// (LTX Studio / equivalent) (ai-video-animation-generation §5.3, design D1/D2/D7). -// -// FLAGGED: user xcodebuild + stable-signed build. -// -// This is the DEFAULT video backend behind the `MediaRuntime` seam — a conformer, NOT a new seam (design -// D1), exactly as `GemmaMLXRuntime` is the chat seam's backend and `MFluxImageRuntime` is the image -// backend. It does real network IO (`URLSession`: upload the prompt + optional seed frame, poll render -// progress, fetch the finished file), so it is `xcodebuild` COMPILE-VERIFY ONLY here: the agent never -// builds/signs/installs the `.app`. REAL correctness — a real clip, real upload, real poll latency, real $ -// spend — is the USER's stable-signed run-verify (task 5.3). -// -// IT SPENDS MONEY + UPLOADS BYTES (design D3): the budget cap (`RollingVideoBudget`) + the `.dangerous` -// confirm gate live UPSTREAM in the sink/contributor — this runtime is the side-effecting executor invoked -// AFTER approval + after a budget unit was consumed. A launch that fails here surfaces a thrown -// `MediaError`; the contributor REFUNDS the spend (the cap stays honest, spec "A failed cloud launch -// refunds its spend"). This runtime does NOT itself touch the ledger. -// -// ERROR TAXONOMY (design D7 / task 8.1): every vendor/OS failure — `NSURLError` (offline / timeout / DNS), -// an HTTP 4xx/5xx, a malformed render response, a file-write IO error — is mapped INTO `MediaError` AT THIS -// BOUNDARY (Core stays network-free). Feature/UI code only ever sees `MediaError`, surfaced through -// `AIError.message(for:)` as a clean bounded headline (raw vendor text only in logs / opt-in copyable -// details — never a headline, never an `NSAlert`). A gen that fails to land is `.failed`; CANCELLATION is a -// stopped stream, NOT a failure (design D10): the stream finishes WITHOUT a `.finished` and WITHOUT throwing. - -import Foundation -import os -import ThreeFingerSwitcherCore - -public final class CloudVideoRuntime: MediaRuntime, @unchecked Sendable { - - static let log = Logger(subsystem: "ThreeFingerSwitcher", category: "CloudVideoRuntime") - - /// VIDEO only — never `.image` (the router never routes a still here, design D1). - public let capabilities: Set = [.video] - - /// The hosted API base (the render submit + poll + fetch endpoints derive from it). - private let endpoint: URL - /// The API credential, injected (never logged, never in a headline / audit summary). - private let apiKey: String - /// Where the fetched clip is written (the gallery root — becomes a Files-band `.fileEntry`). - private let outputDirectory: URL - /// The poll cadence + ceiling (so a stuck remote render fails clean, never hangs forever). - private let pollInterval: TimeInterval - private let pollCeiling: TimeInterval - private let session: URLSession - - public init(endpoint: URL, - apiKey: String, - outputDirectory: URL, - pollInterval: TimeInterval = 3, - pollCeiling: TimeInterval = 600, - session: URLSession = .shared) { - self.endpoint = endpoint - self.apiKey = apiKey - self.outputDirectory = outputDirectory - self.pollInterval = pollInterval - self.pollCeiling = pollCeiling - self.session = session - } - - // MARK: - Generation (upload → poll → fetch — task 5.3) - - public func generate(_ request: MediaRequest) -> AsyncThrowingStream { - let outputDirectory = self.outputDirectory - return AsyncThrowingStream { continuation in - let task = Task { - do { - // (1) SUBMIT — upload the prompt + optional seed PNG (img2video first frame). A present - // seed that isn't a decodable PNG is a clean `MediaError.seedInvalid` (never uploaded). - if let seed = request.seed, !MediaSeedValidation.isDecodablePNG(seed) { - continuation.finish(throwing: MediaError.seedInvalid) - return - } - let jobID = try await self.submit(request) - - // (2) POLL — stream render progress as `.step(index:total:preview:)` until the remote - // job reports finished; honor cancellation between polls (stopped stream, no .finished). - let fileURL = try await self.poll(jobID: jobID, into: continuation) - - // (3) FETCH + WRITE — download the finished clip to the gallery; a write IO failure maps - // to `MediaError.outputWriteFailed` (raw OS reason → copyable details, never a headline). - let asset = try await self.fetch(from: fileURL, request: request, - outputDirectory: outputDirectory) - continuation.yield(.finished(asset)) - continuation.finish() - } catch is CancellationError { - continuation.finish() // discard → stopped stream, NOT a failure - } catch let e as MediaError { - continuation.finish(throwing: e) - } catch { - continuation.finish(throwing: Self.map(error)) // NSURLError / HTTP → MediaError boundary - } - } - continuation.onTermination = { _ in task.cancel() } - } - } - - // MARK: - FLAGGED native network steps (real correctness: user's signed build) - - /// FLAGGED: POST the prompt + optional seed PNG to the hosted render endpoint, return the job id. - /// Maps a non-2xx / `NSURLError` into `MediaError` via `map(_:)`. Native-only (real URLSession IO). - private func submit(_ request: MediaRequest) async throws -> String { - // Real implementation outline (native-only): - // var req = URLRequest(url: endpoint.appendingPathComponent("renders")) - // req.httpMethod = "POST" - // req.setValue("Bearer \(apiKey)", forHTTPHeaderField: "Authorization") // never logged - // let body = RenderSubmit(prompt: request.prompt, seedPNG: request.seed, - // durationMs: request.parameters.durationMs, size: request.parameters.size) - // req.httpBody = try JSONEncoder().encode(body) - // let (data, resp) = try await session.data(for: req) // NSURLError → catch → map - // try Self.checkStatus(resp) // HTTP 4xx/5xx → MediaError - // return try JSONDecoder().decode(RenderSubmitResponse.self, from: data).id - throw MediaError.generationFailed( - headline: "Cloud video generation is verified only in the user's stable-signed build.") - } - - /// FLAGGED: poll the job until finished (or the ceiling), yielding `.step` progress. Honors cancellation - /// between polls. Returns the finished-clip URL. Native-only. - private func poll(jobID: String, - into continuation: AsyncThrowingStream.Continuation) async throws -> URL { - // Real implementation outline (native-only): - // let deadline = Date().addingTimeInterval(pollCeiling) - // while Date() < deadline { - // try Task.checkCancellation() // cancel → stopped stream - // let status = try await fetchStatus(jobID) // NSURLError/HTTP → map - // continuation.yield(.step(index: status.step, total: status.total, preview: status.previewPNG)) - // if let url = status.finishedURL { return url } - // try await Task.sleep(nanoseconds: UInt64(pollInterval * 1_000_000_000)) - // } - // throw MediaError.generationFailed(headline: "The video took too long and was stopped.") - _ = continuation - throw MediaError.cloudUnavailable - } - - /// FLAGGED: download the finished clip + write it into the gallery, returning the `MediaAsset` - /// (kind `.video`, `durationMs` from the request). A write IO failure → `MediaError.outputWriteFailed`. - /// Native-only. - private func fetch(from fileURL: URL, request: MediaRequest, outputDirectory: URL) async throws -> MediaAsset { - // Real implementation outline (native-only): - // let (tmp, resp) = try await session.download(from: fileURL) - // try Self.checkStatus(resp) - // let dest = outputDirectory.appendingPathComponent("\(UUID().uuidString).mp4") - // do { try FileManager.default.moveItem(at: tmp, to: dest) } - // catch { throw MediaError.outputWriteFailed(detail: String(describing: error)) } - // return MediaAsset(url: dest, kind: .video, - // width: request.parameters.size.width, height: request.parameters.size.height, - // durationMs: request.parameters.durationMs) - throw MediaError.outputWriteFailed(detail: nil) - } - - // MARK: - Boundary mapping (NSURLError / HTTP → MediaError) - - /// Map a vendor/OS network error into the shared `MediaError` taxonomy AT THE BOUNDARY (design D7). - /// Connectivity → `.cloudUnavailable`; an auth/HTTP status rides through the shared classifier's clean - /// strings into `.generationFailed`. Raw text goes ONLY to the logger, never a headline. - static func map(_ error: Error) -> MediaError { - let ns = error as NSError - log.error("cloud video network failure: \(String(describing: error), privacy: .public)") - if ns.domain == NSURLErrorDomain { - switch ns.code { - case NSURLErrorNotConnectedToInternet, NSURLErrorNetworkConnectionLost, - NSURLErrorCannotConnectToHost, NSURLErrorCannotFindHost, - NSURLErrorDNSLookupFailed, NSURLErrorTimedOut: - return .cloudUnavailable - default: - // Reuse the shared translator's clean per-status string (no raw NSError text in the headline). - let headline = AIError.message(for: error).headline - return .generationFailed(headline: headline) - } - } - let headline = AIError.message(for: error).headline - return .generationFailed(headline: headline) - } -} diff --git a/Sources/GemmaRuntime/GemmaMLXRuntime.swift b/Sources/GemmaRuntime/GemmaMLXRuntime.swift deleted file mode 100644 index 7ab9454..0000000 --- a/Sources/GemmaRuntime/GemmaMLXRuntime.swift +++ /dev/null @@ -1,740 +0,0 @@ -// GemmaMLXRuntime — the real, in-process Gemma 4 conformer of Core's `LLMRuntime` seam. -// -// It wraps a `Gemma4Pipeline` (from the `Gemma4Swift` package → mlx-swift) and bridges it to the -// Core protocol that all feature code depends on. This file links MLX (Metal shaders), so it lives -// in the isolated `GemmaRuntime` target and is only ever built via `xcodebuild` — Core and the test -// target never see it (design D1/D7). - -import CoreGraphics -import Foundation -import ImageIO -import os -import ThreeFingerSwitcherCore -import Gemma4Swift -import MLX -import MLXRandom -import MLXLMCommon - -/// In-process Gemma 4 (MLX) implementation of `LLMRuntime`. -/// -/// CHANNEL-TAGGED STREAMING. Gemma 4 has a thinking ("thought") channel and a final ("response") channel, -/// delimited in-band by control tokens (`<|think|>`, `<|channel>` + a "thought"/"response" name, ``). -/// This runtime classifies every generated token into Core's `TokenChannel` (`.thinking`/`.response`) and -/// emits `Token(text, channel:)` so the executor can split the two — streaming `.thinking` into the canvas's -/// collapsible section while committing only `.response`. Channel CONTROL tokens are consumed and yield no -/// visible text. Classification needs token-LEVEL access, which `Gemma4Pipeline.chatStream` (it yields decoded -/// `String` deltas, no tokenId) cannot provide — so any request that needs channels routes through a manual -/// generate loop with a per-token classifier (`ChannelClassifier`, replicating `Gemma4TokenFilter.filterToken`). -/// -/// ENABLING THINKING. Gemma 4 does NOT think by default: its chat template, at the generation prompt, emits an -/// empty closed thought channel (`<|channel>thought\n`) that suppresses reasoning UNLESS the template -/// variable `enable_thinking` is true (which also injects `<|think|>` into the leading system turn). So when a -/// request asks for reasoning we pass `["enable_thinking": true]` through `applyChatTemplate(...additionalContext:)`; -/// otherwise we leave it off and the model produces response-only. -/// -/// ROUTING. A plain text request with reasoning OFF keeps the fast `Gemma4Pipeline.chatStream(...)` path -/// unchanged (every chunk is `.response`). A request that is EITHER vision (image present) OR reasoning ON runs -/// the manual loop. The manual loop covers both text (no image) and vision (image → `Gemma4ImageProcessor` → -/// pixelValues + 280 soft tokens; one `<|image|>` placeholder expanded to `boi + image×280 + eoi`; -/// `pendingPixelValues` set on the `Gemma4MultimodalLLMModel`) on the SEPARATE, lazily-loaded multimodal -/// `ModelContainer` — `chatStream` has no image-bearing or token-level entry point. A vision-capable runtime thus -/// holds two resident graphs (text pipeline + multimodal container), acceptable per the spec's high-end-hardware / -/// ample-unified-memory target. When reasoning is OFF on the manual path (a vision command with reasoning off), -/// thinking tokens are dropped (response-only, like `Gemma4TokenFilter` mode `.disabled`). Video/audio are out of -/// scope for v1: `LLMRequest` carries at most one optional PNG `image`, so only the image branch runs. -/// -/// CANCELLATION: discarding the consumer stops token DELIVERY and surfaces `.cancelled` promptly. For the -/// fast `chatStream` text path it does NOT tear down the underlying MLX generation — the vendored -/// `Gemma4Pipeline.chatStream` wrapper task doesn't forward stream termination, so GPU work runs on to -/// `maxTokens` in the background. (A future fix would drive the `ChatSession` stream directly, which cancels -/// properly.) The manual loop checks `Task.isCancelled` between decode steps and finishes with `.cancelled` -/// promptly (throwing `RuntimeError.cancelled`). -/// -/// `@unchecked Sendable`: the wrapped `@MainActor`-isolated pipeline (and the lazily-loaded multimodal -/// container) are only ever touched on the main actor (every hop into them is `@MainActor`); the pipeline -/// reference is immutable after init and the vision state is mutated only under `@MainActor`. -public final class GemmaMLXRuntime: LLMRuntime, @unchecked Sendable { - - /// The MLX-backed pipeline. `@MainActor` per `Gemma4Pipeline`'s own isolation. - private let pipeline: Gemma4Pipeline - - /// v1: text + vision (image-only). The text path runs on `pipeline.chatStream`; the vision path runs - /// on a separate lazily-loaded multimodal `ModelContainer` (see the type doc). - public let capabilities: Set = [.text, .vision] - - /// The on-disk model directory resolved + loaded during `prepare(...)`. The vision path lazily loads - /// its own multimodal container from these SAME already-downloaded files. `nil` until a successful - /// `prepare`; a vision request before that throws `.modelMissing`. Touched only on the main actor. - @MainActor private var loadedModelDir: URL? - - /// The lazily-loaded, resident multimodal container for the vision path (built on first vision request - /// from `loadedModelDir`, then cached). Separate from the text pipeline because `Gemma4Pipeline` loads - /// the text-only graph and keeps its container private. Touched only on the main actor. - @MainActor private var multimodalContainer: ModelContainer? - - /// Breadcrumbs to the unified log so a hard SIGKILL (hang→force-quit, OOM, Metal abort) that leaves - /// NO `.ips` crash report is still diagnosable: `log show --predicate 'subsystem=="ThreeFingerSwitcher"'` - /// reveals exactly which phase (download / load / generate) the process died in. - static let log = Logger(subsystem: "ThreeFingerSwitcher", category: "GemmaMLXRuntime") - - /// Create a runtime over a fresh pipeline. Use `prepare(...)` to download/load weights. - /// `@MainActor` so it can construct the `@MainActor`-isolated `Gemma4Pipeline`. - @MainActor - public convenience init() { - self.init(pipeline: Gemma4Pipeline()) - } - - /// Wrap an existing pipeline (it may already be loaded). Use `prepare(...)` to load weights. - @MainActor - public init(pipeline: Gemma4Pipeline) { - self.pipeline = pipeline - } - - // MARK: - Preparation (download + load) - - /// Download (if needed) and load `model` into the pipeline, reporting a 0…1 fraction. - /// - /// Download is delegated to `GemmaResumableDownloader`, which streams each weight/config file to a - /// `{dest}.part` on disk and resumes via HTTP `Range` after a network drop — so a flaky-wifi failure - /// mid-shard keeps its partial bytes and the next attempt picks up where it left off (the vendored - /// downloader had NO byte resume and failed permanently on `NSURLErrorDomain -1005`). It writes into - /// the exact cache layout `Gemma4ModelCache` reads, so once it returns we load with - /// `downloadIfNeeded: false`. `progress` is the downloader's 0…1 byte fraction. Called from - /// `GemmaRuntime.makeModelManager`'s provisioner. - @MainActor - public func prepare( - model: Gemma4Pipeline.Model, - progress: @escaping @Sendable (Double) -> Void - ) async throws { - if Task.isCancelled { throw RuntimeError.cancelled } - Self.log.notice("prepare: begin \(model.rawValue, privacy: .public)") - do { - // Cache dir = `/models/{org}/{model}` — exactly what `Gemma4ModelCache` reads (it - // exposes `modelsDirectory` publicly, so build the path off that instead of duplicating it). - var modelDir = Gemma4ModelCache.modelsDirectory - for part in model.rawValue.split(separator: "/") { - modelDir = modelDir.appendingPathComponent(String(part)) - } - - // Byte-resumable download (the downloader owns its own retry/backoff per file). - try await GemmaResumableDownloader.ensureModel(model, into: modelDir, progress: progress) - - // The cache is now complete on disk, so load straight from it (no further download). - // Load the TEXT-ONLY graph (multimodal: false) — vision/audio towers would only waste - // memory + load time on the ~17 GB model for this text-only runtime. - Self.log.notice("prepare: download complete → loading weights into MLX (this is the heavy step)…") - try await pipeline.load(model, multimodal: false, downloadIfNeeded: false) - // Remember the resolved on-disk dir so the vision path can lazily load its OWN multimodal - // container from these same files later (the text pipeline keeps its container private). - loadedModelDir = modelDir - Self.log.notice("prepare: model loaded and ready ✓") - } catch is CancellationError { - Self.log.notice("prepare: cancelled") - throw RuntimeError.cancelled - } catch let runtime as RuntimeError { - // Already in the shared taxonomy (e.g. an inner `.cancelled`) — log + rethrow unchanged. - Self.log.error("prepare: FAILED: \(String(describing: runtime), privacy: .public)") - throw runtime - } catch let download as Gemma4DownloadError { - // Map the vendored download-library error into the shared taxonomy HERE, at the boundary - // (design D6) — stop re-throwing it raw. The diagnostic log line is KEEP-as-is. - Self.log.error("prepare: FAILED: \(String(describing: download), privacy: .public)") - throw Self.runtimeError(for: download) - } catch { - // Any other failure is a model-load failure; carry the raw text as OPT-IN detail only - // (never the user-facing headline). The diagnostic log line is KEEP-as-is. - Self.log.error("prepare: FAILED: \(String(describing: error), privacy: .public)") - throw RuntimeError.modelLoadFailed(detail: String(describing: error)) - } - } - - /// Map the vendored `Gemma4DownloadError` into Core's shared `RuntimeError` taxonomy. Reference the - /// vendored type by its PUBLIC shape only — never edit it (it lives in `.build/checkouts`). For a - /// `.networkError`, inspect the wrapped `NSError` code to split a genuine offline state from a - /// transient server failure; HTTP statuses reuse `AIError`'s shared classifier so the boundary and - /// the translator never disagree. - static func runtimeError(for error: Gemma4DownloadError) -> RuntimeError { - switch error { - case let .networkError(_, underlying): - switch (underlying as NSError).code { - case NSURLErrorNotConnectedToInternet, // -1009 - NSURLErrorNetworkConnectionLost, // -1005 - NSURLErrorCannotConnectToHost, // -1004 - NSURLErrorCannotFindHost, // -1003 - NSURLErrorDNSLookupFailed, // -1006 - NSURLErrorTimedOut, // -1001 - NSURLErrorDataNotAllowed, // -1020 - NSURLErrorInternationalRoamingOff: // -1018 - return .offline - default: - return .serverUnavailable - } - case let .httpError(_, code): - return AIError.runtimeError(forHTTPStatus: code) ?? .serverUnavailable - case .apiFailed, .parseError, .noFilesFound: - return .serverUnavailable - case .cancelled: - return .cancelled - } - } - - // MARK: - Streaming text generation - - public func generate(_ request: LLMRequest) -> AsyncThrowingStream { - let prompt = request.prompt - let temperature = Float(request.parameters.temperature) - let maxTokens = request.parameters.maxTokens - let needsVision = request.requiresVision - let reasoning = request.reasoning - let pipeline = self.pipeline - - let images = request.images // design D2: a turn may carry MULTIPLE images - let needsAudio = request.requiresAudio - - return AsyncThrowingStream { continuation in - // Audio refusal contract (`add-voice-computer-use-agent`): the request seam carries audio - // bytes for the future audio-tower path (`Gemma4AudioProcessor`/`pendingAudioFeatures`, - // mirroring the vision integration), but THIS conformer does not feed them yet — refuse - // loudly rather than silently dropping the user's audio. Remove when the tower is wired. - if needsAudio { - continuation.finish(throwing: RuntimeError.unsupportedModality(.audio)) - return - } - // Manual-loop path: any request that needs channel classification — EITHER vision (the text - // pipeline's `chatStream` is text-only AND has no token-level entry point) OR reasoning ON - // (channels are delimited by in-band control tokens only visible at the token level). Streams - // channel-tagged tokens live; `enableThinking` is the request's `reasoning` flag (vision with - // reasoning off is response-only). All of the turn's image bytes are captured by value (PNG `Data`). - if needsVision || reasoning { - let task = Task { @MainActor in - await self.runManual(images: images, prompt: prompt, temperature: temperature, - maxTokens: maxTokens, enableThinking: reasoning, - continuation: continuation) - } - continuation.onTermination = { _ in task.cancel() } - return - } - // Fast path: plain text, no reasoning → keep `chatStream` unchanged (every chunk is `.response`). - let task = Task { @MainActor in - Self.log.notice("generate: begin (first call also materializes weights on the GPU)") - do { - let stream = try pipeline.chatStream( - prompt: prompt, - temperature: temperature, - maxTokens: maxTokens - ) - var count = 0 - for try await delta in stream { - // Honor cancellation promptly: a discarded consumer stops generation. - if Task.isCancelled { - continuation.finish(throwing: RuntimeError.cancelled) - return - } - if count == 0 { Self.log.notice("generate: first token received ✓") } - count += 1 - continuation.yield(Token(delta)) - } - if Task.isCancelled { - continuation.finish(throwing: RuntimeError.cancelled) - return - } - Self.log.notice("generate: finished, \(count, privacy: .public) chunks") - // A final empty terminal token lets consumers finalize without awaiting stream end. - continuation.yield(Token("", isFinal: true)) - continuation.finish() - } catch is CancellationError { - continuation.finish(throwing: RuntimeError.cancelled) - } catch { - Self.log.error("generate: FAILED: \(String(describing: error), privacy: .public)") - continuation.finish(throwing: error) - } - } - continuation.onTermination = { _ in task.cancel() } - } - } - - // MARK: - Channel classification (replicates Gemma4TokenFilter.filterToken) - - /// A small per-token state machine that splits Gemma 4's in-band channels (thinking vs response) the - /// SAME way the vendored `Gemma4TokenFilter.filterToken` does — but instead of filtering, it CLASSIFIES: - /// for every generated token it returns `(visibleText, channel)`, where channel-control tokens - /// (`<|think|>`, `<|channel>`, ``) and the channel-name text (`thought`/`response`) yield NO - /// visible text. The model emits `<|channel>` then a name selecting the channel; tokens inside `thought` - /// are `.thinking`, inside `response` (or outside any channel) are `.response`. - /// - /// `dropThinking`: when true (reasoning OFF on the manual path, e.g. a vision command with reasoning - /// off), thinking-channel text is suppressed (returned empty) so only the response survives — exactly - /// `Gemma4TokenFilter` mode `.disabled`. When false, thinking text rides through tagged `.thinking`. - /// Reference: `Gemma4TokenFilter.filterToken` and `Gemma4Processor.{thinkTokenId,channelStartTokenId, - /// channelEndTokenId}`. - struct ChannelClassifier { - private enum Channel { case none, thinking, response, detecting } - private var channel: Channel = .none - private var pendingName: String = "" - let dropThinking: Bool - - init(dropThinking: Bool) { self.dropThinking = dropThinking } - - /// Classify one generated token → `(visibleText, channel)`. `visibleText` is "" for control/name - /// tokens (and for thinking tokens when `dropThinking`); `channel` is the channel the token sits in. - mutating func classify(tokenId: Int32, text: String) -> (text: String, channel: TokenChannel) { - // Channel-control tokens — consumed, no visible text. - if tokenId == Gemma4Processor.channelStartTokenId { - channel = .detecting // <|channel> — next text names the channel - pendingName = "" - return ("", .response) - } - if tokenId == Gemma4Processor.channelEndTokenId { - channel = .none // — close the current channel - return ("", .response) - } - if tokenId == Gemma4Processor.thinkTokenId { - return ("", .response) // <|think|> — enables thinking; itself invisible - } - - // Detecting the channel name ("thought" vs "response") right after <|channel>. - if channel == .detecting { - pendingName += text - if pendingName.contains("thought") { - channel = .thinking; pendingName = ""; return ("", .thinking) - } else if pendingName.contains("response") { - channel = .response; pendingName = ""; return ("", .response) - } - // Accumulated too much without a match → treat as response (mirrors the filter's >20 guard). - if pendingName.count > 20 { - channel = .response - let buffered = pendingName - pendingName = "" - return (buffered, .response) - } - return ("", .response) // still buffering the name - } - - // Inside a channel (or none → response). - switch channel { - case .thinking: - return (dropThinking ? "" : text, .thinking) - case .response, .none: - return (text, .response) - case .detecting: - return ("", .response) // unreachable - } - } - } - - // MARK: - Manual generate loop (vision and/or reasoning) — channel-tagged streaming - - /// Number of soft tokens a single image expands to (the CLI's `numImageTokens`; `Gemma4ImageProcessor` - /// also defaults to 280 via `maxSoftTokens`). Kept here so the placeholder expansion and the processor - /// agree on one constant. `internal` so the batched runtime (same module) reuses the same value. - static let numImageTokens = 280 - - /// Lazily build (and cache) the resident multimodal container from the SAME already-downloaded files - /// that `prepare(...)` loaded. Mirrors the CLI's `loadLocalMultimodalModel(path:)`: - /// `register(multimodal: true)` + `loadModelContainer(from:using:)` with the pipeline's own tokenizer - /// loader. Throws `.modelMissing` if no model has been prepared yet. The container is multimodal-capable - /// but also serves text-only manual generation (no image → no pixel injection → pure text decode). - /// Internal accessor so the batched conformer (`BatchedGemmaMLXRuntime`) can share this runtime's - /// SINGLE resident multimodal container as the one resident weight graph for its decode loop — the - /// weights load exactly once and every batched stream steps its own KV cache against them. - @MainActor - func ensureMultimodalContainerForBatching() async throws -> ModelContainer { - try await ensureMultimodalContainer() - } - - @MainActor - private func ensureMultimodalContainer() async throws -> ModelContainer { - if let multimodalContainer { return multimodalContainer } - guard let dir = loadedModelDir else { - Self.log.error("manual: no model prepared — cannot load multimodal container") - throw RuntimeError.modelMissing - } - Self.log.notice("manual: loading multimodal container (first manual-path request; this is heavy)…") - do { - await Gemma4Registration.register(multimodal: true) - let container = try await loadModelContainer(from: dir, using: Gemma4TokenizerLoader()) - multimodalContainer = container - Self.log.notice("manual: multimodal container ready ✓") - return container - } catch is CancellationError { - throw RuntimeError.cancelled - } catch let runtime as RuntimeError { - throw runtime - } catch { - // Map any vendor/OS load failure into the shared taxonomy at this boundary (design D6). - Self.log.error("manual: multimodal load FAILED: \(String(describing: error), privacy: .public)") - throw RuntimeError.modelLoadFailed(detail: String(describing: error)) - } - } - - /// Run the manual (vision and/or reasoning) generation end-to-end, pumping channel-tagged tokens into - /// `continuation` LIVE as they are produced. Any thrown error is mapped to `RuntimeError` and used to - /// finish the stream; cancellation is NOT a failure (it finishes with `.cancelled`). - @MainActor - private func runManual( - images: [Data], - prompt: String, - temperature: Float, - maxTokens: Int, - enableThinking: Bool, - continuation: AsyncThrowingStream.Continuation - ) async { - do { - try await manualGenerate( - images: images, prompt: prompt, temperature: temperature, maxTokens: maxTokens, - enableThinking: enableThinking, continuation: continuation) - if Task.isCancelled { - continuation.finish(throwing: RuntimeError.cancelled) - return - } - // A final empty terminal token lets consumers finalize without awaiting stream end. - continuation.yield(Token("", isFinal: true)) - continuation.finish() - } catch is CancellationError { - continuation.finish(throwing: RuntimeError.cancelled) - } catch let runtime as RuntimeError { - Self.log.error("manual: FAILED: \(String(describing: runtime), privacy: .public)") - continuation.finish(throwing: runtime) - } catch { - Self.log.error("manual: FAILED: \(String(describing: error), privacy: .public)") - continuation.finish(throwing: RuntimeError.modelLoadFailed(detail: String(describing: error))) - } - } - - /// The manual decode path used for vision and/or reasoning. Builds + (for vision) expands the multimodal - /// prompt, optionally injects pixels, then runs a greedy/sampled generate loop feeding EVERY generated - /// token through `ChannelClassifier` and yielding `Token(visibleText, channel:)`. Replicates the CLI's - /// Describe loop but keeps token-level access so channels can be classified. `enableThinking` flips the - /// chat template's `enable_thinking` (Gemma 4 does NOT think by default — see the type doc) AND tells the - /// classifier whether to keep (`true`) or drop (`false`) thinking text. Honors Task cancellation between - /// decode steps (throws `RuntimeError.cancelled`). - @MainActor - private func manualGenerate( - images: [Data], - prompt: String, - temperature: Float, - maxTokens: Int, - enableThinking: Bool, - continuation: AsyncThrowingStream.Continuation - ) async throws { - if Task.isCancelled { throw RuntimeError.cancelled } - Self.log.notice("manual: begin (\(images.count, privacy: .public) image(s), thinking=\(enableThinking, privacy: .public))") - - let container = try await ensureMultimodalContainer() - if Task.isCancelled { throw RuntimeError.cancelled } - - // 1) For a vision request, each PNG `Data` → CGImage → pixelValues [1, C, H, W]. A turn may carry - // MULTIPLE images (design D2): process each, then stack along the batch axis into [N, C, H, W] — - // the multimodal model's forward pass already iterates `0..` token positions. Map a decode/preprocess failure to a clean taxonomy case - // (never raw OS text in a headline — the detail rides as opt-in copyable). - let numImageTokens = Self.numImageTokens - var pixelValues: MLXArray? = nil - if !images.isEmpty { - var perImage: [MLXArray] = [] - perImage.reserveCapacity(images.count) - for image in images { - guard let source = CGImageSourceCreateWithData(image as CFData, nil), - let cgImage = CGImageSourceCreateImageAtIndex(source, 0, nil) else { - Self.log.error("manual: could not decode image bytes") - throw RuntimeError.modelLoadFailed(detail: "Could not decode the captured image (PNG).") - } - do { - perImage.append(try Gemma4ImageProcessor.processImage(cgImage, maxSoftTokens: numImageTokens)) - } catch { - Self.log.error("manual: image preprocessing FAILED: \(String(describing: error), privacy: .public)") - throw RuntimeError.modelLoadFailed(detail: String(describing: error)) - } - } - // [1,C,H,W] × N → [N,C,H,W] (a single image is the N==1 case, identical to before). - pixelValues = perImage.count == 1 ? perImage[0] : concatenated(perImage, axis: 0) - Self.log.notice("manual: \(images.count, privacy: .public) image(s) preprocessed → \(pixelValues!.shape.description, privacy: .public)") - } - if Task.isCancelled { throw RuntimeError.cancelled } - - // 2) Build the chat-templated prompt. For vision, prepend ONE "<|image|>" placeholder PER image. To - // enable reasoning we pass `enable_thinking` through the template's additionalContext — Gemma 4's - // chat template otherwise injects an empty closed thought channel that suppresses thinking (see type doc). - let imageCount = images.count - let content = imageCount > 0 - ? String(repeating: "<|image|>\n", count: imageCount) + prompt - : prompt - let messages: [[String: any Sendable]] = [["role": "user", "content": content]] - let additionalContext: [String: any Sendable] = ["enable_thinking": enableThinking] - let baseTokenIds: [Int] = try await container.perform { context in - try context.tokenizer.applyChatTemplate( - messages: messages, tools: nil, additionalContext: additionalContext) - } - - // 3) For vision, expand EACH image placeholder to boi + image_token×N + eoi (like the CLI). With - // multiple placeholders the model splices each image's features at its own run of image tokens. - let imageTokenId = Int(Gemma4Processor.imageTokenId) - let boiTokenId = Int(Gemma4Processor.boiTokenId) - let eoiTokenId = Int(Gemma4Processor.eoiTokenId) - var expanded: [Int] = [] - if imageCount > 0 { - expanded.reserveCapacity(baseTokenIds.count + imageCount * (numImageTokens + 2)) - for tid in baseTokenIds { - if tid == imageTokenId { - expanded.append(boiTokenId) - for _ in 0 ..< numImageTokens { expanded.append(imageTokenId) } - expanded.append(eoiTokenId) - } else { - expanded.append(tid) - } - } - } else { - expanded = baseTokenIds - } - let inputIds = MLXArray(expanded.map { Int32($0) }) - Self.log.notice("manual: \(expanded.count, privacy: .public) input tokens") - if Task.isCancelled { throw RuntimeError.cancelled } - - // 4) For vision, inject the pixels onto the multimodal model so the forward pass scatters them at the - // image token positions. Captured `nonisolated(unsafe)` to cross into the container's closure (the - // MLXArray is not Sendable but is used single-threaded here, mirroring the CLI). - if let pv = pixelValues { - nonisolated(unsafe) let finalPixelValues = pv - await container.perform { context in - if let model = context.model as? Gemma4MultimodalLLMModel { - model.pendingPixelValues = finalPixelValues - } - } - } - - // 5) Manual generate loop: prefill, then autoregressive decode → channel-classified tokens, yielded - // LIVE. Greedy at temperature≈0, sampled otherwise. `dropThinking` is the inverse of reasoning — - // response-only when reasoning is off (mirrors `Gemma4TokenFilter` mode `.disabled`). - nonisolated(unsafe) let capturedInputIds = inputIds - let cappedMaxTokens = max(1, maxTokens) - let eosTokenIds = Gemma4Processor.eosTokenIds - let dropThinking = !enableThinking - // The continuation is Sendable; yielding from inside `perform` streams tokens as they decode. - nonisolated(unsafe) let sink = continuation - - try await container.perform { context in - let params = GenerateParameters(maxTokens: cappedMaxTokens, temperature: temperature, topP: 0.95) - let cache = context.model.newCache(parameters: params) - - // Prefill the full prompt; take argmax of the last position as the first generated token. - let prefill = context.model(capturedInputIds.reshaped(1, -1), cache: cache) - var nextToken = argMax(prefill[0..., prefill.dim(1) - 1, 0...], axis: -1).item(Int32.self) - - var classifier = ChannelClassifier(dropThinking: dropThinking) - var visibleResponseEmitted = 0 - // Budget generously above the visible cap for in-band thinking/channel tokens (mirrors the CLI's - // headroom). Stop on EOS or once the VISIBLE RESPONSE budget is reached (thinking doesn't count). - let hardCap = cappedMaxTokens * 3 - for _ in 0 ..< hardCap { - if Task.isCancelled { throw RuntimeError.cancelled } - if eosTokenIds.contains(nextToken) { break } - - let decoded = context.tokenizer.decode(tokenIds: [Int(nextToken)]) - let (visible, channel) = classifier.classify(tokenId: nextToken, text: decoded) - if !visible.isEmpty { - sink.yield(Token(visible, channel: channel)) - if channel == .response { visibleResponseEmitted += 1 } - } - if visibleResponseEmitted >= cappedMaxTokens { break } - - let nextInput = MLXArray([nextToken]).reshaped(1, 1) - let output = context.model(nextInput, cache: cache) - if temperature <= 0.01 { - nextToken = argMax(output[0..., 0, 0...], axis: -1).item(Int32.self) - } else { - let logits = output[0..., 0, 0...] / temperature - let probs = softmax(logits, axis: -1) - // Qualify `MLX.log` — the unqualified `log` collides with this type's static `log` - // (`Logger`) property, which the compiler would otherwise resolve to. - nextToken = MLXRandom.categorical(MLX.log(probs)).item(Int32.self) - } - } - } - - if Task.isCancelled { throw RuntimeError.cancelled } - Self.log.notice("manual: finished ✓") - } - - // MARK: - Response-only text collection - - /// Collect a generation into a single string of the FINAL ANSWER ONLY — `.response`-channel text, - /// dropping any `.thinking`. The protocol's default `generateText` concatenates EVERY token's text - /// (thinking included), which would feed the model's reasoning into JSON extraction; structured output - /// must parse only the response, so we override here. (For a non-reasoning request this is identical to - /// the default — every chunk is already `.response`.) - public func generateText(_ request: LLMRequest) async throws -> String { - var out = "" - for try await token in generate(request) where token.channel == .response { - out += token.text - } - return out - } - - // MARK: - Structured output (validate → repair/retry → decode → outcome) - - /// A marker the prompt instructs the model to emit when the input does not fit the task. Mirrors - /// `StubLLMRuntime`'s decline contract (design D2): a first-class typed decline, never a fabricated - /// value. We accept either an explicit `{"applicable": false, ...}` or a `{"__declined__": "…"}`. - private static let declineMarkerKey = "__declined__" - - public func structured( - _ request: LLMRequest, - schema: StructuredSchema, - as type: T.Type - ) async throws -> StructuredOutcome { - try Task.checkCancellation() - - let budget = 3 // bounded repair/retry (first attempt + repairs), mirroring the stub - var attempts = 0 - var lastFeedback: String? = nil - - while attempts < budget { - try Task.checkCancellation() - attempts += 1 - - let prompt = Self.buildPrompt( - base: request.prompt, - schema: schema, - repairFeedback: lastFeedback - ) - // Greedy (low temperature) so structured output is as deterministic as the model allows. - // Propagate the caller's `image`/`reasoning` so a vision or reasoning structured request still - // routes through the right path. `generateText` (overridden above) returns RESPONSE-channel text - // ONLY — so JSON is extracted from the final answer, never from thinking (which may contain - // braces that would otherwise mislead `extractJSONObject`). - let genRequest = LLMRequest( - prompt: prompt, - images: request.images, // forward ALL of the request's images (design D2) - parameters: GenerationParameters(maxTokens: request.parameters.maxTokens, temperature: 0), - reasoning: request.reasoning - ) - let raw = try await generateText(genRequest) - - // 1) Decline detection: short-circuit to a typed decline. - if let reason = Self.declineReason(in: raw) { - return .declined(reason: reason) - } - - // 2) Extract the JSON object from the (possibly chatty) model output. - guard let jsonString = Self.extractJSONObject(from: raw), - let data = jsonString.data(using: .utf8) else { - lastFeedback = "Your previous reply contained no JSON object. Reply with ONLY the JSON object." - continue - } - - // A decline can also arrive inside the extracted object. - if let reason = Self.declineReason(in: jsonString) { - return .declined(reason: reason) - } - - // 3) Validate required keys (structural subset of JSON Schema, like the stub). - guard Self.validate(jsonData: data, against: schema) else { - let missing = Self.missingRequiredKeys(jsonData: data, against: schema) - lastFeedback = "The JSON was missing required keys: \(missing.sorted().joined(separator: ", ")). " - + "Return ONLY a JSON object that includes every required key." - continue - } - - // 4) Decode into the target Swift type. - do { - let decoded = try JSONDecoder().decode(T.self, from: data) - return .value(decoded) - } catch { - lastFeedback = "The JSON did not decode into the required type (\(error)). " - + "Fix the value types and return ONLY the corrected JSON object." - continue - } - } - - throw RuntimeError.couldNotProduceValid(attempts: attempts) - } - - // MARK: - Prompt building - - /// Build a JSON-instructing prompt: the base prompt, the schema, the strict output contract, and - /// (on a repair attempt) feedback about what was wrong with the previous reply. - private static func buildPrompt(base: String, schema: StructuredSchema, repairFeedback: String?) -> String { - var p = base - p += "\n\nJSON Schema (target shape \"\(schema.name)\"):\n" - p += schema.json - p += "\n\nReturn ONLY JSON matching this schema. Do not include any prose, code fences, or " - p += "explanation. If the input does not fit, return {\"applicable\": false, \"reason\": \"…\"}." - if let repairFeedback { - p += "\n\nYour previous reply was rejected. \(repairFeedback)" - } - return p - } - - // MARK: - JSON helpers (mirror StubLLMRuntime's validate/decline approach) - - /// If the raw text carries a decline — either `{"__declined__": "reason"}` or - /// `{"applicable": false, "reason": "…"}` — return the reason. - private static func declineReason(in raw: String) -> String? { - guard let json = extractJSONObject(from: raw), - let data = json.data(using: .utf8), - let obj = try? JSONSerialization.jsonObject(with: data) as? [String: Any] else { return nil } - if let reason = obj[declineMarkerKey] as? String { return reason } - if let applicable = obj["applicable"] as? Bool, applicable == false { - return (obj["reason"] as? String) ?? "The model declined: the input does not fit the task." - } - return nil - } - - /// Structural validation: parse the JSON object and check it carries every key named in the - /// schema's top-level `required` array (a pragmatic JSON-Schema subset, like the stub). - private static func validate(jsonData: Data, against schema: StructuredSchema) -> Bool { - guard let value = try? JSONSerialization.jsonObject(with: jsonData) as? [String: Any] else { - return false - } - let required = requiredKeys(fromSchemaJSON: schema.json) - for key in required where value[key] == nil { return false } - return true - } - - /// The required keys absent from the JSON object (for repair feedback). - private static func missingRequiredKeys(jsonData: Data, against schema: StructuredSchema) -> Set { - let required = requiredKeys(fromSchemaJSON: schema.json) - guard let value = try? JSONSerialization.jsonObject(with: jsonData) as? [String: Any] else { - return required - } - return required.filter { value[$0] == nil } - } - - /// Extract the `required: [...]` string array from a JSON-Schema document string. - private static func requiredKeys(fromSchemaJSON json: String) -> Set { - guard let data = json.data(using: .utf8), - let obj = try? JSONSerialization.jsonObject(with: data) as? [String: Any], - let required = obj["required"] as? [String] else { return [] } - return Set(required) - } - - /// Pull the first balanced top-level JSON object out of a model reply that may contain prose or a - /// ```json code fence around the payload. Returns nil if no `{ … }` is found. - private static func extractJSONObject(from raw: String) -> String? { - guard let open = raw.firstIndex(of: "{") else { return nil } - var depth = 0 - var inString = false - var escaped = false - var i = open - while i < raw.endIndex { - let c = raw[i] - if escaped { - escaped = false - } else if c == "\\" && inString { - escaped = true - } else if c == "\"" { - inString.toggle() - } else if !inString { - if c == "{" { depth += 1 } - else if c == "}" { - depth -= 1 - if depth == 0 { - return String(raw[open...i]) - } - } - } - i = raw.index(after: i) - } - return nil - } -} diff --git a/Sources/GemmaRuntime/GemmaResumableDownloader.swift b/Sources/GemmaRuntime/GemmaResumableDownloader.swift deleted file mode 100644 index 39f5ecb..0000000 --- a/Sources/GemmaRuntime/GemmaResumableDownloader.swift +++ /dev/null @@ -1,463 +0,0 @@ -// GemmaResumableDownloader — a byte-resumable HuggingFace model downloader. -// -// WHY THIS EXISTS: the vendored `Gemma4ModelDownloader` skips already-COMPLETE shards on a retry, but -// it has no BYTE-level resume — a wifi drop mid-shard (`NSURLErrorDomain -1005`) discards the whole -// in-flight shard and, after enough drops, fails the download permanently. For a user on flaky wifi -// pulling a ~18 GB model in four shards, that's fatal. This downloader streams each file to a -// `{dest}.part` on disk, APPENDING bytes as they arrive, so a drop leaves partial progress on disk and -// the next attempt resumes from exactly where it left off via an HTTP `Range: bytes=N-` request (the -// `resolve/main` redirect target supports `206 Partial Content`). Completed files are still skipped. -// -// EFFICIENCY: it does NOT iterate `URLSession.AsyncBytes` byte-by-byte (orders of magnitude too slow -// for GBs). A `URLSessionDataDelegate` appends each received `Data` chunk straight to a `FileHandle`, -// and `didCompleteWithError` resumes a continuation. Delegate callbacks are serialized on a dedicated -// serial `OperationQueue`, so the file writes happen in order without extra locking on the hot path. - -import Foundation -import Gemma4Swift - -/// A robust, byte-resumable downloader for an `mlx-community` Gemma 4 model from the HuggingFace Hub. -/// -/// These models are UNGATED, so no token is required; a token is accepted for parity / private repos. -public enum GemmaResumableDownloader { - - // MARK: - Public entry - - /// Ensure every weight/config file of `model` is present and complete under `modelDir`. - /// - /// Fetches the repo file list (with sizes) from the tree API, then for each file: if it already - /// exists at the expected size it's counted done and skipped; otherwise it's downloaded RESUMABLY - /// to `{dest}.part` and atomically moved into place. `progress` reports a clamped 0…1 byte fraction. - /// Honors `Task` cancellation (throws `CancellationError`, cancelling the in-flight request). - public static func ensureModel( - _ model: Gemma4Pipeline.Model, - into modelDir: URL, - token: String? = nil, - progress: @escaping @Sendable (Double) -> Void - ) async throws { - let modelId = model.rawValue - let fm = FileManager.default - try fm.createDirectory(at: modelDir, withIntermediateDirectories: true) - - // 1) Discover the files to fetch (skip docs/attributes — not part of the loadable model). - let files = try await fetchFileList(modelId: modelId, token: token) - .filter { $0.type == "file" && !isSkippable($0.path) } - let total = files.reduce(0) { $0 + max($1.size, 0) } - - // 2) Walk the files. `doneSoFar` accumulates the bytes of every fully-finished file so the - // per-file progress callback can report a global fraction. - var doneSoFar = 0 - for file in files { - try Task.checkCancellation() - let dest = modelDir.appendingPathComponent(file.path) - - // Already complete on disk? Count it and move on. - if let size = fileSize(at: dest), size == file.size { - doneSoFar += file.size - emit(progress, doneSoFar, total) - continue - } - - let baseline = doneSoFar - try await downloadFileResumably( - modelId: modelId, - path: file.path, - expectedSize: file.size, - dest: dest, - token: token, - progress: { bytesThisFile in - emit(progress, baseline + bytesThisFile, total) - } - ) - doneSoFar += file.size - emit(progress, doneSoFar, total) - } - - emit(progress, total, total) - } - - // MARK: - File list (tree API) - - /// One entry from the HF tree API. - struct RepoFile: Sendable { - let type: String // "file" | "directory" - let path: String - let size: Int - } - - /// Files that are part of the repo but NOT part of the loadable model — pure docs / VCS metadata. - static func isSkippable(_ path: String) -> Bool { - let name = (path as NSString).lastPathComponent - if name == ".gitattributes" || name == "README.md" { return true } - if name.lowercased().hasSuffix(".md") { return true } - return false - } - - /// Fetch the model's file list (with sizes) from `GET /api/models/{id}/tree/main`. - static func fetchFileList(modelId: String, token: String?) async throws -> [RepoFile] { - guard let url = URL(string: "https://huggingface.co/api/models/\(modelId)/tree/main") else { - throw Gemma4DownloadError.apiFailed(modelId) - } - var request = URLRequest(url: url) - request.setValue("identity", forHTTPHeaderField: "Accept-Encoding") - if let token { request.setValue("Bearer \(token)", forHTTPHeaderField: "Authorization") } - - let (data, response): (Data, URLResponse) - do { - (data, response) = try await URLSession.shared.data(for: request) - } catch is CancellationError { - throw CancellationError() - } catch { - throw Gemma4DownloadError.networkError(modelId, error) - } - guard let http = response as? HTTPURLResponse, (200..<300).contains(http.statusCode) else { - let code = (response as? HTTPURLResponse)?.statusCode ?? -1 - throw Gemma4DownloadError.httpError(modelId, code) - } - - guard let array = try? JSONSerialization.jsonObject(with: data) as? [[String: Any]] else { - throw Gemma4DownloadError.parseError(modelId) - } - return array.compactMap { obj in - guard let type = obj["type"] as? String, let path = obj["path"] as? String else { return nil } - let size = (obj["size"] as? Int) ?? (obj["size"] as? NSNumber)?.intValue ?? 0 - return RepoFile(type: type, path: path, size: size) - } - } - - // MARK: - Per-file resumable download - - private static let maxAttempts = 10 - - /// Download a single file to `{dest}.part` (appending across attempts) then move it into place. - static func downloadFileResumably( - modelId: String, - path: String, - expectedSize: Int, - dest: URL, - token: String?, - progress: @escaping @Sendable (Int) -> Void - ) async throws { - let fm = FileManager.default - try fm.createDirectory(at: dest.deletingLastPathComponent(), withIntermediateDirectories: true) - let part = dest.appendingPathExtension("part") - - guard let resolveURL = resolveURL(modelId: modelId, path: path) else { - throw Gemma4DownloadError.httpError(path, -1) - } - - var attempt = 0 - while true { - try Task.checkCancellation() - attempt += 1 - - var have = fileSize(at: part) ?? 0 - // Defensive: if `.part` somehow overshot the expected size, restart cleanly. - if expectedSize > 0 && have > expectedSize { - try? fm.removeItem(at: part) - have = 0 - } - // Already fully on disk in `.part`? Just move it. - if expectedSize > 0 && have == expectedSize { - try finalizeMove(part: part, dest: dest) - progress(expectedSize) - return - } - - do { - try await streamOneAttempt( - resolveURL: resolveURL, - part: part, - have: have, - expectedSize: expectedSize, - token: token, - progress: progress - ) - // Success of one attempt means the stream completed without error. Re-check size and - // either finalize or loop again (a short read loops back through Range resume). - let now = fileSize(at: part) ?? 0 - if expectedSize <= 0 || now >= expectedSize { - try finalizeMove(part: part, dest: dest) - progress(expectedSize > 0 ? expectedSize : now) - return - } - // Short read but no error reported: treat as retryable so the next Range request resumes. - if attempt >= maxAttempts { - throw Gemma4DownloadError.networkError(path, ShortReadError(have: now, want: expectedSize)) - } - try await backoff(attempt) - } catch is CancellationError { - throw CancellationError() - } catch let fatal as GemmaDownloadFatalHTTPError { - // 4xx (bad token / gated / missing) — don't burn the retry budget. - throw Gemma4DownloadError.httpError(path, fatal.statusCode) - } catch { - // Network drop / transient HTTP. Retry from the current `.part` size if attempts remain. - if attempt >= maxAttempts { - throw Gemma4DownloadError.networkError(path, error) - } - try await backoff(attempt) - } - } - } - - /// Run exactly ONE network attempt: request the file (Range when resuming), append the streamed - /// bytes to `.part`, and resume when the request completes. Throws on any error/non-2xx-206 status. - private static func streamOneAttempt( - resolveURL: URL, - part: URL, - have: Int, - expectedSize: Int, - token: String?, - progress: @escaping @Sendable (Int) -> Void - ) async throws { - let fm = FileManager.default - - // Ensure the `.part` exists so we can open a writing handle that seeks to end for appends. - if !fm.fileExists(atPath: part.path) { - fm.createFile(atPath: part.path, contents: nil) - } - let handle = try FileHandle(forWritingTo: part) - if have > 0 { - try handle.seekToEnd() - } - - var request = URLRequest(url: resolveURL) - request.setValue("identity", forHTTPHeaderField: "Accept-Encoding") - if have > 0 { request.setValue("bytes=\(have)-", forHTTPHeaderField: "Range") } - if let token { request.setValue("Bearer \(token)", forHTTPHeaderField: "Authorization") } - - let collector = ResponseCollector( - handle: handle, - startOffset: have, - requestedResume: have > 0, - progress: progress - ) - - let queue = OperationQueue() - queue.maxConcurrentOperationCount = 1 - queue.name = "GemmaResumableDownloader.delegate" - let session = URLSession(configuration: .ephemeral, delegate: collector, delegateQueue: queue) - defer { session.finishTasksAndInvalidate() } - - let task = session.dataTask(with: request) - collector.attach(task: task) - - do { - try await withTaskCancellationHandler { - try await withCheckedThrowingContinuation { (continuation: CheckedContinuation) in - collector.setContinuation(continuation) - task.resume() - } - } onCancel: { - task.cancel() - } - } catch { - try? collector.closeHandle() - throw error - } - try collector.closeHandle() - } - - // MARK: - Helpers - - /// `https://huggingface.co/{modelId}/resolve/main/{path}` with each path component percent-encoded. - static func resolveURL(modelId: String, path: String) -> URL? { - let encodedModel = modelId - .split(separator: "/", omittingEmptySubsequences: false) - .map { percentEncodePathSegment(String($0)) } - .joined(separator: "/") - let encodedPath = path - .split(separator: "/", omittingEmptySubsequences: false) - .map { percentEncodePathSegment(String($0)) } - .joined(separator: "/") - return URL(string: "https://huggingface.co/\(encodedModel)/resolve/main/\(encodedPath)") - } - - private static func percentEncodePathSegment(_ segment: String) -> String { - // `urlPathAllowed` keeps `/` allowed, so encode against a stricter set per-segment. - var allowed = CharacterSet.urlPathAllowed - allowed.remove(charactersIn: "/?#[]") - return segment.addingPercentEncoding(withAllowedCharacters: allowed) ?? segment - } - - static func fileSize(at url: URL) -> Int? { - guard let attrs = try? FileManager.default.attributesOfItem(atPath: url.path), - let size = attrs[.size] as? NSNumber else { return nil } - return size.intValue - } - - /// Atomically place `.part` at `dest`, removing any stale `dest` first. - private static func finalizeMove(part: URL, dest: URL) throws { - let fm = FileManager.default - if fm.fileExists(atPath: dest.path) { - try fm.removeItem(at: dest) - } - try fm.moveItem(at: part, to: dest) - } - - private static func backoff(_ attempt: Int) async throws { - let seconds = min(Double(1 << min(attempt, 5)), 30) // 2,4,8,16,30,30… capped at 30s - try await Task.sleep(nanoseconds: UInt64(seconds * 1_000_000_000)) - } - - private static func emit(_ progress: @escaping @Sendable (Double) -> Void, _ done: Int, _ total: Int) { - guard total > 0 else { progress(0); return } - progress(min(max(Double(done) / Double(total), 0), 1)) - } - - /// A short read that exhausted retries (the body ended before `expectedSize`). - private struct ShortReadError: Error { let have: Int; let want: Int } -} - -// MARK: - Shared error types (file-private to the downloader) - -/// A non-retryable HTTP status (4xx) — fail fast rather than burning the retry budget. -struct GemmaDownloadFatalHTTPError: Error { let statusCode: Int } -/// A retryable non-2xx/206 status — the per-file loop backs off and retries from the `.part` size. -struct GemmaDownloadRetryableHTTPError: Error { let statusCode: Int } - -// MARK: - URLSession delegate (chunk → FileHandle) - -/// Streams a single HTTP response body to a `FileHandle`, appending each `Data` chunk as it arrives. -/// -/// All `URLSessionDataDelegate` callbacks run serialized on the session's dedicated single-threaded -/// `delegateQueue`, so the ordered file writes and the continuation handoff need no further locking on -/// that path. The continuation/handle are guarded by a lock only because `closeHandle()` / -/// `setContinuation(_:)` may be touched from the awaiting task as well. -private final class ResponseCollector: NSObject, URLSessionDataDelegate, @unchecked Sendable { - private let lock = NSLock() - private let handle: FileHandle - private let startOffset: Int - private let requestedResume: Bool - private let progress: @Sendable (Int) -> Void - - private var continuation: CheckedContinuation? - private var resumed = false - private var handleClosed = false - private var bytesWritten = 0 - private var truncatedForFullBody = false - private weak var task: URLSessionTask? - - init( - handle: FileHandle, - startOffset: Int, - requestedResume: Bool, - progress: @escaping @Sendable (Int) -> Void - ) { - self.handle = handle - self.startOffset = startOffset - self.requestedResume = requestedResume - self.progress = progress - } - - func attach(task: URLSessionTask) { self.task = task } - - func setContinuation(_ c: CheckedContinuation) { - lock.lock(); defer { lock.unlock() } - continuation = c - } - - func closeHandle() throws { - lock.lock(); defer { lock.unlock() } - guard !handleClosed else { return } - handleClosed = true - try handle.close() - } - - private func finish(_ result: Result) { - lock.lock() - guard !resumed, let c = continuation else { lock.unlock(); return } - resumed = true - continuation = nil - lock.unlock() - switch result { - case .success: c.resume() - case .failure(let e): c.resume(throwing: e) - } - } - - // Decide whether to accept the response and how to treat the body (206 append vs 200 restart). - func urlSession( - _ session: URLSession, - dataTask: URLSessionDataTask, - didReceive response: URLResponse, - completionHandler: @escaping (URLSession.ResponseDisposition) -> Void - ) { - guard let http = response as? HTTPURLResponse else { - completionHandler(.cancel) - finish(.failure(URLError(.badServerResponse))) - return - } - let code = http.statusCode - - // Fatal client errors: don't retry (bad token / gated / missing file). - if code == 401 || code == 403 || code == 404 { - completionHandler(.cancel) - finish(.failure(GemmaDownloadFatalHTTPError(statusCode: code))) - return - } - - if code == 206 { - // Server honored the Range — append from where we left off. Good. - completionHandler(.allow) - return - } - - if code == 200 { - // Range was ignored (or none was sent). If we had partial bytes, the body starts at 0, so - // truncate `.part` to 0 and restart this file from the beginning. - if requestedResume { - lock.lock() - if !truncatedForFullBody { - truncatedForFullBody = true - do { - try handle.seek(toOffset: 0) - try handle.truncate(atOffset: 0) - } catch { - lock.unlock() - completionHandler(.cancel) - finish(.failure(error)) - return - } - } - lock.unlock() - } - completionHandler(.allow) - return - } - - // Any other status (5xx, 3xx-not-followed, etc.): retryable failure. - completionHandler(.cancel) - finish(.failure(GemmaDownloadRetryableHTTPError(statusCode: code))) - } - - func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive data: Data) { - do { - try handle.write(contentsOf: data) - } catch { - task?.cancel() - finish(.failure(error)) - return - } - bytesWritten += data.count - // Report bytes-this-file. When the server ignored Range and restarted at 0, `bytesWritten` is - // the absolute file size; otherwise it's startOffset + what we appended. - let absolute = truncatedForFullBody ? bytesWritten : (startOffset + bytesWritten) - progress(absolute) - } - - func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: Error?) { - // Flush to disk before finishing so the next attempt sees the true `.part` size. - try? handle.synchronize() - if let error { - if (error as NSError).code == NSURLErrorCancelled { - finish(.failure(CancellationError())) - } else { - finish(.failure(error)) - } - } else { - finish(.success(())) - } - } -} diff --git a/Sources/GemmaRuntime/GemmaRuntime.swift b/Sources/GemmaRuntime/GemmaRuntime.swift deleted file mode 100644 index 7890d2a..0000000 --- a/Sources/GemmaRuntime/GemmaRuntime.swift +++ /dev/null @@ -1,304 +0,0 @@ -// GemmaRuntime — the MLX-backed Gemma 4 runtime, isolated in its own SwiftPM target. -// -// WHY A SEPARATE TARGET: it links MLX (`Gemma4Swift` → mlx-swift), whose Metal shaders only -// compile under `xcodebuild`. Keeping it out of `ThreeFingerSwitcherCore` lets the Core library and -// the test target keep building/verifying under plain `swift build` / `swift test` (no Metal). Only -// this target and the app executable that depends on it require `xcodebuild`. -// -// It conforms to Core's public `LLMRuntime` seam and is injected at the model seam from the app -// target's entry point — Core never references a concrete model (design D1/D7). - -import Foundation -import ThreeFingerSwitcherCore -import Gemma4Swift -import MLX - -/// The composition root for the real, in-process Gemma 4 runtime. -public enum GemmaRuntime { - - /// Build a `ModelManager` wired to the real Gemma 4 (MLX) runtime. - /// - /// Uses `ModelCatalog.standard` (whose descriptors carry the mlx-community repo names/sizes) and a - /// `ModelProvisioner` that, on `downloadAndVerify(descriptor)`, creates a `GemmaMLXRuntime` and - /// `prepare`s it — the pipeline downloads the weights from the HuggingFace Hub (which verifies - /// integrity) and loads them resident, reporting a 0…1 fraction back through the manager's - /// `.downloading(progress:)` state. The manager then stores the returned runtime and settles - /// `.loaded`, bypassing its byte-SHA + `runtimeFactory` path entirely. - /// - /// `optedIn` seeds the manager's opt-in from settings (no download happens until opt-in). - /// - /// FLEET + COMPUTE TIERS (wire-compute-fleet): this is the live composition root where the V2.5 - /// fleet and the CPU ternary lane are welded onto the running app. - /// - **Fleet:** `fleet: FleetRoster.standard` is passed into `ModelManager.init` so the manager's - /// `ensureResident(_:)` plan/evict path is REACHABLE (it was dead while `fleet` defaulted nil). The - /// free-memory probe keeps `FleetRoster`'s default 48 GB budget (the user's stable-signed build - /// verifies the real bytes); a fleet-of-one / single-model load still works because the standard - /// roster's plan for the chat target is `{admit:[chat], evict:[]}` — byte-for-byte today's behavior. - /// - **Compute tiers:** the `TernaryCPURuntime` is constructed ONLY when `cpuLaneUnlocked` (the - /// already-resolved `isUnlocked(.cpuLane)` = master ∧ sub-flag ∧ ai-commands, §D1 / §7.1). LOCKED - /// (the default) → no `TernaryCPURuntime` is constructed and the provisioner only ever returns the - /// GPU runtime, so the build behaves exactly as a single-GPU-lane one (no regression). UNLOCKED → a - /// `.cpuTernary`-lane fleet descriptor resolves, through the EXISTING provisioner seam keyed by - /// `descriptor.lane`, to the installed `TernaryCPURuntime` (a stub that yields cleanly until its real - /// bitnet.cpp-class backend — it never fakes tokens; the WIRING + lane selection is what this lands). - /// - /// - **Batched runtime:** the GPU lane's resident runtime is the multi-stream - /// `BatchedGemmaMLXRuntime` ONLY when `batchedRuntimeUnlocked` (§D1 → `batchedRuntimeEnabled` under - /// the master). OFF (the default) → the proven single-session `GemmaMLXRuntime` is resident, so the - /// K-stream / growable-context surface is simply never constructed and the build behaves exactly as - /// today's single-session one (the calm panic-off — no error, just inert). - /// - /// `cpuLaneUnlocked` / `batchedRuntimeUnlocked` are ALREADY resolved by the caller through - /// `FullPotentialGate.isUnlocked(_:)` (the single resolver — so a master-OFF or ai-commands-OFF locks - /// both at once); they default to today's OFF behavior so existing call-sites / tests compile unchanged. - @MainActor - public static func makeModelManager(optedIn: Bool, - cpuLaneUnlocked: Bool = false, - batchedRuntimeUnlocked: Bool = false) -> ModelManager { - configureModelStorage() // relocate the model store off purgeable ~/Library/Caches (see below) - configureGPUCacheLimit() // bound the MLX Metal buffer cache (see below) - // Install the CPU ternary lane ONLY when the gate is on (returns nil when OFF → GPU lane alone, - // today's behavior). Weights live under the same model cache root the chat path loads from, keyed - // off the ternary fleet descriptor's HF repo path (mirrors `makeImageRuntime`'s weights dir). - // `cpuLaneUnlocked` is the already-resolved `isUnlocked(.cpuLane)` (master ∧ sub-flag ∧ ai-commands). - let ternaryLane = cpuLaneUnlocked ? TernaryCPURuntime(weightsURL: ternaryWeightsURL()) : nil - return ModelManager( - registry: .standard, - downloader: HubDownloader(), - optedIn: optedIn, - provisioner: { descriptor, progress in - // §6.1 lane-keyed resolution (design D2): a `.cpuTernary`-lane fleet descriptor loads the - // installed `TernaryCPURuntime` (when the gate is on) through this UNCHANGED provisioner - // seam — feature code selects BY lane, never seeing the concrete type. When the lane is OFF - // (`ternaryLane == nil`) we fall through to the GPU runtime (belt-and-braces: the role→lane - // policy already coerces every role to `.gpu` when OFF, so a `.cpuTernary` descriptor is - // never selected in the first place). - if descriptor.lane == .cpuTernary, let ternary = ternaryLane { - try await ternary.prepare() // maps load failure → RuntimeError at the boundary (D7) - return ternary - } - let model = pipelineModel(for: descriptor) - // The GPU lane's resident runtime, GATED on `batchedRuntimeUnlocked` (the calm panic-off): - // - UNLOCKED → the multi-stream `BatchedGemmaMLXRuntime` (design D9): conforms to - // `LLMRuntime` AND `BatchedLLMRuntime`, subsuming the single-session paths and adding the - // K-stream / growable-context surface the scheduler-driven background advancer downcasts - // to via `BatchedLLMRuntime`. - // - LOCKED (default) → the proven single-session `GemmaMLXRuntime`. No multi-stream surface - // is constructed; the build behaves exactly as today's single-session one (inert, no error). - // Either is stored by `ModelManager` as a plain `LLMRuntime` (no API change); a locked build - // simply never offers the `BatchedLLMRuntime` downcast, so the advancer can't engage it. - if batchedRuntimeUnlocked { - let runtime = BatchedGemmaMLXRuntime(weightBytes: descriptor.sizeBytes, - maxContextTokens: descriptor.maxContextTokens) - try await runtime.prepare(model: model, progress: progress) - return runtime - } - let runtime = GemmaMLXRuntime() - try await runtime.prepare(model: model, progress: progress) - return runtime - }, - // Network-free disk probe so the manager rediscovers an already-downloaded model on launch - // (→ `.ready`) and lazy-loads it on first use, instead of asking the user to "Download" - // again. Deliberately stricter than `Gemma4ModelCache.isDownloaded` (see `isFullyDownloaded`). - // A `.cpuTernary`-lane descriptor's weights aren't a Gemma pipeline model — probe its own - // dir directly (the ternary GGUF-class file), so the chat pipeline isn't probed for it. - provisionedOnDisk: { descriptor in - if descriptor.lane == .cpuTernary { - return FileManager.default.fileExists(atPath: ternaryWeightsURL().path) - } - return isFullyDownloaded(pipelineModel(for: descriptor)) - }, - // Delete the weights from the EXACT dir the app loads from / `isFullyDownloaded` probes, so a - // per-model Delete or the Danger zone genuinely frees them and the model reads as - // not-downloaded afterwards (Core deletes only its own app-support dir, the wrong location). - provisionedDelete: { descriptor in - if descriptor.lane == .cpuTernary { - try? FileManager.default.removeItem(at: ternaryWeightsDirectory()) - return - } - deleteFromCache(pipelineModel(for: descriptor)) - }, - // The FLEET — passing `FleetRoster.standard` makes `ModelManager.ensureResident(_:)`'s - // plan/evict block REACHABLE (it was dead while `fleet` defaulted nil). The default 48 GB - // budget + free-bytes probe keep a fleet-of-one / co-resident load identical to today; the - // user's stable-signed build verifies the real residency/eviction with real weights. - fleet: FleetRoster.standard - ) - } - - /// On-disk location of the CPU ternary lane's weights — under the same model cache root the chat path - /// loads from, keyed off the `ternary-cpu-chat` fleet descriptor's HF repo path (mirrors the image - /// runtime's weights-dir derivation). Returns the directory; `ternaryWeightsURL()` appends the file. - static func ternaryWeightsDirectory() -> URL { - var dir = Gemma4ModelCache.modelsDirectory - // The standard roster always carries the ternary member; fall back to the known repo path defensively. - let repoPath = FleetRoster.standard.descriptor(id: "ternary-cpu-chat")?.downloadURL.path - ?? "/mlx-community/ternary-cpu-chat" - for part in repoPath.split(separator: "/") { - dir.appendPathComponent(String(part)) - } - return dir - } - - /// The ternary weights FILE the `TernaryCPURuntime` loads from (a ternary/BitNet-class GGUF-class - /// file). `TernaryCPURuntime.prepare` does a `fileExists` guard over exactly this path → a missing - /// file is a clean `RuntimeError.modelMissing`, never a fake load. - static func ternaryWeightsURL() -> URL { - ternaryWeightsDirectory().appendingPathComponent("ternary.gguf") - } - - /// Build the image `MediaRuntime` (the `generate_image` backend) for the user-selected `imageModelID`, - /// or `nil` when the selection is unknown (an unknown id is REJECTED, never coerced to a default — - /// `ImageModelCatalog.selected` returns nil, so the contributor simply doesn't advertise the tool). - /// - /// Resolves three things and hands them to `MFluxImageRuntime`: - /// 1. the selected `ImageModelCatalog` descriptor (Q4 default / FP16 opt-in); - /// 2. the on-disk weights dir — `Gemma4ModelCache.modelsDirectory//` derived from the - /// descriptor's `downloadURL` path (the same cache layout the chat path uses). It need not exist: - /// `MFluxImageRuntime.prepare`'s `fileExists` guard turns a missing dir into a clean - /// `MediaError` ("the image model isn't installed yet."), never a fake result; - /// 3. the gallery output dir (`MediaGallery.defaultRoot()`), where a finished PNG would be written. - /// - /// FLAGGED: user xcodebuild + stable-signed build. The runtime's diffusion pipeline is UNBUILT until - /// Wave 2, so a `generate_image` route returns a clean bounded `MediaError`, not a blank PNG. - @MainActor - public static func makeImageRuntime(imageModelID: String?) -> MediaRuntime? { - configureModelStorage() // ensure the non-purgeable model root is in effect (idempotent) - configureGPUCacheLimit() // ensure the bounded Metal buffer cache is in effect (idempotent) - guard let descriptor = ImageModelCatalog.selected(imageModelID: imageModelID) else { - return nil // unknown id → rejected, no dead-end tool - } - // Weights dir off the descriptor's HF repo path (`mlx-community/image-gen-q4`), under the same - // model cache root the chat path loads from — so a real Wave-2 download lands where this reads. - var weightsURL = Gemma4ModelCache.modelsDirectory - for part in descriptor.downloadURL.path.split(separator: "/") { - weightsURL.appendPathComponent(String(part)) - } - return MFluxImageRuntime(descriptor: descriptor, - weightsURL: weightsURL, - outputDirectory: MediaGallery.defaultRoot()) - } - - /// Bound MLX's Metal buffer cache (`fix-model-load-coalescing-and-gpu-cache`). By default the cache - /// is UNBOUNDED: every freed generation buffer is retained for reuse and never returned to the OS — - /// measured live as **24 GB of dirty IOAccelerator memory for a ~17 GB model after one short chat**, - /// creeping further with use (the vendored pipeline clears it only on `unload()`). 2 GB is enough to - /// recycle per-token scratch across a streaming turn while letting a settled app's footprint track - /// the model size; generation beyond the limit falls back to direct Metal allocations (a perf - /// trade-off, never a correctness one). Idempotent; both factory entry points call it before any - /// runtime exists. Tune here, in one place, if profiling ever argues for a different bound. - nonisolated(unsafe) private static var gpuCacheConfigured = false - static func configureGPUCacheLimit() { - guard !gpuCacheConfigured else { return } - gpuCacheConfigured = true - MLX.GPU.set(cacheLimit: 2 * 1024 * 1024 * 1024) - } - - /// The model store must live under a **non-purgeable** root. The vendored `Gemma4ModelCache` defaults - /// to `~/Library/Caches/models`, which macOS PURGES under disk pressure — silently forcing a ~17 GB - /// re-download (see `docs/postmortem-idle-cpu-spin.md`). Point the cache at - /// `ModelManager.defaultStorageRoot()` (`~/Library/Application Support/ThreeFingerSwitcher/models`), - /// migrate any copy already sitting in the old Caches root (same volume → an instant rename, NOT a - /// re-download), and exclude it from Time Machine (large, re-downloadable weights). Idempotent; the - /// factory entry points call it once before any download / load. - nonisolated(unsafe) private static var storageConfigured = false - /// `@MainActor` because `ModelManager.defaultStorageRoot()` is main-actor-isolated (ModelManager is - /// `@MainActor`); both callers (`makeModelManager` / `makeImageRuntime`) are already on the main actor. - @MainActor - static func configureModelStorage() { - guard !storageConfigured else { return } - storageConfigured = true - let fm = FileManager.default - let newRoot = ModelManager.defaultStorageRoot() - try? fm.createDirectory(at: newRoot, withIntermediateDirectories: true) - // Application Support IS backed up (unlike Caches) — keep the re-downloadable weights out of it. - var root = newRoot - var values = URLResourceValues() - values.isExcludedFromBackup = true - try? root.setResourceValues(values) - // One-time migration off the old purgeable Caches root, if a model already downloaded there. - let oldRoot = fm.urls(for: .cachesDirectory, in: .userDomainMask).first! - .appendingPathComponent("models", isDirectory: true) - if oldRoot != newRoot, - let entries = try? fm.contentsOfDirectory(at: oldRoot, includingPropertiesForKeys: nil) { - for src in entries { - let dst = newRoot.appendingPathComponent(src.lastPathComponent) - if !fm.fileExists(atPath: dst.path) { try? fm.moveItem(at: src, to: dst) } - } - try? fm.removeItem(at: oldRoot) // best-effort: drop the now-empty old dir - } - // From here, every `Gemma4ModelCache.modelsDirectory` read/write uses the non-purgeable root. - Gemma4ModelCache.customModelsDirectory = newRoot - } - - /// Remove `model`'s weights from `Gemma4ModelCache.modelsDirectory//` — the exact dir - /// `GemmaMLXRuntime.prepare` loads from and `isFullyDownloaded` probes — so a delete truly frees the - /// weights. Mirrors `isFullyDownloaded`'s path construction; a missing dir is a silent no-op. - static func deleteFromCache(_ model: Gemma4Pipeline.Model) { - var dir = Gemma4ModelCache.modelsDirectory - for part in model.rawValue.split(separator: "/") { - dir.appendPathComponent(String(part)) - } - try? FileManager.default.removeItem(at: dir) - } - - /// Whether `model`'s weights are COMPLETELY present in the EXACT directory the app loads from — - /// the resumable downloader's cache dir (`Gemma4ModelCache.modelsDirectory//`, the same - /// path `GemmaMLXRuntime.prepare` builds). A model is "fully downloaded" only when it has a - /// `config.json` + at least one `*.safetensors` AND no leftover `*.part` shard from an interrupted - /// download. - /// - /// This is intentionally STRICTER than `Gemma4ModelCache.isDownloaded`, which (a) also accepts the - /// HuggingFace cache (`~/.cache/huggingface/...`) that the app's `ensureModel` does NOT load from — - /// so a HF-only copy would re-download — and (b) ignores `.part` files, so a half-finished - /// multi-shard download would read as present and then fail to load. Matching the app's real load - /// location makes the rediscovery `.ready` mean exactly "loadable without any network." - static func isFullyDownloaded(_ model: Gemma4Pipeline.Model) -> Bool { - var dir = Gemma4ModelCache.modelsDirectory - for part in model.rawValue.split(separator: "/") { - dir.appendPathComponent(String(part)) - } - let fm = FileManager.default - // Walk the model dir (shallow + any subdirs) so a `.part` shard anywhere disqualifies it. - guard let enumerator = fm.enumerator(at: dir, includingPropertiesForKeys: nil) else { return false } - var hasConfig = false, hasWeights = false - for case let url as URL in enumerator { - let name = url.lastPathComponent - if name.hasSuffix(".part") { return false } // an interrupted download is NOT ready - if name == "config.json" { hasConfig = true } - if name.hasSuffix(".safetensors") { hasWeights = true } - } - return hasConfig && hasWeights - } - - /// Map a Core `ModelDescriptor` (selected from `ModelCatalog.standard` or the §C1 `FleetRoster`) to a - /// concrete `Gemma4Pipeline.Model`. Default = the flagship 31B 4-bit. Honors the selected descriptor - /// id; an unrecognized id falls back to the default rather than failing the build of a request. - /// - /// FLAGGED: user xcodebuild + stable-signed build. The fleet's extended `ModelDescriptor` fields - /// (`role`/`lane`/`provider`/`residencyBytes`) compile in this MLX path unchanged — only the chat - /// `.role`/`.gpu` members map to a Gemma pipeline here; the ternary/image/video/cloud members route - /// through their own runtimes/escalation (not this Gemma factory), so they intentionally hit the - /// default. The chat id (`gemma-4-31b`) is shared with the FleetRoster, so a fleet chat selection - /// resolves the same pipeline model with no API change. - static func pipelineModel(for descriptor: ModelDescriptor) -> Gemma4Pipeline.Model { - switch descriptor.id { - case "gemma-4-31b": return .b31b4bit // chat (FleetRoster + catalog); ~17 GB - case "gemma-4-26b-a4b": return .a4b4bit // mlx-community/gemma-4-26b-a4b-it-4bit (~14 GB) - case "gemma-4-12b": return .e4b4bit // mlx-community/gemma-4-e4b-it-4bit (~5 GB, audio-capable) - default: return .b31b4bit - } - } - - /// A `ModelDownloading` placeholder to satisfy `ModelManager`'s required `downloader` dependency. - /// On the real path the PROVISIONER owns the download (via `Gemma4Pipeline` / the HF Hub), so this - /// is never actually invoked; it throws to make any accidental use loud rather than silent. - private struct HubDownloader: ModelDownloading { - func download(_ descriptor: ModelDescriptor, to destination: URL, - progress: @Sendable (Double) -> Void) async throws -> Data { - throw RuntimeError.unavailable( - reason: "GemmaRuntime downloads via the pipeline/HF Hub through the provisioner, not the byte downloader." - ) - } - } -} diff --git a/Sources/GemmaRuntime/LaneDispatch.swift b/Sources/GemmaRuntime/LaneDispatch.swift deleted file mode 100644 index b407c61..0000000 --- a/Sources/GemmaRuntime/LaneDispatch.swift +++ /dev/null @@ -1,169 +0,0 @@ -// LaneDispatch — lane-keyed runtime wiring + additive lane-affinity dispatch (ai-compute-tiers §6). -// -// FLAGGED: user xcodebuild + stable-signed build. -// -// This welds the CPU ternary lane onto the EXISTING seams ADDITIVELY — without editing the sibling -// slices' files: -// - §6.1 (D2): resolve a runtime BY `ComputeLane` through the existing provisioner/`runtimeFactory` -// seam. The GPU lane keeps returning `BatchedGemmaMLXRuntime`; the CPU lane returns -// `TernaryCPURuntime`. NO `ModelManager` API change. -// - §6.2 (D4): read `ParkScheduler.runnableSessions(now:maxSlots:)`'s returned IDs and ATTACH a -// `LaneAffinity` per session via an ADDITIVE accessor — the pinned `runnableSessions` signature is -// UNCHANGED and the `ai-parked-sessions` files are NOT edited. -// - §6.3 (D4): route `.cpuTernary`-affined sessions to `TernaryCPURuntime` while the GPU -// `batchStep(...)` keeps serving `.gpu`-affined + foreground sessions — ADDITIVE, the -// `ai-batched-runtime-and-context` files are NOT edited. -// -// Net effect: a `parkedSubagent` session advances on the CPU lane CONCURRENTLY with a foreground GPU -// generation, instead of waiting for a GPU batch slot. Native-linked (it constructs the MLX GPU runtime -// + the bitnet.cpp-class CPU runtime), so it is `xcodebuild` compile-verify ONLY; the live two-lane -// concurrency is the USER's stable-signed run-verify (task 8.3). -// -// NOTE on the fleet descriptor (addendum §C1, owned by `ai-model-fleet`): the lane-tagged -// `ModelDescriptor.lane`/`ModelRole.ternaryChat` are CONSUMED here, not redefined. Until that slice -// lands, the CPU-lane descriptor is keyed by `ComputeLane.cpuTernary` directly; when the fleet's -// lane-tagged descriptor lands, `runtime(for:)` reads `descriptor.lane` with no call-site change. - -import Foundation -import ThreeFingerSwitcherCore - -public enum LaneDispatch { - - // MARK: - §7.1 Master-toggle install gate - - /// Whether to INSTALL the CPU ternary lane (design D6 / §7.1): gated on `cpuLaneEnabled` UNDER the - /// master `fullPotentialEnabled` (addendum §D1, owned by `ai-full-potential-toggle` — read, not - /// owned). OFF → no `TernaryCPURuntime` is constructed, and the OFF-coercion `GatedLaneRouting` - /// decorator routes every role to `.gpu`, so the build behaves exactly as a single-lane one (no - /// regression). The same `fullPotentialEnabled && cpuLaneEnabled` predicate as `GatedLaneRouting`, - /// kept here as the install-side gate so the runtime is never even allocated when OFF. - public static func shouldInstallCPULane(fullPotentialEnabled: Bool, cpuLaneEnabled: Bool) -> Bool { - fullPotentialEnabled && cpuLaneEnabled - } - - /// FLAGGED: construct the CPU ternary runtime ONLY when the gate is on. Returns `nil` when OFF, so the - /// dispatch path above runs the GPU lane alone (today's behavior). Native-only (it allocates the - /// bitnet.cpp-class runtime); the agent compile-verifies the gate, the user run-verifies the install. - public static func installCPULane(fullPotentialEnabled: Bool, - cpuLaneEnabled: Bool, - weightsURL: URL) -> TernaryCPURuntime? { - guard shouldInstallCPULane(fullPotentialEnabled: fullPotentialEnabled, - cpuLaneEnabled: cpuLaneEnabled) else { return nil } - return TernaryCPURuntime(weightsURL: weightsURL) - } - - // MARK: - §6.1 Lane-keyed runtime resolution (no ModelManager API change) - - /// Resolve the runtime for a `ComputeLane` through the existing provisioner seam (design D2). The - /// GPU lane is the resident `BatchedGemmaMLXRuntime` (already the provisioner's return); the CPU lane - /// is a `TernaryCPURuntime` installed ONLY when `cpuLaneActive` (gated upstream, §7). Feature code - /// never sees these concrete types — it selects BY lane. - /// - /// Returns `nil` for `.cpuTernary` when the CPU lane is not installed (OFF), so the caller falls back - /// to the GPU lane (the OFF-coercion decorator already routes every role to `.gpu`, so this is a - /// belt-and-braces guard). - public static func runtime(for lane: ComputeLane, - gpu: LLMRuntime, - ternary: TernaryCPURuntime?) -> LLMRuntime? { - switch lane { - case .gpu: return gpu - case .cpuTernary: return ternary - } - } - - // MARK: - §6.2 Additive lane-affinity attachment (runnableSessions signature UNCHANGED) - - /// Attach a `LaneAffinity` to each runnable parked session WITHOUT changing `runnableSessions`'s - /// pinned signature (design D4). The scheduler returns plain `AgentSessionID`s as before; this - /// additive accessor derives each session's lane from its work role via the `LaneRouting` policy. - /// - /// A parked session's work IS a `parkedSubagent` advance → `.cpuTernary` (under the default policy); - /// when the CPU lane is OFF the injected `routing` is the `GatedLaneRouting` decorator, which coerces - /// every role to `.gpu`, so the affinity correctly reads `.gpu` and the session stays on the GPU lane. - public static func affinities(for runnable: [AgentSessionID], - role: AgentWorkRole = .parkedSubagent, - routing: LaneRouting) -> [LaneAffinity] { - runnable.map { LaneAffinity(sessionID: $0, role: role, routing: routing) } - } - - // MARK: - §6.3 Additive dispatch split (batchStep + parked files NOT edited) - - /// The result of partitioning a runnable set by lane affinity (design D4) — pure, so the partition - /// logic is testable; the actual stepping below drives the two lanes. - public struct LanePartition: Equatable, Sendable { - /// `.gpu`-affined + foreground sessions → served by the batched GPU `batchStep(...)` (unchanged). - public var gpu: [AgentSessionID] - /// `.cpuTernary`-affined sessions → served by `TernaryCPURuntime`, concurrently with the GPU. - public var cpuTernary: [AgentSessionID] - public init(gpu: [AgentSessionID] = [], cpuTernary: [AgentSessionID] = []) { - self.gpu = gpu - self.cpuTernary = cpuTernary - } - } - - /// Split runnable sessions by their attached lane affinity. The foreground session (if any) is always - /// forced onto the GPU partition (its role is `foregroundGeneration` → `.gpu` by the policy, and the - /// long reply is GPU-only — D5). Pure + deterministic. - public static func partition(_ affinities: [LaneAffinity], - foreground: AgentSessionID? = nil) -> LanePartition { - var p = LanePartition() - for a in affinities { - if a.sessionID == foreground { p.gpu.append(a.sessionID); continue } - switch a.lane { - case .gpu: p.gpu.append(a.sessionID) - case .cpuTernary: p.cpuTernary.append(a.sessionID) - } - } - if let fg = foreground, !p.gpu.contains(fg) { p.gpu.insert(fg, at: 0) } - return p - } - - /// FLAGGED: drive both lanes for one dispatch round. The GPU `batchStep(...)` (§3.6, the - /// `ai-batched-runtime-and-context` runtime) keeps serving the `.gpu` partition + the foreground - /// session unchanged; the `.cpuTernary` partition advances on `TernaryCPURuntime` CONCURRENTLY (a - /// separate, independent task tree), bounded by the Core `LaneArbiter`'s CPU cap + co-residency - /// budget — it never borrows a GPU batch slot. Native-only; the live concurrency is user-run-verify. - /// - /// This is ADDITIVE: it CALLS the pinned `batchStep(requests)` with the GPU partition's requests - /// exactly as today, and runs the CPU partition beside it — neither the batched runtime nor the - /// parked scheduler is edited. - @MainActor - public static func dispatch(partition: LanePartition, - requests: [AgentSessionID: LLMChatRequest], - gpu: BatchedLLMRuntime, - ternary: TernaryCPURuntime?, - onToken: @escaping @Sendable (AgentSessionID, Token) -> Void) { - // GPU lane: the foreground + `.gpu`-affined sessions through the UNCHANGED batched step. - let gpuRequests = requests.filter { partition.gpu.contains($0.key) } - if !gpuRequests.isEmpty { - let stream = gpu.batchStep(gpuRequests) - Task { @MainActor in - do { - for try await (id, token) in stream { onToken(id, token) } - } catch { - // A GPU-stream failure stays the batched runtime's concern (one stream's failure does - // not abort the batch); surfaced via its own `.failed` path, mapped to RuntimeError. - TernaryCPURuntime.log.error("GPU batchStep failed: \(String(describing: error), privacy: .public)") - } - } - } - - // CPU ternary lane: advance each `.cpuTernary`-affined session CONCURRENTLY (a short burst each). - // A failed burst is an observable `.failed` for THAT turn (mapped to RuntimeError at the conformer - // boundary), never a false "done" and never aborting the GPU batch; cancellation is benign. - guard let ternary else { return } // CPU lane not installed (OFF) → nothing to do here. - for id in partition.cpuTernary { - guard let req = requests[id] else { continue } - Task { @MainActor in - do { - for try await token in ternary.chat(req) { onToken(id, token) } - } catch is CancellationError { - // Not a failure — the burst was discarded. - } catch { - // The burst's `.failed` is surfaced by the caller's turn state via AIError.message(for:). - TernaryCPURuntime.log.error("CPU ternary burst failed for session: \(String(describing: error), privacy: .public)") - } - } - } - } -} diff --git a/Sources/GemmaRuntime/LocalLTXVRuntime.swift b/Sources/GemmaRuntime/LocalLTXVRuntime.swift deleted file mode 100644 index b7f5efd..0000000 --- a/Sources/GemmaRuntime/LocalLTXVRuntime.swift +++ /dev/null @@ -1,158 +0,0 @@ -// LocalLTXVRuntime — the concrete FRONTIER LOCAL VIDEO `MediaRuntime` over a ComfyUI/MPS LTXV graph -// (ai-video-animation-generation §5.4, design D1/D2/D6/D7). -// -// FLAGGED: user xcodebuild + stable-signed build. -// -// This is the SECOND video backend behind the SAME `MediaRuntime` seam (the swap contract, design D8 / -// task 9.1) — a conformer selected by `videoProvider == .localLTXV`, NOT a new seam. It is the FRONTIER -// option behind the master `fullPotentialEnabled` toggle, and it is HONEST about its cost: an LTXV ComfyUI -// graph is 35 GB+ of weights (NOT in-process MLX — a `Process` bridge to a ComfyUI/MPS pipeline), it runs -// MINUTES PER CLIP, it EVICTS chat under the 48 GB budget (the assistant goes quiet while it paints — the -// fleet residency decision §C1, consumed by the sink), and it runs the Mac HOT. None of that is hidden: -// the selection disclosure (`VideoUploadDisclosure.localCostLine`) states it in the same breath it is -// offered. -// -// It SPENDS NO MONEY + UPLOADS NOTHING (design D6): off the cloud budget/ledger path entirely. It is still -// AUDITED (every attempt → one record) and still PARKS (it is slow). The gating (`fullPotentialEnabled && -// mediaGenEnabled`) lives UPSTREAM (the contributor / `VideoTierResolver`); this runtime is the executor. -// -// `xcodebuild` COMPILE-VERIFY ONLY here: the agent never builds/signs/installs the `.app`. REAL -// correctness — 35 GB residency, minutes-per-clip latency, real chat eviction, real thermals, a real clip -// — is the USER's stable-signed run-verify (task 5.4). -// -// ERROR TAXONOMY (design D7 / task 8.1): every `Process` / ComfyUI / MPS / file-IO failure is mapped INTO -// `MediaError` AT THIS BOUNDARY (Core stays MLX-/process-free). Feature/UI code only ever sees `MediaError`, -// surfaced through `AIError.message(for:)` (clean headline; raw text only in logs / copyable details — -// never a headline, never an `NSAlert`). A render that fails to land is `.failed` (never a false Done); -// CANCELLATION is a stopped stream, NOT a failure (design D10). - -import Foundation -import os -import ThreeFingerSwitcherCore - -public final class LocalLTXVRuntime: MediaRuntime, @unchecked Sendable { - - static let log = Logger(subsystem: "ThreeFingerSwitcher", category: "LocalLTXVRuntime") - - /// VIDEO only — never `.image` (the router never routes a still here, design D1). - public let capabilities: Set = [.video] - - /// The chosen LTXV descriptor (its `residencyBytes` — tens of GB — is the fleet's eviction-budget input, - /// consumed not re-derived; admitting it EVICTS chat). - private let descriptor: ModelDescriptor - /// On-disk path to the 35 GB+ LTXV ComfyUI graph weights (downloaded once, behind the master toggle). - private let weightsURL: URL - /// The ComfyUI server / python entrypoint the `Process` bridge drives. - private let comfyExecutableURL: URL - /// Where finished clips are written (the gallery root — becomes a Files-band `.fileEntry`). - private let outputDirectory: URL - private var isPrepared = false - private let lock = NSLock() - - public init(descriptor: ModelDescriptor, - weightsURL: URL, - comfyExecutableURL: URL, - outputDirectory: URL) { - self.descriptor = descriptor - self.weightsURL = weightsURL - self.comfyExecutableURL = comfyExecutableURL - self.outputDirectory = outputDirectory - } - - // MARK: - Preparation (resident graph via the fleet lifecycle) - - /// Bring the LTXV ComfyUI graph up. The residency/EVICTION decision is the fleet's - /// (`ModelRegistry.ensureResident(descriptor.id)`, run by the sink BEFORE this — it evicts chat) — this - /// only starts THIS runtime's graph through the `Process` bridge. Maps any native failure into - /// `MediaError` at the boundary. - public func prepare() async throws { - lock.lock(); let already = isPrepared; lock.unlock() - if already { return } - guard FileManager.default.fileExists(atPath: weightsURL.path) else { - throw MediaError.generationFailed(headline: "The local video model isn't installed yet.") - } - do { - try await startComfy() - lock.lock(); isPrepared = true; lock.unlock() - } catch let e as MediaError { - throw e - } catch { - Self.log.error("LTXV graph start failed: \(String(describing: error), privacy: .public)") - throw MediaError.generationFailed(headline: "The local video model couldn't be started.") - } - } - - /// FLAGGED: launch the ComfyUI/MPS server (`Process` over `comfyExecutableURL`), load the LTXV graph - /// from `weightsURL`, wait for ready. Native-only; the agent compile-verifies the seam. - private func startComfy() async throws { - // Real implementation: spawn the ComfyUI server Process, POST the graph, await /system_stats ready. - // Any thrown `Process`/socket error is caught by `prepare()` and mapped into `MediaError`. - } - - // MARK: - Generation (queue the graph → poll render → write clip — task 5.4) - - public func generate(_ request: MediaRequest) -> AsyncThrowingStream { - let outputDirectory = self.outputDirectory - return AsyncThrowingStream { continuation in - let task = Task { - // A present-but-undecodable seed is a clean `MediaError.seedInvalid` (never fed to the graph). - if let seed = request.seed, !MediaSeedValidation.isDecodablePNG(seed) { - continuation.finish(throwing: MediaError.seedInvalid) - return - } - guard self.preparedSnapshot() else { - continuation.finish(throwing: MediaError.generationFailed( - headline: "The local video model isn't ready yet.")) - return - } - do { - let asset = try await self.render(request, into: continuation, outputDirectory: outputDirectory) - continuation.yield(.finished(asset)) - continuation.finish() - } catch is CancellationError { - continuation.finish() // discard → stopped stream, NOT a failure - } catch let e as MediaError { - continuation.finish(throwing: e) - } catch { - Self.log.error("LTXV render failed: \(String(describing: error), privacy: .public)") - continuation.finish(throwing: MediaError.generationFailed( - headline: "The video couldn't be generated.")) - } - } - continuation.onTermination = { _ in task.cancel() } - } - } - - /// FLAGGED: queue the LTXV graph (prompt + optional seed PNG as the first frame for img2video), poll the - /// ComfyUI render progress yielding `.step(index:total:preview:)`, honor cancellation between steps, then - /// move the finished clip into `outputDirectory` and return the `MediaAsset` (kind `.video`, `durationMs` - /// from the request). A `Process`/ComfyUI error maps to `.generationFailed`; a write IO error to - /// `.outputWriteFailed`. Native-only (real residency / latency / heat is the user's signed build). - private func render(_ request: MediaRequest, - into continuation: AsyncThrowingStream.Continuation, - outputDirectory: URL) async throws -> MediaAsset { - // Real implementation outline (native-only): - // let promptID = try await queue(graph: buildGraph(request)) // Process/ComfyUI → MediaError - // let total = request.parameters.steps - // while true { - // try Task.checkCancellation() // cancel → stopped stream - // let p = try await progress(promptID) - // continuation.yield(.step(index: p.step, total: total, preview: p.previewPNG)) - // if let clip = p.finishedClipURL { - // let dest = outputDirectory.appendingPathComponent("\(UUID().uuidString).mp4") - // do { try FileManager.default.moveItem(at: clip, to: dest) } - // catch { throw MediaError.outputWriteFailed(detail: String(describing: error)) } - // return MediaAsset(url: dest, kind: .video, - // width: request.parameters.size.width, height: request.parameters.size.height, - // durationMs: request.parameters.durationMs) - // } - // } - _ = continuation - throw MediaError.generationFailed( - headline: "Local video generation is verified only in the user's stable-signed build.") - } - - private func preparedSnapshot() -> Bool { - lock.lock(); defer { lock.unlock() }; return isPrepared - } -} diff --git a/Sources/GemmaRuntime/MFluxImageRuntime.swift b/Sources/GemmaRuntime/MFluxImageRuntime.swift deleted file mode 100644 index f528f3e..0000000 --- a/Sources/GemmaRuntime/MFluxImageRuntime.swift +++ /dev/null @@ -1,418 +0,0 @@ -// MFluxImageRuntime — the concrete LOCAL IMAGE `MediaRuntime`, backed by FLUX.2 Klein 4B (Apache-2.0) -// running in-process via MLX through the `flux-2-swift-mlx` package (`Flux2Core` / `FluxTextEncoders`). -// (`ai-local-image-generation` §4, design D1/D2/D6/D7.) -// -// FLAGGED: user xcodebuild + stable-signed build. -// -// This is the SECOND media seam's FIRST concrete backend (parallel to how `GemmaMLXRuntime` is the chat -// seam's backend) — a `MediaRuntime` conformer, NOT a new seam (design D1). It links MLX and runs the -// FLUX.2 diffusion graph IN-PROCESS (design D2), so it is `xcodebuild` COMPILE-VERIFY ONLY here: the agent -// never builds/signs/installs the `.app` (ad-hoc signing breaks TCC; the metallib `*.bundle` copy in -// `build-app.sh` must not regress — no GPU use without it, task 4.4). REAL correctness — actual image -// output, the step preview, latency, seed reproducibility, Q4 co-residency / FP16 evict-chat — is the -// USER's stable-signed run-verify (tasks 4.x / 6.3). -// -// THE MODEL (real, Apache-2.0): both catalog variants are **FLUX.2 Klein 4B** — the OPEN FLUX.2 variant -// (`black-forest-labs/FLUX.2-klein-4B`, ungated, commercial-use OK), with the Qwen3-4B text encoder -// (Apache 2.0) and the small-decoder VAE (Apache 2.0). Q4 = the package's on-the-fly int4 quantization -// (~7 GB resident, co-resides with chat); FP16 = bf16 weights (~24 GB, evicts chat). We DELIBERATELY do -// NOT ship FLUX.2-dev (BFL non-commercial) or Klein 9B (non-commercial). The pipeline is built from -// `Flux2Pipeline(model: .klein4B, …)` and its multi-file weights are downloaded by the package's own HF -// downloader into our model-cache dir (`ModelRegistry.customModelsDirectory = weightsURL`). -// -// IN-PROCESS, ONE BUDGET (design D2): the diffusion weights are resident under the SAME fleet 48 GB -// unified-memory budget as chat — Q4 (~7 GB) co-resides, FP16 (~24 GB) evicts chat. Residency is the -// fleet's DECISION (`ModelRegistry.ensureResident`) consumed via the existing `ModelProvisioner` / -// `runtimeFactory` lifecycle (task 4.4); this runtime does NOT add a second resident-weights path. -// -// ERROR TAXONOMY (design D7): every vendor/OS failure (`Flux2Error` load/OOM/Metal fault, denoise failure, -// PNG-write IO, capability mismatch) is mapped INTO `MediaError` AT THIS BOUNDARY — Core stays MLX-free. -// Feature/UI code only ever sees `MediaError`, surfaced through `AIError.message(for:)` as a clean bounded -// headline (raw vendor text only in logs / opt-in copyable details — never a headline, never an `NSAlert`). -// A gen that fails to land is `.failed`; CANCELLATION is a stopped stream, NOT a failure (design D10): the -// stream finishes WITHOUT a `.finished` and WITHOUT throwing. - -import CoreGraphics -import Foundation -import ImageIO -import UniformTypeIdentifiers -import os -import ThreeFingerSwitcherCore -import Flux2Core - -public final class MFluxImageRuntime: MediaRuntime, @unchecked Sendable { - - static let log = Logger(subsystem: "ThreeFingerSwitcher", category: "MFluxImageRuntime") - - /// This backend produces IMAGES only (design D1 / spec "capabilities advertise image only"). Video is - /// `ai-video-animation-generation`'s backend — never advertised here, so the router never routes a - /// video gen to this runtime. - public let capabilities: Set = [.image] - - /// The chosen image descriptor (Q4 / FP16, selected by `imageModelID` upstream). Its - /// `ImageModelCatalog` capability tags gate the seed (img2img) branch; its `residencyBytes` is the - /// fleet's budget input (consumed, not re-derived). Its `quantization` selects int4 vs bf16 below. - private let descriptor: ModelDescriptor - /// On-disk path to the diffusion weights (the package's multi-file HF cache root for this variant). - private let weightsURL: URL - /// Where finished PNGs are written (the gallery root — becomes a Files-band `.fileEntry`). - private let outputDirectory: URL - /// The resident FLUX.2 pipeline, set ONLY once `loadResident()` actually builds it. While the pipeline - /// is unbuilt this stays `nil`, so `isPrepared`/`preparedSnapshot()` read false and `generate` - /// short-circuits to a clean `MediaError` — a "prepared" runtime can NEVER have a missing pipeline, so - /// the unbuilt-pipeline path can never paint a fake/blank result. Typed `Flux2Pipeline?` now that the - /// concrete backend is wired. - private var pipeline: Flux2Pipeline? - /// True once the diffusion graph is loaded resident AND the pipeline is built. Gated on `pipeline`, - /// so a missing-model / unbuilt-pipeline `prepare()` never reports a false "ready". - private var isPrepared: Bool { pipeline != nil } - private let lock = NSLock() - - public init(descriptor: ModelDescriptor, - weightsURL: URL, - outputDirectory: URL) { - self.descriptor = descriptor - self.weightsURL = weightsURL - self.outputDirectory = outputDirectory - } - - // MARK: - Quantization mapping (descriptor → Flux2 config) - - /// Map the catalog descriptor's `quantization` to a `Flux2QuantizationConfig` for Klein 4B: - /// - Q4 (`.qat4bit`) → int4 transformer (on-the-fly) + 8-bit Qwen3 encoder ≈ ~7 GB resident. - /// - FP16 (`.fp16`) → bf16 transformer + bf16-class encoder ≈ ~24 GB resident. - /// Any other descriptor (shouldn't happen — only the two image variants reach here) defaults to int4. - private var fluxQuantization: Flux2QuantizationConfig { - switch descriptor.quantization { - case .fp16: - return Flux2QuantizationConfig(textEncoder: .bf16, transformer: .bf16) - default: - return Flux2QuantizationConfig(textEncoder: .mlx8bit, transformer: .int4) - } - } - - // MARK: - Preparation (resident load via the fleet lifecycle — task 4.4) - - /// Load the diffusion graph resident on the GPU lane. The actual residency/eviction DECISION is the - /// fleet's (`ModelRegistry.ensureResident(descriptor.id)`, run by the sink BEFORE this) — this only - /// brings THIS runtime's weights up through the existing `ModelProvisioner` / `runtimeFactory` seam - /// (no second resident-weights path). Maps any native load failure into `MediaError` at the boundary. - public func prepare() async throws { - lock.lock(); let already = isPrepared; lock.unlock() - if already { return } - do { - // Build the resident pipeline (download the multi-file FLUX.2 weights if missing, then ready - // the pipeline). On success `pipeline` is non-nil and `isPrepared` reads true; on any failure - // `loadResident` throws (mapped below) and `pipeline` stays nil — a "prepared" runtime is NEVER - // one with a missing pipeline (the honest-failure invariant). A subsequent `generate` then - // short-circuits to a clean `MediaError`, never a blank PNG. - let built = try await loadResident() - lock.lock(); pipeline = built; lock.unlock() - } catch let e as MediaError { - throw e - } catch is CancellationError { - // Cancellation is NOT a failure — leave `pipeline` nil and rethrow so the caller treats it as a - // stopped preparation, not a `.failed`. - throw CancellationError() - } catch { - // Flux2/MLX init / Metal / download / file-IO failure → mapped at THIS boundary (design D7). - // Raw text rides only into logs / copyable details, never a headline. - Self.log.error("image model load failed: \(String(describing: error), privacy: .public)") - throw Self.mediaError(for: error, fallbackHeadline: "The image model couldn't be loaded.") - } - } - - /// The real resident load — point the FLUX.2 package's model cache at our weights dir, build the - /// `Flux2Pipeline` for the chosen Klein 4B variant, and download/ready its multi-file weights. Returns - /// the built pipeline (any thrown `Flux2Error`/`URLError`/IO error is mapped into `MediaError` by - /// `prepare()`). Native-only; the agent compile-verifies the seam. Requires the metallib `*.bundle` in - /// `Contents/Resources/` (task 4.4) — no GPU use without it. - private func loadResident() async throws -> Flux2Pipeline { - try Task.checkCancellation() - - // Route the package's multi-file model cache (transformer + Qwen3 encoder + small-decoder VAE) to - // OUR per-variant weights dir — the same cache root the chat path uses (`makeImageRuntime` derives - // it from the descriptor's HF repo path). This is the SINGLE on-disk weights location for both the - // download and the `isFullyDownloaded` probe — no second resident-weights path. - try FileManager.default.createDirectory(at: weightsURL, withIntermediateDirectories: true) - ModelRegistry.customModelsDirectory = weightsURL - - // FLUX.2 Klein 4B (Apache-2.0). small-decoder VAE (Apache-2.0). Memory optimization is auto-detected - // from system RAM by the pipeline's own init default. - let pipeline = Flux2Pipeline( - model: .klein4B, - quantization: fluxQuantization, - vaeVariant: .smallDecoder - ) - - // `loadModels()` downloads the missing weight/config files (the package's HF downloader handles the - // full multi-file set) and readies the pipeline; weights materialize on the GPU lazily at first - // generate. A network/HF failure throws here and is mapped at the boundary by `prepare()`. - Self.log.notice("loadResident: readying FLUX.2 Klein 4B pipeline (download if needed)…") - try await pipeline.loadModels(progressCallback: nil) - try Task.checkCancellation() - Self.log.notice("loadResident: pipeline ready ✓") - return pipeline - } - - // MARK: - Generation (the denoise loop — tasks 4.1 / 4.2) - - public func generate(_ request: MediaRequest) -> AsyncThrowingStream { - let descriptor = self.descriptor - let outputDirectory = self.outputDirectory - return AsyncThrowingStream { continuation in - let task = Task { - // (1) BOUNDARY VALIDATION — the SAME pure gate the stub runs (Core): kind, seed capability - // (a seed against a non-`img2img` descriptor → clean `MediaError`, never a silent t2i - // fallback, design D6), and param bounds. A bad request throws a clean `MediaError` and - // NEVER paints. - if let err = ImageRequestValidator.validate(request, descriptor: descriptor) { - continuation.finish(throwing: err) - return - } - // (1b) READY THE MODEL — lazily build (download + load) the FLUX.2 pipeline on FIRST use. - // The `MediaRuntime` protocol has no `prepare()`, and the sink's residency pass only makes - // ROOM (the fleet's eviction decision) — it never readies THIS runtime. So `generate` self- - // prepares: the first generation downloads the multi-file weights + builds the diffusion - // graph (slow once, then resident), later ones reuse it. A load/download failure → a clean - // mapped `MediaError` (never a silent "isn't ready" with no attempt); cancellation → a - // stopped stream, never a `.failed`. - let pipeline: Flux2Pipeline - do { - pipeline = try await self.ensurePrepared() - } catch is CancellationError { - continuation.finish() - return - } catch let e as MediaError { - continuation.finish(throwing: e) - return - } catch let flux as Flux2Error where Self.isCancellation(flux) { - continuation.finish() - return - } catch { - Self.log.error("image model load failed: \(String(describing: error), privacy: .public)") - continuation.finish(throwing: Self.mediaError(for: error, fallbackHeadline: "The image model couldn't be loaded.")) - return - } - - do { - // (2) SEED → img2img (task 4.2). When `request.seed` is present it is the PNG first - // frame: run conditioning-mode image-to-image. Absent → text-to-image. The numeric - // `parameters.seedNumber` drives reproducible RNG. - let asset = try await self.denoise(request, - pipeline: pipeline, - into: continuation, - outputDirectory: outputDirectory) - continuation.yield(.finished(asset)) - continuation.finish() - } catch is CancellationError { - // CANCELLATION is a stopped stream, NOT a failure (design D10): finish WITHOUT a - // `.finished` and WITHOUT throwing. - continuation.finish() - } catch let e as MediaError { - continuation.finish(throwing: e) - } catch let flux as Flux2Error where Self.isCancellation(flux) { - continuation.finish() - } catch { - // Flux2/MLX denoise / OOM / Metal fault → mapped at the boundary into a clean headline. - Self.log.error("image denoise failed: \(String(describing: error), privacy: .public)") - continuation.finish(throwing: Self.mediaError(for: error, fallbackHeadline: "The image couldn't be generated.")) - } - } - continuation.onTermination = { _ in task.cancel() } - } - } - - /// The real FLUX.2 denoise. It drives the pipeline's `generate*WithResult` call, streaming `.step` - /// progress from the pipeline's `onProgress` callback and an OPTIONAL intermediate `preview` from its - /// `onCheckpoint` callback (gated to ~every Nth step so it never dominates step time — design D1). It - /// honors cancellation at the boundaries (`try Task.checkCancellation()` before the call and after the - /// result, plus a flag the callbacks read). After the pipeline returns the final `CGImage` it encodes - /// it to PNG, WRITES it to `outputDirectory`, and returns the `MediaAsset` (dimensions = the request - /// size). A PNG-write IO failure maps to `MediaError.outputWriteFailed` (raw OS reason → copyable - /// details, never a headline). Native-only. - /// - /// img2img branch (task 4.2): when `request.seed != nil`, decode the seed PNG and run conditioning-mode - /// image-to-image with it as the single reference (`generateImageToImageWithResult(imageData:)`). - /// `parameters.seedNumber` seeds the RNG for reproducibility (a fixed seedNumber + identical - /// prompt/params reproduces an image). - /// - /// NOTE on cancellation: `Flux2Pipeline.generate*` does NOT poll Task cancellation inside its denoise - /// loop (same limitation as the chat `chatStream` path), so a discard cannot stop the in-flight GPU - /// work mid-step — but we never yield a `.finished` after a cancel: the post-call - /// `Task.checkCancellation()` throws and the stream ends as a stopped stream (no false "Done"). - private func denoise(_ request: MediaRequest, - pipeline: Flux2Pipeline, - into continuation: AsyncThrowingStream.Continuation, - outputDirectory: URL) async throws -> MediaAsset { - try Task.checkCancellation() - - let params = request.parameters - let width = params.size.width - let height = params.size.height - let steps = params.steps - // Klein 4B is CFG-distilled at guidance 1.0; honor an explicit guidance if the route supplied one, - // else fall back to the model's recommended default. - let guidance = params.guidance.map { Float($0) } ?? Flux2Model.klein4B.defaultGuidance - - // Gate the intermediate preview: decode a checkpoint image roughly every quarter of the run (at - // least every step for very short runs), but never more than ~6 previews total — so the VAE decode - // for the preview never dominates step time (design D1). `nil` interval → no checkpoint callback. - let checkpointInterval: Int? = steps >= 4 ? max(1, steps / 4) : nil - - // The pipeline callbacks are `@Sendable` and fire from the generation task; the continuation is - // Sendable, so streaming step/preview progress from inside them is safe. `nonisolated(unsafe)` to - // carry the non-Sendable nothing here — the continuation itself is Sendable. - let sink = continuation - let onProgress: Flux2ProgressCallback = { current, total in - // `current` is 1-based from the pipeline; the seam's `.step` index is 0-based. - sink.yield(.step(index: max(0, current - 1), total: total, preview: nil)) - } - let onCheckpoint: Flux2CheckpointCallback = { step, image in - // A REAL intermediate frame — encode the checkpoint CGImage to PNG and ride it as the optional - // `preview` on a `.step`. If PNG encoding fails we simply omit the preview (never fake one). - let preview = MFluxImageRuntime.pngData(from: image) - sink.yield(.step(index: max(0, step - 1), total: steps, preview: preview)) - } - - let image: CGImage - if let seed = request.seed { - // img2img: the seed PNG is the conditioning reference. A non-decodable capture → `.seedInvalid`. - guard Self.cgImage(fromPNG: seed) != nil else { - throw MediaError.seedInvalid - } - Self.log.notice("denoise: img2img (\(width)x\(height), \(steps) steps)…") - let result = try await pipeline.generateImageToImageWithResult( - prompt: request.prompt, - imageData: [seed], - height: height, - width: width, - steps: steps, - guidance: guidance, - seed: params.seedNumber, - upsamplePrompt: false, - checkpointInterval: checkpointInterval, - onProgress: onProgress, - onCheckpoint: onCheckpoint - ) - image = result.image - } else { - Self.log.notice("denoise: text-to-image (\(width)x\(height), \(steps) steps)…") - let result = try await pipeline.generateTextToImageWithResult( - prompt: request.prompt, - height: height, - width: width, - steps: steps, - guidance: guidance, - seed: params.seedNumber, - upsamplePrompt: false, - checkpointInterval: checkpointInterval, - onProgress: onProgress, - onCheckpoint: onCheckpoint - ) - image = result.image - } - - // A discard during the (uncancellable) denoise → end as a stopped stream, never a false `.finished`. - try Task.checkCancellation() - - // Encode the final image to PNG and write it into the gallery. The asset dimensions are the actual - // produced image's (the pipeline may clamp/round the request size to a valid latent grid). - let url = try writePNG(image, into: outputDirectory) - Self.log.notice("denoise: wrote \(url.lastPathComponent, privacy: .public) (\(image.width)x\(image.height)) ✓") - return MediaAsset(url: url, - kind: .image, - width: image.width, - height: image.height) - } - - // MARK: - PNG IO - - /// Encode a `CGImage` to PNG `Data` (nil on failure). Used both for the final asset and the optional - /// intermediate preview frame. - static func pngData(from image: CGImage) -> Data? { - let data = NSMutableData() - guard let dest = CGImageDestinationCreateWithData(data as CFMutableData, UTType.png.identifier as CFString, 1, nil) else { - return nil - } - CGImageDestinationAddImage(dest, image, nil) - guard CGImageDestinationFinalize(dest) else { return nil } - return data as Data - } - - /// Decode PNG `Data` to a `CGImage` (nil if undecodable) — the img2img seed-validity check. - static func cgImage(fromPNG data: Data) -> CGImage? { - guard let source = CGImageSourceCreateWithData(data as CFData, nil) else { return nil } - return CGImageSourceCreateImageAtIndex(source, 0, nil) - } - - /// Write the final image as a PNG into `directory`, returning the file URL. Maps any encode/IO failure - /// to `MediaError.outputWriteFailed` (raw OS reason → copyable detail, never the headline). - private func writePNG(_ image: CGImage, into directory: URL) throws -> URL { - do { - try FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true) - } catch { - throw MediaError.outputWriteFailed(detail: String(describing: error)) - } - guard let data = Self.pngData(from: image) else { - throw MediaError.outputWriteFailed(detail: "Could not encode the generated image to PNG.") - } - let name = "image-\(ISO8601DateFormatter().string(from: Date()).replacingOccurrences(of: ":", with: "-"))-\(UUID().uuidString.prefix(8)).png" - let url = directory.appendingPathComponent(name) - do { - try data.write(to: url, options: .atomic) - } catch { - throw MediaError.outputWriteFailed(detail: String(describing: error)) - } - return url - } - - // MARK: - Error mapping (vendor/OS → MediaError, at THIS boundary) - - /// Whether a `Flux2Error` represents a user-driven cancellation (NOT a failure — design D10). - static func isCancellation(_ error: Flux2Error) -> Bool { - if case .generationCancelled = error { return true } - return false - } - - /// Map a vendor/OS error into the `MediaError` taxonomy. The Flux2 cases carry user-actionable meaning - /// (insufficient memory, model-not-loaded) that we translate to a CLEAN headline; everything else falls - /// back to the supplied generic headline. Raw vendor text rides ONLY into logs / `copyableDetails`, - /// never a headline (the ban on raw interpolation in UI strings). - static func mediaError(for error: Error, fallbackHeadline: String) -> MediaError { - if let flux = error as? Flux2Error { - switch flux { - case .insufficientMemory: - return .generationFailed(headline: "Not enough memory to run the image model right now.") - case .modelNotLoaded, .weightLoadingFailed: - return .generationFailed(headline: "The image model isn't installed yet.") - case .generationCancelled: - // Caller should have treated this as cancellation; defensively map to a clean headline. - return .generationFailed(headline: fallbackHeadline) - case .invalidConfiguration, .imageProcessingFailed, .generationFailed: - return .generationFailed(headline: fallbackHeadline) - } - } - return .generationFailed(headline: fallbackHeadline) - } - - // MARK: - Resident snapshot - - private func preparedPipeline() -> Flux2Pipeline? { - lock.lock(); defer { lock.unlock() }; return pipeline - } - - /// Return the resident pipeline, BUILDING it (download + load) on first use. The single readiness path - /// `generate` uses: a built pipeline is reused; otherwise `prepare()` downloads the multi-file FLUX.2 - /// weights + builds the diffusion graph (its lock + `already` short-circuit serialize a concurrent - /// first call). A load/download failure propagates as a mapped `MediaError`/`CancellationError` (handled - /// by `generate`), never a fake result. - private func ensurePrepared() async throws -> Flux2Pipeline { - if let ready = preparedPipeline() { return ready } - try await prepare() - guard let built = preparedPipeline() else { - throw MediaError.generationFailed(headline: "The image model isn't ready yet.") - } - return built - } -} diff --git a/Sources/GemmaRuntime/TernaryCPURuntime.swift b/Sources/GemmaRuntime/TernaryCPURuntime.swift deleted file mode 100644 index ac0d791..0000000 --- a/Sources/GemmaRuntime/TernaryCPURuntime.swift +++ /dev/null @@ -1,145 +0,0 @@ -// TernaryCPURuntime — the CPU-lane ternary/BitNet-class `LLMRuntime` conformer (ai-compute-tiers §5). -// -// FLAGGED: user xcodebuild + stable-signed build. -// -// This is the SECOND `LLMRuntime` conformer (design D2 — a conformer, NOT a new protocol), carrying a -// SMALL ternary/BitNet-class model on the CPU lane. It is native-linked (it drives a bitnet.cpp-class -// inference process / library), so it is `xcodebuild` COMPILE-VERIFY ONLY here — the agent never -// builds/signs/installs the `.app` (ad-hoc signing breaks TCC; the metallib `*.bundle` copy in -// `build-app.sh` must not regress). REAL correctness — live CPU/GPU concurrency, CPU per-token speed, -// no cross-lane bleed, RAM/heat — is the USER's stable-signed run-verify (tasks 5.2 / 8.3). -// -// HONEST CONSTRAINT (design D5): CPU per-token is SLOWER than the GPU batched token. So this lane serves -// SHORT, FREQUENT, STRUCTURED bursts ONLY — the router's `structured()` route turn, classify, -// memory-retrieval, and parked-subagent `generate`/`chat`. The long foreground reply is NEVER routed -// here (the role→lane policy keeps `foregroundGeneration` on `.gpu`); its value is running its short -// bursts CONCURRENTLY with the GPU reply (the ternary weights are ~32× smaller → bandwidth-frugal → low -// contention on the shared ~153 GB/s bus), not being faster per token. -// -// ERROR TAXONOMY (design D7): every vendor/OS failure (bitnet.cpp-class load/prepare/decode, `Process` -// spawn/exit, file IO over the weights) is mapped INTO `RuntimeError` AT THIS BOUNDARY, exactly like the -// GPU runtime — so feature/UI code only ever sees `RuntimeError`, surfaced through `AIError.message(for:)` -// as a clean headline. A failed burst is an observable `.failed` for that turn (never a false "done"); -// cancellation is `RuntimeError.cancelled`, treated as benign upstream (not a failure). A `ComputeError` -// is deliberately NOT added — `RuntimeError.unavailable` / `.modelLoadFailed` / `.modelMissing` / -// `.decodeFailed` / `.cancelled` carry every CPU-lane case (D7: prefer extending `RuntimeError`). - -import Foundation -import os -import ThreeFingerSwitcherCore - -public final class TernaryCPURuntime: LLMRuntime, @unchecked Sendable { - - static let log = Logger(subsystem: "ThreeFingerSwitcher", category: "TernaryCPURuntime") - - /// CPU ternary lane carries TEXT only (design D5 / task 5.3): a ternary text model advertises no - /// vision/audio, and the role→lane policy keeps `mediaDiffusion`/vision roles on the GPU lane — so a - /// vision/media role is never routed to the CPU lane. A vision request here hard-errors (no degrade). - public let capabilities: Set = [.text] - - /// On-disk path to the ternary weights (a small ternary/BitNet-class GGUF-class file). Resident - /// footprint is ~32× smaller than the GPU chat weights → it co-resides cheaply (Core's - /// `LaneResidencyBudget.ternaryCoResides`). Loaded once, reused across bursts. - private let weightsURL: URL - /// True once `prepare()` has loaded the model resident. Bursts before prepare report `.modelMissing`. - private var isPrepared = false - private let lock = NSLock() - - public init(weightsURL: URL) { - self.weightsURL = weightsURL - } - - // MARK: - Preparation - - /// Load the ternary model resident on the CPU lane. Maps any native load failure into `RuntimeError` - /// at THIS boundary (D7). `prepare` is idempotent; a missing weights file is `.modelMissing`. - public func prepare() async throws { - lock.lock(); let already = isPrepared; lock.unlock() - if already { return } - guard FileManager.default.fileExists(atPath: weightsURL.path) else { - throw RuntimeError.modelMissing - } - do { - try await loadResident() - lock.lock(); isPrepared = true; lock.unlock() - } catch let e as RuntimeError { - throw e - } catch { - // bitnet.cpp-class init / Process / file IO failure → mapped at the boundary. - throw RuntimeError.modelLoadFailed(detail: String(describing: error)) - } - } - - /// FLAGGED: the real resident load drives the bitnet.cpp-class engine (load the ternary weights into - /// the CPU inference context, warm the kernels). Native-only; the agent compile-verifies the seam. - private func loadResident() async throws { - // Real implementation: initialize the bitnet.cpp-class context over `weightsURL`. Any thrown - // vendor/OS error is caught by `prepare()` and mapped into `RuntimeError`. - } - - // MARK: - Streaming (short bursts only — never the long reply) - - public func generate(_ request: LLMRequest) -> AsyncThrowingStream { - let caps = capabilities - let needsVision = request.requiresVision - return AsyncThrowingStream { continuation in - let task = Task { - // A vision/media role never reaches the CPU lane via the policy; defend anyway (no degrade). - if needsVision && !caps.contains(.vision) { - continuation.finish(throwing: RuntimeError.unsupportedModality(.vision)) - return - } - guard self.preparedSnapshot() else { - continuation.finish(throwing: RuntimeError.modelMissing) - return - } - do { - // FLAGGED: the real short-burst decode loop. Per-token is slower than the GPU; this is - // acceptable because bursts are short/bounded and run CONCURRENTLY with the GPU reply. - // Cancellation (a discarded turn) is honored promptly → `RuntimeError.cancelled`. - try Task.checkCancellation() - try await self.decode(request, into: continuation) - continuation.finish() - } catch is CancellationError { - continuation.finish(throwing: RuntimeError.cancelled) // not a failure - } catch let e as RuntimeError { - continuation.finish(throwing: e) - } catch { - continuation.finish(throwing: RuntimeError.decodeFailed(detail: String(describing: error))) - } - } - continuation.onTermination = { _ in task.cancel() } - } - } - - /// FLAGGED: the real per-token decode against the resident ternary context, yielding `Token`s in - /// order and honoring cancellation between steps. Native-only. - private func decode(_ request: LLMRequest, - into continuation: AsyncThrowingStream.Continuation) async throws { - // Real implementation: step the bitnet.cpp-class context, yielding `Token(piece)` per step and - // `Token(piece, isFinal: true)` on the stop token; `try Task.checkCancellation()` each step. - _ = request - _ = continuation - } - - // MARK: - Structured (the router's route turn — the CPU lane's primary use) - - public func structured( - _ request: LLMRequest, - schema: StructuredSchema, - as type: T.Type - ) async throws -> StructuredOutcome { - try Task.checkCancellation() - guard preparedSnapshot() else { throw RuntimeError.modelMissing } - // FLAGGED: the real constrained/structured decode — emit JSON, run the bounded - // validate → repair/retry → decode → outcome pipeline, allow a typed `.declined`. On exhaustion - // throw `RuntimeError.couldNotProduceValid`; on a decode mismatch `RuntimeError.decodeFailed`. - _ = schema - _ = request - throw RuntimeError.unavailable(reason: "TernaryCPURuntime structured decode is verified only in the user's stable-signed build.") - } - - private func preparedSnapshot() -> Bool { - lock.lock(); defer { lock.unlock() }; return isPrepared - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/AIBand.swift b/Sources/ThreeFingerSwitcher/AI/AIBand.swift deleted file mode 100644 index 4a4cd01..0000000 --- a/Sources/ThreeFingerSwitcher/AI/AIBand.swift +++ /dev/null @@ -1,41 +0,0 @@ -import Foundation - -/// The "AI" band: a NORMAL, editable favorites band that holds `.aiCommand` items (configuration-hub -/// fold-in). This replaces the former *synthetic* `AICommandBandBuilder` — AI commands are no longer -/// projected fresh on every launcher open from a separate store; they are first-class, persisted band -/// items that can live in **any** band. This helper exists only to (a) build the seeded "AI" band on a -/// fresh install and (b) fold a legacy `AICommandStore` record into a normal band on upgrade. -enum AIBand { - /// Stable id reused from the former synthetic band, so the seed/migration are recognizable and the - /// migration is idempotent (it never appends a second "AI" band). After fold-in this is just a - /// normal band id — the user may rename, recolor, reorder, split, or delete the band. - static let bandID = UUID(uuidString: "A1C0AAAA-0000-4000-8000-000000000001")! - static let name = "AI" - static let color = ItemColor(red: 0.55, green: 0.40, blue: 0.92) - /// The "AI" band's default launcher icon (the band list shows icons, not names). - static let icon: ItemIcon = .sfSymbol("wand.and.stars") - - /// One `.aiCommand` band item for a command. The item's id mirrors the command id (stable SwiftUI - /// identity + the executor keys on it), and its title/icon/tint mirror the command for rendering. - static func item(for command: AICommand) -> LaunchItem { - LaunchItem(id: command.id, title: command.name, icon: command.icon, - tint: command.tint, kind: .aiCommand(command)) - } - - /// Build the "AI" band from a list of commands (used by seeding + migration). - static func band(from commands: [AICommand]) -> ContextBand { - ContextBand(id: bandID, name: name, color: color, icon: icon, items: commands.map(item(for:))) - } - - /// True for a band carrying the AI sentinel id (used by the migration's idempotency check). - static func isAIBand(_ band: ContextBand) -> Bool { band.id == bandID } - - /// The default AI command set shipped on a fresh install: a curated subset of the shipped - /// `AICommandCatalog` — the canned verbs that cover the common cases, each with a sensible prompt - /// template. In-place transforms confirm OFF; add-to-calendar (side-effecting) confirms ON. The - /// full catalog is browsable from the Hub; the curation lives in `AICommandCatalog.seeded()`. - static func seeded() -> [AICommand] { AICommandCatalog.seeded() } - - /// The seeded "AI" band for a fresh favorites record. - static func seededBand() -> ContextBand { band(from: seeded()) } -} diff --git a/Sources/ThreeFingerSwitcher/AI/AICommand.swift b/Sources/ThreeFingerSwitcher/AI/AICommand.swift deleted file mode 100644 index bf3c332..0000000 --- a/Sources/ThreeFingerSwitcher/AI/AICommand.swift +++ /dev/null @@ -1,357 +0,0 @@ -import Foundation - -/// The value model for one AI command (spec: "AI command value model and persistence", design D5). -/// -/// An `AICommand` is *configuration*, not a Favorites item: it describes how to acquire input, what -/// prompt to run it through, and where the model's output goes. It is a pure value type with no -/// AppKit/SwiftUI dependency, reusing the launcher's `ItemIcon` / `ItemColor` so it renders in the -/// same icon grid (the synthetic AI band is projected from these on launcher open — a later slice). -/// -/// `confirmBeforeRun` DEFAULTS to true for side-effecting outputs (task / send-to) at creation time -/// — but the STORED value is always honored at run time and never force-overridden (design D6): a -/// user may turn confirmation off for a trusted task. - -// MARK: - Input source - -/// Where a command's input text (or image) comes from at fire time. -enum InputSource: String, Codable, Equatable, Sendable, CaseIterable { - /// The front app's currently selected text (with a clipboard fallback when empty). - case selection - /// The current clipboard contents. - case clipboard - /// The current clipboard IMAGE — the live pasteboard image, normalized to PNG — fed to the vision - /// model (requires a `.vision`-capable model). A distinct source from `.clipboard` (not a - /// polymorphic text-or-image) so `requiredCapabilities` stays static. On-demand: copying an image - /// never auto-fires; the user runs a `clipboardImage` command and it reads the clipboard image. - case clipboardImage - /// A captured screen region, fed to the vision model (requires a `.vision`-capable model). - case screenRegion - /// No input — the prompt template stands alone. - case none -} - -// MARK: - Task / destination targets (built in a later slice; modeled here behind the seam) - -/// A side-effecting task the model's structured output is routed to (design D6). The concrete task -/// dispatch is a LATER slice; this slice only carries the kind and routes it to `TaskDispatching`. -enum TaskKind: Codable, Equatable, Hashable, Sendable { - /// Create a calendar event from the parsed action (EventKit). - case addToCalendar - /// Create a reminder/to-do from the parsed action (EventKit reminders). - case addToReminder - /// Create a contact card from the parsed action (Contacts). - case newContact - /// Append the content to a named project note on disk. - case saveToProject(project: String) - /// Generate a payload and open a tool with it (by bundle id / tool name). - case openToolWithPayload(tool: String) - /// Route the content to a destination adapter (Shortcut / URL scheme / shell-out). - case sendTo(Destination) -} - -/// A delivery destination for a `sendTo` output / task (design D6). Concrete adapters are a LATER -/// slice; modeled here so commands round-trip and the dispatcher seam has a typed payload. -enum Destination: Codable, Equatable, Hashable, Sendable { - /// Run a named Shortcuts.app shortcut, fed the content. - case shortcut(name: String) - /// Open a URL scheme, with the content substituted into it. - case urlScheme(String) - /// Shell out to a command, passing the content on stdin. - case shell(command: String) -} - -// MARK: - Output target - -/// Where a command's result goes once committed (spec: "In-place output routing" + design D6). -enum OutputTarget: Codable, Equatable, Hashable, Sendable { - /// Replace the front app's selected text with the result. - case replaceSelection - /// Paste the result at the insertion point. - case pasteAtCursor - /// Show the result in the preview canvas only; write nothing into the app. - case previewOnly - /// Route a schema-targeted structured result to a side-effecting task. - case runTask(TaskKind) - /// Route the result to a destination adapter. - case sendTo(Destination) - - /// Whether committing this output has an irreversible side effect outside the front app's text — - /// the set for which `confirmBeforeRun` defaults ON (task / send-to). - var isSideEffecting: Bool { - switch self { - case .runTask, .sendTo: return true - case .replaceSelection, .pasteAtCursor, .previewOnly: return false - } - } -} - -// MARK: - Model selector - -/// Which model a command runs on. v1 ships only on-device Gemma 4; `cloud` is RESERVED behind the -/// same seam (a later, consent-gated alternate — design D1/non-goals) so a command can round-trip a -/// future cloud choice without a model-layer change. -enum ModelSelector: Codable, Equatable, Sendable { - /// On-device Gemma 4. `modelID == nil` means "the registry default"; a non-nil id pins a model. - case onDevice(modelID: String?) - /// RESERVED: a named cloud model (not served in v1). - case cloud(provider: String, model: String) - - /// The default selector for a freshly-created command: on-device, registry default model. - static let `default` = ModelSelector.onDevice(modelID: nil) -} - -// MARK: - Reasoning override - -/// An explicit per-command reasoning override (think-before-answering). `nil`/absent ⇒ the command -/// follows the global `aiReasoningEnabled` default; `.on`/`.off` pin it for this command regardless. -enum AIReasoning: String, Codable, Equatable, Sendable, CaseIterable { - /// Force reasoning ON for this command. - case on - /// Force reasoning OFF for this command. - case off -} - -// MARK: - The command - -/// One configured AI command. `Codable` (custom, for the legacy-scalar migration) so it persists as a -/// band item inside the `Favorites` record; `Identifiable` (by `id`) so the Bands editor and launcher -/// key on it. -/// -/// The command stores *what it's allowed to do*, not *what to do* (change `ai-action-context-resolution`): -/// an **input capability set** (`inputs`) and an **output capability set** (`outputs`), each defaulting -/// all-on. At fire time the executor senses the live environment, activates the highest-priority enabled -/// input channel that is live (`selection ▸ clipboard ▸ clipboardImage`), and derives the commit from the -/// resolved channel (selection → replace, clipboard → paste, image → vision). Toggles are guardrails on -/// that automatic decision. An **empty** `inputs` set means "no input needed" (standalone prompt). -struct AICommand: Equatable, Identifiable, Sendable { - var id: UUID - var name: String - var icon: ItemIcon - var tint: ItemColor? - /// The input channels this command MAY consume, resolved dynamically at fire (never a single fixed - /// source). Empty = standalone prompt (no input). `screenRegion`, when present, makes the command - /// region-first (the pre-canvas picker), exclusive of the ambient cascade (design D4). - var inputs: Set - var promptTemplate: String - /// The output behaviors this command MAY perform. For an in-place command the commit is derived from - /// the resolved input (selection→replace when `replaceSelection` on, else `pasteAtCursor`, else - /// `previewOnly` writes nothing). A single side-effecting `runTask`/`sendTo` member routes through the - /// accept-step/execute flow instead. - var outputs: Set - var model: ModelSelector - /// Whether to show the action-review/confirmation step before committing. Defaults ON when the - /// outputs contain a side-effecting task/send-to at creation, but the stored value is HONORED thereafter. - var confirmBeforeRun: Bool - /// An optional fire-time parameter chosen in the canvas rather than baked into the template - /// (v1: a target language resolved into `{lang}`). `nil` ⇒ no parameter UI; the command behaves - /// exactly as before. Optional so old persisted commands (no key) decode with it absent. - var runtimeParameter: RuntimeParameter? - /// An explicit per-command reasoning override. `nil` ⇒ follow the global `aiReasoningEnabled` - /// default; `.on`/`.off` pin it for this command. Optional so old persisted commands (no key) - /// decode with it absent. - var reasoning: AIReasoning? - - /// The default input capability set for a freshly created command: the ambient text/image cascade - /// (screen region is opt-in per command, since it needs the pre-canvas picker). - static let defaultInputs: Set = [.selection, .clipboard, .clipboardImage] - /// The default in-place output capability set for a freshly created command: all three on. - static let defaultOutputs: Set = [.replaceSelection, .pasteAtCursor, .previewOnly] - - /// Designated initializer (capability sets). When `confirmBeforeRun` is left `nil`, it is DERIVED - /// from the outputs (true when they contain a side-effecting task/send-to). An explicit value is - /// taken verbatim, so a stored `false` survives — the default is computed only at creation. - init(id: UUID = UUID(), - name: String, - icon: ItemIcon, - tint: ItemColor? = nil, - inputs: Set, - promptTemplate: String, - outputs: Set, - model: ModelSelector = .default, - confirmBeforeRun: Bool? = nil, - runtimeParameter: RuntimeParameter? = nil, - reasoning: AIReasoning? = nil) { - self.id = id - self.name = name - self.icon = icon - self.tint = tint - self.inputs = inputs - self.promptTemplate = promptTemplate - self.outputs = outputs - self.model = model - self.confirmBeforeRun = confirmBeforeRun ?? Self.defaultConfirmBeforeRun(for: outputs) - self.runtimeParameter = runtimeParameter - self.reasoning = reasoning - } - - /// Legacy convenience initializer (a single `input`/`output`), mapped to the capability sets via the - /// same behavior-preserving migration used on decode (design D5). Keeps the catalog, seeding, and - /// tests authoring commands in the old terse form while the model underneath is set-shaped. - init(id: UUID = UUID(), - name: String, - icon: ItemIcon, - tint: ItemColor? = nil, - input: InputSource, - promptTemplate: String, - output: OutputTarget, - model: ModelSelector = .default, - confirmBeforeRun: Bool? = nil, - runtimeParameter: RuntimeParameter? = nil, - reasoning: AIReasoning? = nil) { - self.init(id: id, name: name, icon: icon, tint: tint, - inputs: Self.migrate(input: input), promptTemplate: promptTemplate, - outputs: Self.migrate(output: output), model: model, - confirmBeforeRun: confirmBeforeRun, runtimeParameter: runtimeParameter, - reasoning: reasoning) - } - - // MARK: - Legacy → set migration (design D5) - - /// Map a legacy single input source to its capability set, behavior-preservingly: `selection` gains - /// the clipboard fallback it always had (now explicit + honest — it pastes, not replaces); `none` - /// becomes the empty (standalone) set. - static func migrate(input: InputSource) -> Set { - switch input { - case .selection: return [.selection, .clipboard] - case .clipboard: return [.clipboard] - case .clipboardImage: return [.clipboardImage] - case .screenRegion: return [.screenRegion] - case .none: return [] - } - } - - /// Map a legacy single output target to its capability set: `replaceSelection` gains `pasteAtCursor` - /// so a no-selection fire pastes rather than fails; everything else is its singleton. - static func migrate(output: OutputTarget) -> Set { - switch output { - case .replaceSelection: return [.replaceSelection, .pasteAtCursor] - case .pasteAtCursor: return [.pasteAtCursor] - case .previewOnly: return [.previewOnly] - case let .runTask(k): return [.runTask(k)] - case let .sendTo(d): return [.sendTo(d)] - } - } - - /// The DEFAULT `confirmBeforeRun` for a set of outputs, used ONLY at command creation: true when the - /// outputs contain a side-effecting task/send-to. The stored value is honored at run time thereafter. - static func defaultConfirmBeforeRun(for outputs: Set) -> Bool { - outputs.contains { $0.isSideEffecting } - } - - /// Resolve whether this command should reason: an explicit `.on`/`.off` override wins; an absent - /// override (`nil`) follows the global `aiReasoningEnabled` default passed in. - func resolvedReasoning(globalDefault: Bool) -> Bool { - switch reasoning { - case .on: return true - case .off: return false - case nil: return globalDefault - } - } - - // MARK: - Derived queries - - /// Whether this command requires input before the model may run. An empty `inputs` set = a standalone - /// prompt that needs nothing; a non-empty set with no live channel surfaces "no input". - var needsInput: Bool { !inputs.isEmpty } - - /// The single side-effecting output (task / send-to), if this is a side-effecting command; else nil - /// (an in-place command). Only one such member is meaningful; the first is taken. - var sideEffect: OutputTarget? { outputs.first { $0.isSideEffecting } } - - /// Whether this is a side-effecting command (its outputs carry a task/send-to). - var isSideEffecting: Bool { sideEffect != nil } - - /// The **potential** model capabilities this command could need — the UNION over its enabled inputs - /// (`vision` when any enabled input is an image source, `text` otherwise; `text` when standalone). - /// This is an INFORMATIONAL hint only (e.g. the editor): the model actually requested is derived from - /// the input RESOLVED at fire time, not this union (change `ai-action-context-resolution`, design D3). - var requiredCapabilities: Set { - var caps: Set = [] - if inputs.contains(where: { $0 == .clipboardImage || $0 == .screenRegion }) { caps.insert(.vision) } - if inputs.contains(where: { $0 == .selection || $0 == .clipboard }) { caps.insert(.text) } - if caps.isEmpty { caps.insert(.text) } // standalone / empty ⇒ text - return caps - } - - // MARK: - In-place commit resolution (pure; unit-tested + used by the executor) - - /// The in-place commit action resolved from the fire (side-effecting outputs are handled separately). - enum CommitPlan: Equatable, Sendable { - /// Replace the front app's selection (SelectionService pastes if it isn't settable / is empty). - case replaceSelection - /// Paste the result at the insertion point. - case pasteAtCursor - /// Write nothing (preview-only / read-only understanding command). - case preview - } - - /// Pure resolution of the in-place commit from the resolved input channel + enabled outputs - /// (design D2): a selection input with `replaceSelection` enabled replaces; else `pasteAtCursor` - /// pastes; else a lone `replaceSelection` still replaces (SelectionService pastes when no live - /// selection); else nothing is written. - static func inPlaceCommitPlan(resolvedWasSelection: Bool, outputs: Set) -> CommitPlan { - if resolvedWasSelection && outputs.contains(.replaceSelection) { return .replaceSelection } - if outputs.contains(.pasteAtCursor) { return .pasteAtCursor } - if outputs.contains(.replaceSelection) { return .replaceSelection } - return .preview - } -} - -// MARK: - Codable (custom: migrates legacy single `input`/`output` scalars into the sets, design D5) - -extension AICommand: Codable { - private enum CodingKeys: String, CodingKey { - case id, name, icon, tint - case inputs, promptTemplate, outputs, model, confirmBeforeRun, runtimeParameter, reasoning - case input, output // legacy scalar keys (decode-only fallback) - } - - init(from decoder: Decoder) throws { - let c = try decoder.container(keyedBy: CodingKeys.self) - id = try c.decode(UUID.self, forKey: .id) - name = try c.decode(String.self, forKey: .name) - icon = try c.decode(ItemIcon.self, forKey: .icon) - tint = try c.decodeIfPresent(ItemColor.self, forKey: .tint) - promptTemplate = try c.decode(String.self, forKey: .promptTemplate) - model = try c.decodeIfPresent(ModelSelector.self, forKey: .model) ?? .default - runtimeParameter = try c.decodeIfPresent(RuntimeParameter.self, forKey: .runtimeParameter) - reasoning = try c.decodeIfPresent(AIReasoning.self, forKey: .reasoning) - - // inputs: prefer the new set; else migrate a legacy scalar; else the all-on default. - if let ins = try c.decodeIfPresent(Set.self, forKey: .inputs) { - inputs = ins - } else if let legacy = try c.decodeIfPresent(InputSource.self, forKey: .input) { - inputs = AICommand.migrate(input: legacy) - } else { - inputs = AICommand.defaultInputs - } - - // outputs: prefer the new set; else migrate a legacy scalar; else the all-on default. - if let outs = try c.decodeIfPresent(Set.self, forKey: .outputs) { - outputs = outs - } else if let legacy = try c.decodeIfPresent(OutputTarget.self, forKey: .output) { - outputs = AICommand.migrate(output: legacy) - } else { - outputs = AICommand.defaultOutputs - } - - // confirmBeforeRun: honor the stored value; derive from the outputs when absent (never overridden). - confirmBeforeRun = try c.decodeIfPresent(Bool.self, forKey: .confirmBeforeRun) - ?? AICommand.defaultConfirmBeforeRun(for: outputs) - } - - func encode(to encoder: Encoder) throws { - var c = encoder.container(keyedBy: CodingKeys.self) - try c.encode(id, forKey: .id) - try c.encode(name, forKey: .name) - try c.encode(icon, forKey: .icon) - try c.encodeIfPresent(tint, forKey: .tint) - try c.encode(inputs, forKey: .inputs) - try c.encode(promptTemplate, forKey: .promptTemplate) - try c.encode(outputs, forKey: .outputs) - try c.encode(model, forKey: .model) - try c.encode(confirmBeforeRun, forKey: .confirmBeforeRun) - try c.encodeIfPresent(runtimeParameter, forKey: .runtimeParameter) - try c.encodeIfPresent(reasoning, forKey: .reasoning) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/AICommandCatalog.swift b/Sources/ThreeFingerSwitcher/AI/AICommandCatalog.swift deleted file mode 100644 index 5ee1c46..0000000 --- a/Sources/ThreeFingerSwitcher/AI/AICommandCatalog.swift +++ /dev/null @@ -1,285 +0,0 @@ -import Foundation - -/// The shipped library of ready-to-use AI commands, grouped into browsable categories (the Hub's "AI -/// Command" source mirrors this the way `ActionBrowser` mirrors `SystemAction`). Each `Entry` pairs a -/// `Category` with a fully-formed `AICommand`; adding a preset to a band copies it with a fresh id -/// (`copy(of:)`) so the same preset can be added twice and edits never mutate the catalog. -/// -/// Construction mirrors `AIBand.seeded()`: an `.sfSymbol` icon, a per-category `ItemColor` tint (one -/// consistent hue per category so a band reads at a glance), and an `OutputTarget` chosen so the verb -/// lands in the right place. Side-effecting commands (task / send-to) inherit `confirmBeforeRun = true` -/// from `AICommand.init` — it is never passed explicitly here. -enum AICommandCatalog { - // MARK: - Categories - - /// A browsable grouping of presets. The `rawValue` is the user-facing `title`; `sfSymbol` is the - /// section glyph shown in the source browser. - enum Category: String, CaseIterable, Identifiable { - case writing = "Writing" - case tone = "Tone" - case understand = "Understand" - case translate = "Translate" - case developer = "Developer" - case reply = "Reply" - case capture = "Capture" - case vision = "Vision" - case format = "Format" - - var id: String { rawValue } - - /// The user-facing section title (identical to the `rawValue`). - var title: String { rawValue } - - /// A representative SF Symbol for the category's section header. - var sfSymbol: String { - switch self { - case .writing: return "pencil" - case .tone: return "theatermasks" - case .understand: return "brain" - case .translate: return "character.bubble" - case .developer: return "chevron.left.forwardslash.chevron.right" - case .reply: return "arrowshape.turn.up.left" - case .capture: return "tray.and.arrow.down" - case .vision: return "eye" - case .format: return "tablecells" - } - } - - /// The shared tint for every command in this category, so a band of presets reads as a coherent - /// palette (one hue per category, mirroring `AIBand.seeded()`'s per-command tints). - var tint: ItemColor { - switch self { - case .writing: return ItemColor(red: 0.25, green: 0.72, blue: 0.40) // green - case .tone: return ItemColor(red: 0.95, green: 0.55, blue: 0.30) // orange - case .understand: return ItemColor(red: 0.95, green: 0.70, blue: 0.20) // amber - case .translate: return ItemColor(red: 0.66, green: 0.36, blue: 0.86) // purple - case .developer: return ItemColor(red: 0.30, green: 0.62, blue: 0.78) // teal - case .reply: return ItemColor(red: 0.20, green: 0.48, blue: 0.93) // blue - case .capture: return ItemColor(red: 0.90, green: 0.30, blue: 0.30) // red - case .vision: return ItemColor(red: 0.40, green: 0.50, blue: 0.92) // indigo - case .format: return ItemColor(red: 0.50, green: 0.55, blue: 0.60) // slate - } - } - } - - // MARK: - Entries - - /// One catalog row: a `Category` paired with its preset command. - struct Entry { - let category: Category - let command: AICommand - } - - /// All shipped presets, in category order. The catalog is the single source of truth; the Hub - /// source browser groups these by `category`, and `seeded()` curates a subset for a fresh install. - static let entries: [Entry] = writing + tone + understand + translate + developer + reply + capture + vision + format - - // MARK: - Queries - - /// The preset commands in a category, in catalog order (used by the source browser's sections). - static func commands(in category: Category) -> [AICommand] { - entries.filter { $0.category == category }.map(\.command) - } - - /// A value copy of a command with a FRESHLY minted id, so the same preset can be added to a band - /// twice without an id collision and later edits never mutate the catalog's stored command. - static func copy(of command: AICommand) -> AICommand { - var c = command - c.id = UUID() - return c - } - - /// A curated, one-band subset shipped on a fresh install: the handful of verbs that cover the most - /// common cases, in launcher order. (The full catalog is browsable from the Hub.) - static func seeded() -> [AICommand] { - let names = ["Fix Grammar", "Make Concise", "Improve Writing", "Translate", - "Explain", "Summarize", "Draft a Reply", "Add to Calendar"] - // Map by name, then emit in the curated order so the seeded band reads deliberately. - let byName = Dictionary(entries.map { ($0.command.name, $0.command) }, uniquingKeysWith: { first, _ in first }) - return names.compactMap { byName[$0] }.map(copy(of:)) - } -} - -// MARK: - Preset definitions -// -// One `Entry` array per category. Each command is built the same way as `AIBand.seeded()`: an -// `.sfSymbol` icon, the category's shared tint, an input source, a prompt template (templates that -// must yield only the result say so), and an output target. - -private extension AICommandCatalog { - /// Sugar for a preset: applies the category's tint so call sites stay terse and consistent. - static func preset(_ category: Category, _ name: String, icon: String, input: InputSource, - output: OutputTarget, template: String, - runtimeParameter: RuntimeParameter? = nil) -> Entry { - Entry(category: category, - command: AICommand(name: name, icon: .sfSymbol(icon), tint: category.tint, - input: input, promptTemplate: template, output: output, - runtimeParameter: runtimeParameter)) - } - - // MARK: Writing - - static let writing: [Entry] = [ - preset(.writing, "Fix Grammar", icon: "text.badge.checkmark", input: .selection, output: .replaceSelection, - template: "Fix the spelling and grammar of the following text. Return only the corrected text, with no commentary:\n\n{input}"), - preset(.writing, "Improve Writing", icon: "wand.and.stars", input: .selection, output: .replaceSelection, - template: "Improve the clarity and flow of the following while preserving its meaning and voice. Return only the rewritten text:\n\n{input}"), - preset(.writing, "Make Concise", icon: "scissors", input: .selection, output: .replaceSelection, - template: "Rewrite the following to be as concise as possible while preserving its meaning. Return only the rewritten text:\n\n{input}"), - preset(.writing, "Expand", icon: "arrow.up.left.and.arrow.down.right", input: .selection, output: .replaceSelection, - template: "Expand the following terse note into clear, complete prose. Return only the rewritten text:\n\n{input}"), - preset(.writing, "Simplify", icon: "text.append", input: .selection, output: .replaceSelection, - template: "Rewrite the following in plain language a non-expert can follow. Return only the rewritten text:\n\n{input}"), - preset(.writing, "Proofread", icon: "checklist", input: .selection, output: .previewOnly, - template: "List the spelling, grammar, and clarity issues in the following as a short bulleted list:\n\n{input}"), - preset(.writing, "Bulletize", icon: "list.bullet", input: .selection, output: .replaceSelection, - template: "Rewrite the following as a concise bulleted list. Return only the list:\n\n{input}"), - preset(.writing, "Active Voice", icon: "bolt.fill", input: .selection, output: .replaceSelection, - template: "Rewrite the following in the active voice. Return only the rewritten text:\n\n{input}"), - ] - - // MARK: Tone - - static let tone: [Entry] = [ - preset(.tone, "Make Professional", icon: "briefcase.fill", input: .selection, output: .replaceSelection, - template: "Rewrite the following in a professional, workplace-appropriate tone. Return only the rewritten text:\n\n{input}"), - preset(.tone, "Make Friendly", icon: "face.smiling", input: .selection, output: .replaceSelection, - template: "Rewrite the following in a warmer, friendlier tone. Return only the rewritten text:\n\n{input}"), - preset(.tone, "Make Confident", icon: "hand.thumbsup.fill", input: .selection, output: .replaceSelection, - template: "Rewrite the following to remove hedging and sound confident. Return only the rewritten text:\n\n{input}"), - preset(.tone, "De-escalate", icon: "wind", input: .selection, output: .replaceSelection, - template: "Rewrite the following to be calm, neutral, and professional, removing any anger or snark while keeping the substance. Return only the rewritten text:\n\n{input}"), - preset(.tone, "Make Funnier", icon: "face.dashed", input: .selection, output: .replaceSelection, - template: "Rewrite the following to be a bit funnier without changing the meaning. Return only the rewritten text:\n\n{input}"), - ] - - // MARK: Understand - - static let understand: [Entry] = [ - preset(.understand, "Explain", icon: "lightbulb", input: .selection, output: .previewOnly, - template: "Explain the following clearly and concisely for a curious non-expert:\n\n{input}"), - preset(.understand, "Summarize", icon: "text.line.first.and.arrowtriangle.forward", input: .selection, output: .previewOnly, - template: "Summarize the following in a few short bullet points:\n\n{input}"), - preset(.understand, "TL;DR", icon: "text.alignleft", input: .selection, output: .previewOnly, - template: "Give a one-sentence TL;DR of the following:\n\n{input}"), - preset(.understand, "Key Points & Action Items", icon: "checklist.checked", input: .selection, output: .previewOnly, - template: "Extract the key points and any action items from the following as two short bulleted lists:\n\n{input}"), - preset(.understand, "Define", icon: "character.book.closed", input: .selection, output: .previewOnly, - template: "Define the selected word or phrase in this context, briefly:\n\n{input}"), - preset(.understand, "Pros & Cons", icon: "arrow.up.arrow.down", input: .selection, output: .previewOnly, - template: "List the pros and cons of the following as two short bulleted lists:\n\n{input}"), - preset(.understand, "Counterargument", icon: "arrow.uturn.backward", input: .selection, output: .previewOnly, - template: "Give the strongest counterargument to the following:\n\n{input}"), - ] - - // MARK: Translate - - static let translate: [Entry] = [ - preset(.translate, "Translate", icon: "character.bubble", input: .selection, output: .previewOnly, - template: "Translate the following to {lang}. Return only the translation:\n\n{input}", - runtimeParameter: .language(default: "English")), - preset(.translate, "Translate in Place", icon: "character.bubble.fill", input: .selection, output: .replaceSelection, - template: "Translate the following to {lang}. Return only the translation:\n\n{input}", - runtimeParameter: .language(default: "English")), - preset(.translate, "Detect & Translate to English", icon: "globe", input: .selection, output: .previewOnly, - template: "Detect the language of the following and translate it to English. Return only the translation:\n\n{input}"), - preset(.translate, "Explain Idiom", icon: "quote.bubble", input: .selection, output: .previewOnly, - template: "Explain what this idiom or expression means (not a literal translation):\n\n{input}"), - ] - - // MARK: Developer - - static let developer: [Entry] = [ - preset(.developer, "Explain Code", icon: "curlybraces", input: .selection, output: .previewOnly, - template: "Explain what this code does, step by step, concisely:\n\n{input}"), - preset(.developer, "Explain Error", icon: "exclamationmark.triangle", input: .selection, output: .previewOnly, - template: "Explain this error or stack trace and the most likely fix:\n\n{input}"), - preset(.developer, "Commit Message", icon: "checkmark.seal", input: .clipboard, output: .pasteAtCursor, - template: "Write a concise Conventional Commits message for this diff. Return only the message:\n\n{input}"), - preset(.developer, "Add Docstring", icon: "text.quote", input: .selection, output: .previewOnly, - template: "Add a clear docstring and comments to this code. Return only the updated code:\n\n{input}"), - preset(.developer, "Regex from Description", icon: "asterisk", input: .selection, output: .previewOnly, - template: "Write a regular expression that does the following. Return only the regex:\n\n{input}"), - preset(.developer, "Explain Regex", icon: "magnifyingglass", input: .selection, output: .previewOnly, - template: "Explain what this regular expression matches, concisely:\n\n{input}"), - preset(.developer, "Rewrite in Language", icon: "arrow.triangle.2.circlepath", input: .selection, output: .previewOnly, - template: "Rewrite this code in {lang}. Return only the code:\n\n{input}", - runtimeParameter: .codeLanguage(default: "Python")), - preset(.developer, "Shell Command", icon: "terminal", input: .selection, output: .previewOnly, - template: "Write a single shell command that does the following. Return only the command:\n\n{input}"), - preset(.developer, "Name This", icon: "tag", input: .selection, output: .previewOnly, - template: "Suggest 3 clear, idiomatic names for this. Return only the names, one per line:\n\n{input}"), - ] - - // MARK: Reply - - static let reply: [Entry] = [ - preset(.reply, "Draft a Reply", icon: "arrowshape.turn.up.left", input: .selection, output: .previewOnly, - template: "Draft a clear, friendly reply to the following message. Return only the reply:\n\n{input}"), - preset(.reply, "Polite Decline", icon: "hand.raised", input: .selection, output: .previewOnly, - template: "Draft a polite, brief decline in response to the following. Return only the reply:\n\n{input}"), - preset(.reply, "Quick Acknowledge", icon: "checkmark.circle", input: .selection, output: .previewOnly, - template: "Draft a short acknowledgement reply (e.g. \"Got it, will do\") to the following:\n\n{input}"), - preset(.reply, "Summarize Thread then Reply", icon: "bubble.left.and.bubble.right", input: .selection, output: .previewOnly, - template: "Summarize this thread in one line, then draft a suggested reply:\n\n{input}"), - ] - - // MARK: Capture (task outputs; `confirmBeforeRun` derives ON automatically — never passed here) - - static let capture: [Entry] = [ - preset(.capture, "Add to Calendar", icon: "calendar.badge.plus", input: .selection, output: .runTask(.addToCalendar), - template: "Extract a calendar event from the following text. Today is {date}. If the text does not describe an event, decline.\n\n{input}"), - preset(.capture, "Add to Reminders", icon: "checklist", input: .selection, output: .runTask(.addToReminder), - template: "Extract a to-do/reminder from the following text. Today is {date}. If it describes no task, decline.\n\n{input}"), - preset(.capture, "New Contact", icon: "person.crop.circle.badge.plus", input: .selection, output: .runTask(.newContact), - template: "Extract contact details (name, email, phone, organization) from the following. If there are none, decline.\n\n{input}"), - preset(.capture, "Save to Project", icon: "tray.and.arrow.down.fill", input: .selection, output: .runTask(.saveToProject(project: "Inbox")), - template: "Return the following content to save, lightly cleaned up:\n\n{input}"), - preset(.capture, "Open with Tool…", icon: "arrow.up.forward.app", input: .selection, output: .runTask(.openToolWithPayload(tool: "")), - template: "{input}"), - preset(.capture, "Send to Shortcut…", icon: "bolt.fill", input: .selection, output: .sendTo(.shortcut(name: "")), - template: "{input}"), - ] - - // MARK: Vision (output `.previewOnly` — these need a vision model). Image comes from a captured - // screen region (`.screenRegion`) or the live clipboard image (`.clipboardImage`, on-demand). - - static let vision: [Entry] = [ - preset(.vision, "What Is This?", icon: "questionmark.circle", input: .screenRegion, output: .previewOnly, - template: "What is shown here? Answer concisely."), - preset(.vision, "Describe Clipboard Image", icon: "photo", input: .clipboardImage, output: .previewOnly, - template: "What is shown in this image? Answer concisely."), - preset(.vision, "Clipboard Image → Text (OCR)", icon: "doc.text.viewfinder", input: .clipboardImage, output: .previewOnly, - template: "Transcribe all the text shown in this image exactly. Return only the text."), - preset(.vision, "Extract Text (OCR)", icon: "text.viewfinder", input: .screenRegion, output: .previewOnly, - template: "Transcribe all the text shown here exactly. Return only the text."), - preset(.vision, "Explain This Chart", icon: "chart.bar", input: .screenRegion, output: .previewOnly, - template: "Explain what this chart or diagram shows, concisely."), - preset(.vision, "Solve This", icon: "function", input: .screenRegion, output: .previewOnly, - template: "Solve the problem shown here and show the key steps."), - preset(.vision, "Transcribe Handwriting", icon: "hand.draw", input: .screenRegion, output: .previewOnly, - template: "Transcribe the handwriting shown here. Return only the text."), - preset(.vision, "Extract Table to Markdown", icon: "tablecells", input: .screenRegion, output: .previewOnly, - template: "Extract the table shown here as a Markdown table. Return only the table."), - preset(.vision, "Translate Image Text", icon: "character.bubble", input: .screenRegion, output: .previewOnly, - template: "Translate the text shown here to {lang}. Return only the translation.", - runtimeParameter: .language(default: "English")), - ] - - // MARK: Format - - static let format: [Entry] = [ - preset(.format, "Format as Markdown Table", icon: "tablecells", input: .selection, output: .replaceSelection, - template: "Convert the following into a Markdown table. Return only the table:\n\n{input}"), - preset(.format, "Extract Emails", icon: "envelope", input: .selection, output: .previewOnly, - template: "Extract all email addresses from the following, one per line. Return only the list:\n\n{input}"), - preset(.format, "Extract URLs", icon: "link", input: .selection, output: .previewOnly, - template: "Extract all URLs from the following, one per line. Return only the list:\n\n{input}"), - preset(.format, "Strip Formatting", icon: "textformat", input: .selection, output: .replaceSelection, - template: "Return the following as clean plain text with consistent spacing. Return only the text:\n\n{input}"), - preset(.format, "JSON to YAML", icon: "doc.plaintext", input: .selection, output: .previewOnly, - template: "Convert this JSON to YAML. Return only the YAML:\n\n{input}"), - preset(.format, "Clean Up Whitespace", icon: "space", input: .selection, output: .replaceSelection, - template: "Normalize the whitespace in the following (collapse runs, trim lines). Return only the text:\n\n{input}"), - ] -} diff --git a/Sources/ThreeFingerSwitcher/AI/AICommandExecutor.swift b/Sources/ThreeFingerSwitcher/AI/AICommandExecutor.swift deleted file mode 100644 index e0ba78a..0000000 --- a/Sources/ThreeFingerSwitcher/AI/AICommandExecutor.swift +++ /dev/null @@ -1,526 +0,0 @@ -import Foundation -import Combine -import os - -/// Shares the `Selection` category with `SelectionService` so one Console.app filter -/// (`subsystem:ThreeFingerSwitcher category:Selection`) shows the whole chain: which channel the executor -/// resolved, and (from `SelectionService`) why the selection read did or didn't produce text. -private let aiInputLog = Logger(subsystem: "ThreeFingerSwitcher", category: "Selection") - -/// Orchestrates one AI command fire end-to-end (spec: "Command input acquisition" + "In-place output -/// routing"; tasks phase 9), behind seams so the slices stay decoupled: it talks to the model only -/// through `ModelManager` → `LLMRuntime`, to the front app only through `SelectionProviding`, and to -/// side effects only through `TaskDispatching`. It never sees a concrete model, selection service, or -/// task dispatcher. -/// -/// The fire is two-stage (design D4): `fire(_:)` acquires input, resolves the template, selects the -/// model, and STREAMS the result into observable `state` (so slice 5's canvas can render live and a -/// horizontal discard swipe can cancel); `commit()` then routes the ready result per the command's output target. -/// -/// `@MainActor` (and `ObservableObject`) because it holds observable UI state, matching the project's -/// convention (`AppSettings`, `ClipboardStore`, `ModelManager`). -@MainActor -final class AICommandExecutor: ObservableObject { - - /// The executor's observable state — the contract slice 5's canvas binds to. - enum State: Equatable { - /// Nothing in flight. - case idle - /// Resolving + loading the model (a visible state, never a silent block — design D4). - case loadingModel - /// An input-requiring command got no input; the model was NOT invoked (spec: "No input - /// available is surfaced"). - case noInput - /// Generation in flight; `partial` is the text accumulated so far. - case streaming(partial: String) - /// A finished, uncommitted result awaiting the commit (down-swipe) for in-place outputs. - case ready(result: String) - /// A side-effecting task's parsed action awaiting the armed-confirmation commit (design D6): - /// the review carries the preview `fields` slice 5 renders. Reached ONLY when the command's - /// `confirmBeforeRun` is on; when off, the side effect commits directly to `.committed`. - case reviewingAction(TaskReview) - /// A structured/task path the model declined (design D2) — carries the reason. - case declined(reason: String) - /// A typed failure with a human-readable message. - case failed(message: String) - /// AI can't produce a result yet: the opt-in is off, or the model isn't downloaded/ready. The - /// canvas shows an enable/download affordance + a model picker; nothing is generated. A - /// horizontal discard dismisses, and any download started continues in the background - /// (configuration-hub: fire-time availability resolves in the canvas, not by hiding items). - case unavailable - /// Committed and done (in-place written or task dispatched). - case committed - - /// Whether a turn is actively being produced (`fix-evict-thrash-and-hot-path`): the - /// quiescence input mirroring `NotchSessionEngine.isTurnInFlight` — automatic model - /// eviction must never fire while the CANVAS executor is loading or streaming. - /// `.reviewingAction` counts as foreground-active (mid-conversation), not turn-in-flight. - var isTurnInFlight: Bool { - switch self { - case .loadingModel, .streaming: return true - default: return false - } - } - - /// Whether a DOWN-swipe commit should COMMIT. Only a ready in-place result or a task action - /// awaiting armed-confirmation is committable; a DOWN swipe in any other state (still - /// loading/streaming, no input, declined, failed) is IGNORED — the user waits, and only a - /// horizontal discard swipe cancels the in-flight generation (so nothing is ever leaked). - var isCommittable: Bool { - switch self { - case .ready, .reviewingAction: return true - default: return false - } - } - - /// Value equality. `reviewingAction` compares by its review's discriminant + preview fields - /// (the payload is opaque), which is all the UI / tests observe. - static func == (lhs: State, rhs: State) -> Bool { - switch (lhs, rhs) { - case (.idle, .idle), (.loadingModel, .loadingModel), (.noInput, .noInput), - (.unavailable, .unavailable), (.committed, .committed): - return true - case let (.streaming(a), .streaming(b)): return a == b - case let (.ready(a), .ready(b)): return a == b - case let (.declined(a), .declined(b)): return a == b - case let (.failed(a), .failed(b)): return a == b - case let (.reviewingAction(a), .reviewingAction(b)): - return TaskReview.previewEqual(a, b) - default: - return false - } - } - } - - @Published private(set) var state: State = .idle - /// The model's streamed REASONING for the in-flight command ("show the model's thinking"). The - /// text-path streaming loop appends every `.thinking`-channel token here (live, so the canvas's - /// collapsible Thinking section updates as it streams) — while ONLY `.response` tokens accumulate - /// into `state`/commit. Reset at the start of every `fire(...)` and on `cancel()` so a re-run or - /// discard never shows stale thinking. Empty when the runtime emits no thinking (today's default). - @Published private(set) var thinking: String = "" - /// The active runtime language of the command in flight (spec: runtime parameter). `nil` when the - /// active command declares no language parameter — drives whether the canvas shows the dropdown - /// and what it shows selected. Resolved on every `fire` from persistence → declared default. - @Published private(set) var activeLanguage: String? - /// Whether the canvas's scrollable content is scrolled to the TOP — written by the canvas view from - /// its scroll position, read by the resolve gate so a fresh **down** swipe commits ONLY when there's - /// nothing more to scroll up to (otherwise the down-swipe is a scroll, not an apply). Not `@Published` - /// (it's a gate input the view writes, not state the view renders) so updating it never re-renders the - /// canvas. Reset to `true` on every `fire` (fresh content starts at the top). - var canvasAtTop = true - - private let modelManager: ModelManager - private let selection: SelectionProviding - private let dispatcher: TaskDispatching - /// The fire-time context provider (front app name / URL). Injected so the executor doesn't reach - /// into AppKit itself; the input text is filled in by acquisition. - private let contextProvider: @MainActor () -> FireContext - /// Per-command remembered runtime language (the next-run default). Injected as closures so the - /// executor stays AppKit/AppSettings-free; the app wires these to `AppSettings`, tests pass theirs. - private let loadLanguage: @MainActor (UUID) -> String? - private let saveLanguage: @MainActor (UUID, String) -> Void - /// Whether the model should reason (think) before answering — thinking is filtered from the - /// result. Injected as a closure so the executor stays AppSettings-free; the app wires it to the - /// `aiReasoningEnabled` pref, tests pass their own. - private let reasoning: @MainActor () -> Bool - - /// The command currently being executed (set by `fire`, read by `commit`). - private(set) var activeCommand: AICommand? - /// The streaming task, retained so `cancel()` / a new fire can stop it (horizontal discard swipe). - private var generationTask: Task? - /// For a `screenRegion` command, the capture outcome supplied at fire time by the region picker - /// (the picker captures the designated rectangle BEFORE the canvas opens — the executor never - /// captures the screen itself). Retained across a same-command language re-run (`setLanguage`) so the - /// re-translate reuses the captured image. `nil` for non-vision commands. - private var presuppliedCapture: ScreenCaptureOutcome? - /// Whether the input RESOLVED at the last fire came from a selection (design D2). Remembered from - /// `fire` → `commit` so an in-place commit REPLACES the selection when it was the source, and PASTES - /// at the cursor otherwise. Reset on every fire before resolution. - private var resolvedWasSelection = false - /// Called on the main actor AFTER the fire has ACQUIRED its input (or resolved to `.unavailable`), so - /// the canvas panel may take key focus for its controls WITHOUT stealing the selection read: the - /// controller keeps the panel pass-through until this fires. The selection MUST be read while the - /// captured front app still holds key focus (a `.nonactivatingPanel` that becomes key first makes the - /// front app's AX focused element / ⌘C fallback come back empty → a wrong fall-through to the - /// clipboard). Injected by the app; nil in tests. - var onReadyForInteraction: (@MainActor () -> Void)? - /// The last fire's acquired input, retained so a same-command language re-run (`setLanguage`) reuses - /// the SAME source instead of re-reading the selection (which would fail once the canvas panel has - /// taken key focus) — mirroring how `presuppliedCapture` is retained for a vision re-translate. - private var retainedAcquisition: Acquisition? - - /// The outcome of acquiring a fire's input (before the model runs). - private enum Acquisition { - /// Input ready (text and/or image; both nil = a standalone prompt). `wasSelection` drives commit. - case ok(text: String?, image: Data?, wasSelection: Bool) - /// An input-requiring command had no live input channel. - case noInput - /// Acquisition failed with a human-readable message (e.g. a missing Screen-Recording grant). - case failed(String) - } - - init(modelManager: ModelManager, - selection: SelectionProviding, - dispatcher: TaskDispatching, - contextProvider: @escaping @MainActor () -> FireContext = { FireContext() }, - loadLanguage: @escaping @MainActor (UUID) -> String? = { _ in nil }, - saveLanguage: @escaping @MainActor (UUID, String) -> Void = { _, _ in }, - reasoning: @escaping @MainActor () -> Bool = { false }) { - self.modelManager = modelManager - self.selection = selection - self.dispatcher = dispatcher - self.contextProvider = contextProvider - self.loadLanguage = loadLanguage - self.saveLanguage = saveLanguage - self.reasoning = reasoning - } - - /// The active language for `command`: the remembered per-command choice, falling back to the - /// command's declared `.language` default. `nil` when the command declares no language parameter. - func resolvedLanguage(for command: AICommand) -> String? { - guard case let .languageChoice(def, _)? = command.runtimeParameter else { return nil } - return loadLanguage(command.id) ?? def - } - - // MARK: - Fire (acquire → resolve → stream) - - /// Start executing `command`: acquire its input, resolve the template, select + load the model, - /// and stream the result into `state`. Returns immediately; progress is observed via `state`. - /// Cancels any in-flight generation first (a new fire supersedes the old). - /// - /// `screenCapture` is the region picker's capture outcome for a `screenRegion` command (the picker - /// captures the designated rectangle before this fire); `nil` for non-vision commands and for the - /// no-image overload. The outcome (not raw bytes) is passed so the executor maps a permission gap → - /// `.failed` and an unavailable capture → `.noInput` itself, keeping the error taxonomy in one place. - func fire(_ command: AICommand, screenCapture: ScreenCaptureOutcome? = nil, reuseInput: Bool = false) { - cancel() - thinking = "" // clear any previous run's reasoning before the new fire streams its own - canvasAtTop = true // fresh content starts at the top (so a first down-swipe can apply) - presuppliedCapture = screenCapture - if !reuseInput { retainedAcquisition = nil } // a fresh fire re-reads; a language re-run reuses - activeCommand = command - // Resolve the active runtime language up front (persisted choice → declared default → nil), so - // the canvas dropdown reflects it even while loading / in the `.unavailable` state. - activeLanguage = resolvedLanguage(for: command) - - // Fire-time availability gate (configuration-hub): if AI can't produce a result yet — the - // opt-in is off, or the model isn't downloaded/ready — open the canvas in the `.unavailable` - // state (enable/download + model picker) instead of generating. The model is never invoked - // here; the user enables/downloads from the canvas (the download continues in the background), - // and a horizontal discard dismisses. - guard modelManager.optedIn, Self.modelIsOnDisk(modelManager.state) else { - state = .unavailable - onReadyForInteraction?() // the canvas needs key focus for Enable / Download / model-picker - return - } - - state = .loadingModel - generationTask = Task { [weak self] in - guard let self else { return } - await self.run(command, reuseInput: reuseInput) - } - } - - /// Whether the model's weights are present (downloaded/loaded) so a fire can produce a result - /// without a download. A download/verify still in flight — or not-downloaded / failed — is treated - /// as unavailable (the canvas offers download and reflects progress). - static func modelIsOnDisk(_ state: ModelLifecycleState) -> Bool { - switch state { - case .ready, .loading, .loaded: return true - case .notDownloaded, .downloading, .verifying, .failed: return false - } - } - - private func run(_ command: AICommand, reuseInput: Bool) async { - // 1) Acquire the input against the live environment (change `ai-action-context-resolution`) — or - // REUSE the retained acquisition on a same-command language re-run, so a re-translate never - // re-reads the selection after the canvas panel has taken key focus. - let acquisition: Acquisition - if reuseInput, let retained = retainedAcquisition { - acquisition = retained - } else { - acquisition = await acquire(command) - retainedAcquisition = acquisition - } - if Task.isCancelled { return } - - // The input has now been READ while the front app still held key focus. Signal the controller it - // may make the canvas key-interactive — taking key focus BEFORE this point steals the read (the - // front app's AX focused element / ⌘C fallback come back empty), which is the "recognizes the - // selection then falls back to the clipboard" bug this ordering fixes. - onReadyForInteraction?() - - let inputText: String? - let image: Data? - resolvedWasSelection = false - switch acquisition { - case let .failed(message): - state = .failed(message: message) // e.g. a missing Screen-Recording grant - return - case .noInput: - state = .noInput // input-requiring command with no live channel - return - case let .ok(text, img, wasSelection): - inputText = text - image = img - resolvedWasSelection = wasSelection // drives the in-place commit (replace vs paste) - } - - // 2) Build the fire context and resolve the prompt template (`{lang}` ⇐ the active language). - var context = contextProvider() - context.inputText = inputText - let prompt = PromptTemplate.resolve(command.promptTemplate, with: context, activeLanguage: activeLanguage) - - // 3) Select + load the model for the RESOLVED input's capability — a vision model only when the - // resolved channel is actually an image, else a text model (design D3: capability follows the - // live input, not the authored superset). - let runtime: LLMRuntime - do { - let caps: Set = (image != nil) ? [.vision] : [.text] - runtime = try await modelManager.runtime(requiring: caps) - } catch { - state = .failed(message: Self.message(for: error)) - return - } - - if Task.isCancelled { return } - - // Resolve reasoning ONCE for this command: an explicit per-command override wins, else the - // global default (the injected closure). The executor owns this resolution and threads the - // result into both the text request and the task path. - let useReasoning = command.resolvedReasoning(globalDefault: reasoning()) - - // 4) Branch on the output's nature. A SIDE-EFFECTING output (`.runTask` / `.sendTo`) does NOT - // stream text — it resolves a schema-targeted, validated, parsed ACTION via the dispatcher and - // lands in `.reviewingAction` (armed-confirmation) / `.declined` / `.failed`, or — when the - // command's `confirmBeforeRun` is OFF — commits the side effect directly (honoring the stored - // value; design D6). An IN-PLACE output streams as before. - if let kind = Self.taskKind(for: command) { - await runTask(kind, command: command, resolvedPrompt: prompt, context: context, - reasoning: useReasoning) - return - } - - // In-place: stream generation into observable state (so the canvas renders live). Tokens are - // split by channel: `.thinking` chunks accumulate into the observable `thinking` (the canvas's - // collapsible reasoning section) and NEVER reach the committed result; only `.response` chunks - // accumulate into `accumulated` → `state` → commit ("show the thinking, commit the response"). - let request = LLMRequest(prompt: prompt, image: image, reasoning: useReasoning) - state = .streaming(partial: "") - var accumulated = "" - do { - for try await token in runtime.generate(request) { - if Task.isCancelled { return } - switch token.channel { - case .thinking: - thinking += token.text // live into the canvas's Thinking section; never committed - case .response: - accumulated += token.text - state = .streaming(partial: accumulated) - } - } - if Task.isCancelled { return } - state = .ready(result: accumulated) // RESPONSE ONLY — thinking is never part of the result - } catch let error as RuntimeError { - if case .cancelled = error { return } // a discard is not a failure - state = .failed(message: Self.message(for: error)) - } catch is CancellationError { - return - } catch { - state = .failed(message: Self.message(for: error)) - } - } - - /// Prepare (and, when review is off, fire) a side-effecting task. Maps the dispatcher's review to - /// state: `.declined` → `.declined`; `.unavailable` → `.failed`; `.action` → `.reviewingAction` - /// when `confirmBeforeRun` is on, else `execute` it directly → `.committed`. - private func runTask(_ kind: TaskKind, command: AICommand, resolvedPrompt: String, - context: FireContext, reasoning: Bool) async { - let source = TaskSource(appName: context.capturedAppName, url: context.url, timestamp: context.date) - let review = await dispatcher.prepare(kind, resolvedPrompt: resolvedPrompt, source: source, - reasoning: reasoning) - if Task.isCancelled { return } - - switch review { - case let .declined(reason): - state = .declined(reason: reason) - case let .unavailable(reason): - state = .failed(message: reason) - case .action: - if command.confirmBeforeRun { - // Armed-confirmation: the side effect fires on the NEXT commit (slice 5 renders fields). - state = .reviewingAction(review) - } else { - // Review skipped (the user disabled it): commit the side effect now (no extra gate). - do { - try await dispatcher.execute(review) - state = .committed - } catch { - state = .failed(message: Self.message(for: error)) - } - } - } - } - - /// The `TaskKind` a command's side-effecting output routes to (a `.sendTo` output maps to the - /// `.sendTo` task kind), or nil for an in-place command. Reads the command's single side-effecting - /// output capability (`sideEffect`). - private static func taskKind(for command: AICommand) -> TaskKind? { - switch command.sideEffect { - case let .runTask(kind): return kind - case let .sendTo(destination): return .sendTo(destination) - default: return nil - } - } - - // MARK: - Commit (route the ready result) - - /// Commit per the current state. An IN-PLACE `.ready` result routes through `SelectionProviding` - /// exactly as before. A `.reviewingAction` (armed-confirmation, reached only when the command's - /// `confirmBeforeRun` is on) fires the reviewed side effect through `TaskDispatching.execute`. A - /// side-effecting command with review OFF already committed in `run()`, so there's nothing here. - /// No-op for any other state. Throws task errors so the caller can surface them. - func commit() async throws { - // Armed-confirmation commit: fire the reviewed side effect (design D6). On a throw (e.g. the - // default-calendar path is denied), surface `.failed` with a human message AND rethrow so the - // caller still sees the error — consistent with the review-OFF branch in `runTask`. - if case let .reviewingAction(review) = state { - do { - try await dispatcher.execute(review) - state = .committed - } catch { - state = .failed(message: Self.message(for: error)) - throw error - } - return - } - - guard case let .ready(result) = state, let command = activeCommand else { return } - - // Resolve the in-place commit from the input channel that won at fire + the enabled outputs - // (design D2): a selection source replaces; a clipboard/image source pastes; preview-only writes - // nothing. Side-effecting outputs never land in `.ready` (they route through `runTask`). - switch AICommand.inPlaceCommitPlan(resolvedWasSelection: resolvedWasSelection, outputs: command.outputs) { - case .replaceSelection: - // Honesty (spec D5): a write that didn't actually land is a failure, not a "Done". - if await selection.replaceSelection(result) { - state = .committed - } else { - state = .failed(message: "Couldn't apply the result to the active app.") - } - case .pasteAtCursor: - if await selection.pasteAtCursor(result) { - state = .committed - } else { - state = .failed(message: "Couldn't paste the result into the active app.") - } - case .preview: - // Deliberately writes nothing into the app (spec: "Preview-only never writes"). - state = .committed - } - } - - /// Discard the current fire: cancel any in-flight generation and reset to idle. Writes nothing. - /// Also clears any streamed reasoning so a discard never leaves stale thinking behind. - func cancel() { - generationTask?.cancel() - generationTask = nil - thinking = "" - state = .idle - } - - // MARK: - Runtime parameter (in-canvas language re-run) - - /// Re-run the active command against a newly chosen runtime `language` (launcher-overlay: the - /// in-canvas dropdown re-translates in place). Persists the choice per command — so the next run - /// defaults to it — then re-fires, which cancels the in-flight generation (cancellation is not a - /// failure) and streams the new language into the same canvas. A no-op when the active command - /// declares no language parameter, or when the language is unchanged (avoids a redundant re-run). - func setLanguage(_ language: String) { - guard let command = activeCommand, - case .languageChoice? = command.runtimeParameter, - language != activeLanguage else { return } - saveLanguage(command.id, language) - // Re-read the just-persisted language via `resolvedLanguage`; re-pass the picker's capture so a - // vision re-translate (e.g. "Translate Image Text") reuses the captured image. `reuseInput: true` - // reuses the ALREADY-ACQUIRED source text/image rather than re-reading the selection — which would - // now fail, since the canvas panel took key focus after the first fire. - fire(command, screenCapture: presuppliedCapture, reuseInput: true) - } - // MARK: - Input acquisition (ambient cascade) - - /// Acquire a fire's input from the live environment. A `screenRegion` command uses the pre-supplied - /// picker capture (region-first; a permission gap → `.failed`, an unavailable/cancelled capture → - /// `.noInput`); everything else walks the ambient cascade `selection ▸ clipboard-text ▸ clipboard-image` - /// and remembers whether the selection won (drives the in-place commit). - private func acquire(_ command: AICommand) async -> Acquisition { - if command.inputs.contains(.screenRegion) { - switch presuppliedCapture { - case let .captured(data): - return .ok(text: nil, image: data, wasSelection: false) - case .permissionDenied: - return .failed("Screen Recording permission is required for this command. " - + "Enable it in System Settings ▸ Privacy & Security ▸ Screen Recording.") - case .unavailable, .none: - return .noInput - } - } - switch await resolveAmbientInput(for: command) { - case let .selection(text): return .ok(text: text, image: nil, wasSelection: true) - case let .clipboardText(text): return .ok(text: text, image: nil, wasSelection: false) - case let .clipboardImage(data): return .ok(text: nil, image: data, wasSelection: false) - case .none: - // A command that requires input but has no live channel surfaces "no input" (no model run); a - // standalone command (empty `inputs`) proceeds with no input (its template stands alone). - return command.needsInput ? .noInput : .ok(text: nil, image: nil, wasSelection: false) - } - } - - /// The input channel resolved from the live environment (change `ai-action-context-resolution`). - /// `.none` = no enabled channel was live. - private enum ResolvedInput { - case selection(String) - case clipboardText(String) - case clipboardImage(Data) - case none - } - - /// Resolve the active ambient input by walking the command's ENABLED channels in priority order and - /// returning the first that is live (design D1): a non-empty selection ▸ non-empty clipboard text ▸ a - /// decodable clipboard image. Text beats image so a plain transform stays a cheap text run; the image - /// channel engages only when there is no usable text anywhere. `screenRegion` is handled separately - /// (region-first), never here. Whitespace-only reads count as empty (the read primitives normalize). - private func resolveAmbientInput(for command: AICommand) async -> ResolvedInput { - aiInputLog.notice("resolveAmbientInput: enabled = [\(command.inputs.map(\.rawValue).sorted().joined(separator: ","), privacy: .public)]") - if command.inputs.contains(.selection), - let sel = await selection.readSelectedText(), - !sel.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty { - aiInputLog.notice("resolveAmbientInput → SELECTION (\(sel.count) chars)") - return .selection(sel) - } - if command.inputs.contains(.clipboard), - let clip = selection.readClipboardText(), - !clip.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty { - aiInputLog.notice("resolveAmbientInput → CLIPBOARD TEXT (\(clip.count) chars) — selection empty/disabled, fell back") - return .clipboardText(clip) - } - if command.inputs.contains(.clipboardImage), - let img = selection.readClipboardImage() { - aiInputLog.notice("resolveAmbientInput → CLIPBOARD IMAGE (\(img.count) bytes)") - return .clipboardImage(img) - } - aiInputLog.notice("resolveAmbientInput → NONE (no live channel)") - return .none - } - - // MARK: - Messaging - - /// Map any error to a short, user-facing message for the `.failed` state, via the single central - /// translator (`AIError`). This guarantees the canvas shows the SAME clean headline the Settings - /// row shows for the same error, and that a non-`LocalizedError` can never dump raw text into the - /// canvas (the old `?? "\(error)"` fallback is gone — `AIError` returns a safe generic instead). - private static func message(for error: Error) -> String { - AIError.message(for: error).headline - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/AIError.swift b/Sources/ThreeFingerSwitcher/AI/AIError.swift deleted file mode 100644 index 2bd78fe..0000000 --- a/Sources/ThreeFingerSwitcher/AI/AIError.swift +++ /dev/null @@ -1,195 +0,0 @@ -import Foundation - -/// The AI feature's SINGLE error→message translator (design D1). Every user-facing AI error surface — -/// the Settings model-status row, the overlay canvas, any alert — routes its message through here, so -/// the same underlying error yields the SAME concise headline everywhere (spec: "Single error taxonomy -/// and translator"). It also enforces the clean-message invariant: raw error text (`"\(error)"`, -/// `String(describing:)`, an OS error's `.localizedDescription`) is NEVER used as a headline — it rides -/// only on the opt-in `details` payload (and logs). -/// -/// HOME (task 1.6): this lives in `ThreeFingerSwitcherCore`, alongside the shared `RuntimeError` -/// taxonomy. Core is visible to BOTH the `GemmaRuntime` (MLX) target and the app target, so both reach -/// it without a layering violation, and Core stays MLX-free (it references only Foundation `NSError` -/// and the app's own `RuntimeError`/`TaskError`). Vendor errors — e.g. `Gemma4DownloadError`, which is -/// only visible inside `GemmaRuntime` — are mapped into `RuntimeError` at the runtime boundary (design -/// D6, `GemmaMLXRuntime.prepare`) BEFORE they reach this translator, so `AIError` never needs to import -/// the vendor package. - -/// A presentable error: a concise, user-facing `headline` plus optional, separately-carried `details` -/// (the raw technical text, for an opt-in "Show details / Copy" affordance and logs — never inline by -/// default). Carrying `details` apart from `headline` is what lets the UI bound the headline while -/// still exposing the full diagnostic on demand (design D4). -public struct AIPresentedError: Equatable, Sendable { - /// The short, human-readable sentence shown by default on every surface. - public let headline: String - /// Raw technical detail for an opt-in disclosure / copy / log. `nil` when there is nothing more - /// useful than the headline itself. - public let details: String? - - public init(headline: String, details: String? = nil) { - self.headline = headline - self.details = details - } -} - -public enum AIError { - - /// The generic, safe headline for an error the translator does not recognize. - public static let unknownHeadline = "Something went wrong." - - /// Translate ANY error into a clean `AIPresentedError`. Resolution order (spec / design D1): - /// 1. The app's own `LocalizedError` taxonomy (`RuntimeError`, `TaskError`) → its `errorDescription` - /// as the headline (these are authored to be clean and per-case). - /// 2. Cancellation (`CancellationError`) → a benign "Cancelled." headline (callers treat cancellation - /// as not-a-failure; the translator still returns a clean string for any surface that asks). - /// 3. The vendor/OS classifier for a bare `NSError` (connectivity / HTTP status) → a taxonomy case. - /// 4. The generic fallback headline. - /// - /// The raw `String(describing: error)` is ALWAYS stashed as `details` (so the full technical text is - /// available on demand and in logs), except where the error carries a cleaner detail of its own - /// (e.g. `RuntimeError.modelLoadFailed(detail:)`) — and it is NEVER used as the headline. - public static func message(for error: Error) -> AIPresentedError { - // 1) The app's own self-describing taxonomy. We match our OWN types explicitly (not any - // `LocalizedError`) so a vendor `LocalizedError` can't smuggle raw interpolation into a - // headline — vendor errors are mapped to `RuntimeError` at the boundary instead. - if let runtime = error as? RuntimeError { - return AIPresentedError(headline: runtime.errorDescription ?? unknownHeadline, - details: details(for: runtime)) - } - if let task = error as? TaskError { - return AIPresentedError(headline: task.errorDescription ?? unknownHeadline, - details: String(describing: task)) - } - if let skill = error as? SkillError { - return AIPresentedError(headline: skill.errorDescription ?? unknownHeadline, - details: String(describing: skill)) - } - if let memory = error as? MemoryError { - return AIPresentedError(headline: memory.errorDescription ?? unknownHeadline, - details: details(for: memory)) - } - if let park = error as? ParkError { - return AIPresentedError(headline: park.errorDescription ?? unknownHeadline, - details: park.rawDetail) - } - if let audit = error as? AuditError { - return AIPresentedError(headline: audit.errorDescription ?? unknownHeadline, - details: audit.rawDetail) - } - if let handoff = error as? HandoffError { - return AIPresentedError(headline: handoff.errorDescription ?? unknownHeadline, - details: handoff.copyableDetails) - } - if let fleet = error as? FleetError { - // The eviction list rides in `details` (copyable), never the headline (design D7, task 4.2). - return AIPresentedError(headline: fleet.errorDescription ?? unknownHeadline, - details: fleet.copyableDetails) - } - if let voice = error as? VoiceError { - // Voice failures (`add-voice-computer-use-agent`): mic/speech/OS-floor issues with clean - // per-case headlines; raw AVFoundation/Speech text rides only in the copyable details. - return AIPresentedError(headline: voice.errorDescription ?? unknownHeadline, - details: voice.copyableDetails) - } - if let ax = error as? AXActionError { - // Computer-use failures: AX read/act problems mapped at the AXUIElement boundary. - return AIPresentedError(headline: ax.errorDescription ?? unknownHeadline, - details: ax.copyableDetails) - } - if let media = error as? MediaError { - // Media-specific failures (no-backend, seed required/invalid, generation/write failed, cloud - // budget/unavailable). Raw OS/vendor text rode into `copyableDetails` at the boundary; the - // headline stays clean (`ai-media-runtime`, design D10). - return AIPresentedError(headline: media.errorDescription ?? unknownHeadline, - details: media.copyableDetails) - } - - // 2) Cancellation is not a real failure — give a benign, clean headline if asked. - if error is CancellationError { - return AIPresentedError(headline: RuntimeError.cancelled.errorDescription ?? "Cancelled.", - details: nil) - } - - // 3) Vendor/OS classifier: map a bare NSError (connectivity, HTTP status) into the taxonomy and - // reuse that case's clean description. The raw NSError dump becomes the copyable details. - let ns = error as NSError - let rawDetails = String(describing: error) - if let mapped = classify(ns) { - return AIPresentedError(headline: mapped.errorDescription ?? unknownHeadline, details: rawDetails) - } - - // 4) Unknown → safe generic headline; the raw text is opt-in details only, never the headline. - return AIPresentedError(headline: unknownHeadline, details: rawDetails) - } - - // MARK: - Vendor / OS classifier - - /// Classify a bare `NSError` into the shared taxonomy. Connectivity failures (URL-loading system) - /// map to `.offline`; other URL-loading failures and 5xx map to `.serverUnavailable`; the - /// auth/forbidden/not-found HTTP statuses map to `.authOrAccessDenied`. Returns `nil` when the - /// error doesn't look like a recognized network/HTTP failure (→ generic fallback upstream). - /// - /// NOTE: in production the Gemma download library's errors are already mapped to `RuntimeError` at - /// the runtime boundary, so this NSError path is mostly a safety net (and is what the unit tests - /// pin over synthetic `NSError`s). - static func classify(_ error: NSError) -> RuntimeError? { - if error.domain == NSURLErrorDomain { - switch error.code { - case NSURLErrorNotConnectedToInternet, // -1009 - NSURLErrorNetworkConnectionLost, // -1005 - NSURLErrorCannotConnectToHost, // -1004 - NSURLErrorCannotFindHost, // -1003 - NSURLErrorDNSLookupFailed, // -1006 - NSURLErrorTimedOut, // -1001 - NSURLErrorDataNotAllowed, // -1020 - NSURLErrorInternationalRoamingOff: // -1018 - return .offline - default: - return .serverUnavailable - } - } - // An HTTP status carried as a plain NSError code (e.g. a mapped vendor/network error). - return runtimeError(forHTTPStatus: error.code) - } - - /// Map an HTTP status code to the taxonomy: 401/403/404 → access denied; any other 4xx/5xx → - /// server-unavailable. Returns `nil` for non-HTTP codes. `public` and shared so a runtime boundary - /// (e.g. `GemmaMLXRuntime`, which has the status in hand) classifies identically to this translator. - public static func runtimeError(forHTTPStatus code: Int) -> RuntimeError? { - switch code { - case 401, 403, 404: return .authOrAccessDenied - case 400..<600: return .serverUnavailable - default: return nil - } - } - - // MARK: - Details derivation - - /// The copyable `details` for a `RuntimeError`: prefer a carried diagnostic (e.g. - /// `modelLoadFailed(detail:)`) over the bare enum reflection; `nil` when the headline already says - /// everything (no extra technical text to expose). - private static func details(for runtime: RuntimeError) -> String? { - switch runtime { - case let .modelLoadFailed(detail): - return detail - case let .decodeFailed(detail): - return detail - case .offline, .serverUnavailable, .authOrAccessDenied, .modelMissing, - .integrityFailed, .cancelled, .couldNotProduceValid, .unsupportedModality, .unavailable: - return nil - } - } - - /// The copyable `details` for a `MemoryError`: the carried raw OS/parse text (kept OUT of the - /// headline). `nil` when the headline already says everything (no extra technical text). - private static func details(for memory: MemoryError) -> String? { - switch memory { - case let .unreadableCore(detail), let .writeFailed(detail): - return detail - case let .malformedSubfile(_, detail): - return detail - case .subfileNotFound, .capExceeded: - return nil - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/AIRuntimeInjection.swift b/Sources/ThreeFingerSwitcher/AI/AIRuntimeInjection.swift deleted file mode 100644 index 37e0b6b..0000000 --- a/Sources/ThreeFingerSwitcher/AI/AIRuntimeInjection.swift +++ /dev/null @@ -1,36 +0,0 @@ -import Foundation - -/// The injection seam that lets the app executable wire a REAL `LLMRuntime`-backed `ModelManager` -/// into Core WITHOUT Core ever referencing a concrete model or framework (design D1/D7). -/// -/// Core (and `swift test`) stay MLX-free: the factory is nil by default, so `AppCoordinator` falls -/// back to `DevAIRuntime.makeModelManager` (the stub). The app target's `main.swift` sets the factory -/// to `GemmaRuntime.makeModelManager` BEFORE `runThreeFingerSwitcher()`, so the lazily-built -/// `modelManager` resolves to the in-process Gemma 4 (MLX) runtime in a real build. -public enum AIRuntimeInjection { - /// When set, `AppCoordinator` builds its `ModelManager` from this factory (the real Gemma runtime). - /// nil → the dev-stub path. `@MainActor` to match `ModelManager`'s isolation. - /// - /// The factory carries the live Full-Potential gate flags (`fullPotentialEnabled` / `cpuLaneEnabled` - /// / `batchedRuntimeEnabled`, §D1 — read, not owned) so `GemmaRuntime.makeModelManager` can install - /// the CPU ternary lane AND the multi-stream batched runtime behind their gates. OFF (the default for - /// each) → no CPU lane is constructed and the GPU runtime is the proven single-session - /// `GemmaMLXRuntime`, so the build behaves exactly as today's single-GPU-lane one; the dev-stub - /// fallback ignores them. The wiring resolves each flag through `FullPotentialGate.isUnlocked` (the - /// single resolver) at the call site, so a master-OFF locks both at once (the calm panic-off). - @MainActor public static var modelManagerFactory: - (@MainActor (_ optedIn: Bool, - _ cpuLaneUnlocked: Bool, - _ batchedRuntimeUnlocked: Bool) -> ModelManager)? = nil - - /// When set, `AppCoordinator` builds the image `MediaRuntime` the `MediaToolContributor`/`MediaGenSink` - /// consume (the `generate_image` backend). Parallel to `modelManagerFactory`: nil keeps Core MLX-free - /// (no image runtime ⇒ the contributor advertises `generate_image` only if some other runtime is - /// wired, i.e. NOT at all in a Core/test build), and the app target's `main.swift` sets it to - /// `GemmaRuntime.makeImageRuntime`. The app passes the user-selected `imageModelID` (the catalog - /// selection); the GemmaRuntime side resolves the `ImageModelCatalog` descriptor + weights + the - /// gallery output dir and constructs `MFluxImageRuntime`. The runtime's pipeline is unbuilt until - /// Wave 2, so it surfaces a CLEAN `MediaError` (never a blank PNG) — an honest "model isn't ready yet." - @MainActor public static var imageRuntimeFactory: - (@MainActor (_ imageModelID: String?) -> MediaRuntime?)? = nil -} diff --git a/Sources/ThreeFingerSwitcher/AI/Agent/AgentContextBudgetProvider.swift b/Sources/ThreeFingerSwitcher/AI/Agent/AgentContextBudgetProvider.swift deleted file mode 100644 index 95a17f1..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Agent/AgentContextBudgetProvider.swift +++ /dev/null @@ -1,45 +0,0 @@ -import Foundation - -/// Reads an optional per-skill context-override (design D5). The value rides on the skill file (owned by -/// `ai-skills-as-files`); this slice only READS it through this seam. The default returns nil (no skill -/// raises the context). -protocol SkillContextOverriding: Sendable { - /// The context-token override the active skill declares, if any. - func contextOverride(forSkill skillID: String?) -> Int? -} - -/// The default: no skill raises the context (used until `ai-skills-as-files` supplies a real source). -struct NoSkillContextOverride: SkillContextOverriding { - init() {} - func contextOverride(forSkill skillID: String?) -> Int? { nil } -} - -/// The CONCRETE context budget (design D5, integration fix C3). It conforms to `ai-conversation-runtime`'s -/// `ContextBudgetProviding` — that protocol is OWNED there; this slice supplies the concrete value, so -/// conversation-runtime's compaction reads the same budget the user chose without a DAG back-edge. -/// -/// The effective budget is the user's `agentContextTokens`, raised to a heavy skill's override if larger, -/// and CLAMPED to the model's architectural `maxContextTokens` — so growing the slider directly raises the -/// compaction trigger and the two never disagree about "the budget." -struct AgentContextBudgetProvider: ContextBudgetProviding { - let userContextTokens: Int - let modelMaxContextTokens: Int - let activeSkillID: String? - let skillOverrides: SkillContextOverriding - - init(userContextTokens: Int, - modelMaxContextTokens: Int, - activeSkillID: String? = nil, - skillOverrides: SkillContextOverriding = NoSkillContextOverride()) { - self.userContextTokens = userContextTokens - self.modelMaxContextTokens = modelMaxContextTokens - self.activeSkillID = activeSkillID - self.skillOverrides = skillOverrides - } - - var maxContextTokens: Int { - let skillRaise = skillOverrides.contextOverride(forSkill: activeSkillID) ?? 0 - let desired = max(userContextTokens, skillRaise) - return min(max(1, desired), max(1, modelMaxContextTokens)) // ∩ model max, never below 1 - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Agent/AgentContextCostModel.swift b/Sources/ThreeFingerSwitcher/AI/Agent/AgentContextCostModel.swift deleted file mode 100644 index 76caf7b..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Agent/AgentContextCostModel.swift +++ /dev/null @@ -1,78 +0,0 @@ -import Foundation - -/// The Hub cost surface model (design D5, house requirement — never silent OOM). Given the chosen context -/// preset/tokens + the compact-KV toggle + the selected model's weight size & max context, it derives — -/// from the SAME pure `ConcurrencyBudget`/`KVCacheCost` the batched conformer uses — the estimated resident -/// RAM, the concurrent-stream count (so the user sees how many BACKGROUND sessions a context affords), and -/// a relative speed note. Pure + MLX-free (the live free-memory probe is `ProcessInfo.physicalMemory`, -/// available in Core) so it is `swift test`-buildable and tracks the preset/toggle live. -/// -/// The KV constants mirror `BatchedGemmaMLXRuntime.currentBudget` (representative Gemma-class numbers) so -/// the Hub's displayed RAM/stream count agrees with what the conformer actually computes for K. -struct AgentContextCostModel { - /// The resolved context budget (clamped to the model max), the compact-KV toggle, and the model. - let contextTokens: Int - let compactKV: Bool - let weightBytes: Int64 - /// Injected so the math is unit-testable with a fixed memory size (defaults to the live probe). - let unifiedMemoryBytes: Int64 - - init(contextTokens: Int, compactKV: Bool, weightBytes: Int64, - unifiedMemoryBytes: Int64 = Int64(ProcessInfo.processInfo.physicalMemory)) { - self.contextTokens = max(1, contextTokens) - self.compactKV = compactKV - self.weightBytes = weightBytes - self.unifiedMemoryBytes = unifiedMemoryBytes - } - - /// The pure budget, built with the same KV cost model the GemmaRuntime conformer uses. - var budget: ConcurrencyBudget { - let bf16PerTokenPerLayer = 2_048.0 - let perTokenPerLayer = compactKV ? bf16PerTokenPerLayer / 2 : bf16PerTokenPerLayer - let kv = KVCacheCost(slidingLayers: 40, globalLayers: 8, slidingWindow: 1_024, - kvBytesPerTokenPerLayer: perTokenPerLayer) - return ConcurrencyBudget(unifiedMemoryBytes: unifiedMemoryBytes, - weightBytes: weightBytes, - reservedBytes: 6 * 1_000_000_000, - kv: kv) - } - - /// K — total concurrent streams that fit at this context (≥ 1, foreground always fits). - var maxStreams: Int { budget.maxStreams(contextTokens: contextTokens) } - - /// Background sessions afforded beyond the foreground slot (0 when only the foreground fits). - var backgroundStreams: Int { max(0, maxStreams - 1) } - - /// Estimated resident RAM at this (K, context) for the cost surface. - var estimatedRAMBytes: Int64 { budget.estimatedRAM(streams: maxStreams, contextTokens: contextTokens) } - - /// A human "~NN GB" string for the estimated RAM (one decimal below 100 GB). - var ramText: String { - let gb = Double(estimatedRAMBytes) / 1_000_000_000 - return gb >= 100 ? String(format: "~%.0f GB", gb) : String(format: "~%.1f GB", gb) - } - - /// "3 background sessions" / "1 background session" / "no background sessions" (honest at K=1). - var backgroundText: String { - switch backgroundStreams { - case 0: return "no background sessions" - case 1: return "1 background session" - default: return "\(backgroundStreams) background sessions" - } - } - - /// A relative speed note — longer context = slower per token (design D5). Buckets by context size. - var speedNote: String { - switch contextTokens { - case ..<12_000: return "fastest per-token speed" - case ..<40_000: return "moderate per-token speed" - default: return "slower per-token speed" - } - } - - /// The single-line cost summary the Hub shows, updating live with the preset/toggle. e.g. - /// "~24.0 GB · 3 background sessions · moderate per-token speed". - var summary: String { - "\(ramText) · \(backgroundText) · \(speedNote)" - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Agent/AgentContextPreset.swift b/Sources/ThreeFingerSwitcher/AI/Agent/AgentContextPreset.swift deleted file mode 100644 index dc81aa4..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Agent/AgentContextPreset.swift +++ /dev/null @@ -1,34 +0,0 @@ -import Foundation - -/// The user-facing context-size preset (design D5). Three comprehensible presets plus `custom`; the -/// resolved token budget is always clamped to the model's `maxContextTokens`. Longer context trades -/// concurrency (fewer background streams) and per-token speed for recall — the Hub surfaces that cost. -public enum AgentContextPreset: String, Codable, Sendable, CaseIterable { - case balanced // a comfortable mid value — the default - case long - case max // the model's architectural maximum - case custom // an explicit `agentContextTokens` value - - /// The token budget this preset resolves to, clamped to the model max. `custom` uses the explicit - /// stored value. - public func tokens(modelMax: Int, custom: Int) -> Int { - // `max`/`min` are qualified — inside this enum body the bare names resolve to the `.max` case. - let clamped: (Int) -> Int = { Swift.min(Swift.max(1, $0), Swift.max(1, modelMax)) } - switch self { - case .balanced: return clamped(8_192) - case .long: return clamped(32_768) - case .max: return Swift.max(1, modelMax) - case .custom: return clamped(custom) - } - } - - /// A short human label for the Hub segmented control. - public var title: String { - switch self { - case .balanced: return "Balanced" - case .long: return "Long" - case .max: return "Max" - case .custom: return "Custom" - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Agent/AgentConversation.swift b/Sources/ThreeFingerSwitcher/AI/Agent/AgentConversation.swift deleted file mode 100644 index 546714f..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Agent/AgentConversation.swift +++ /dev/null @@ -1,240 +0,0 @@ -import Foundation - -/// The canonical multi-turn conversation types (blueprint §3.1 / design D1). This slice -/// (`ai-conversation-runtime`, Wave 1) OWNS them; every other V2 slice — tool-routing, the -/// conversational canvas, parked sessions, memory, background autonomy, the batched runtime — imports -/// these shapes verbatim rather than inventing conflicting ones. MLX-free Core (`swift test`-verified). -/// -/// THE LOAD-BEARING INVARIANT: a message's committed text (`text`) is the ONLY content ever re-fed into -/// the model's context on a later turn; its reasoning (`thinking`) is retained for DISPLAY only and is -/// NEVER re-fed as ground truth. Thinking lives in a separate field so the exclusion is *structural* — -/// the assembler (`ChatTemplate.flatten`) reads `text` only and never touches `thinking`. A bug that -/// re-fed reasoning would both bloat the context window and violate the contract; keeping the two fields -/// apart makes that bug impossible by construction rather than by a fragile strip step. The ordered -/// `segments` timeline (`notch-timeline-and-tuning`) sits in the same DISPLAY-only tier as `thinking`: -/// assembly never reads it. - -/// One display-only block of an assistant turn's TIMELINE (`notch-timeline-and-tuning`): the turn's -/// thinking and answer in token-arrival order, a new segment starting wherever the channel flips — so the -/// transcript can re-render the interleaved thinking/answer sequence exactly as it streamed. Display-only -/// like `thinking` (never re-fed; assembly reads `text` only). -public struct TurnSegment: Codable, Equatable, Sendable { - public enum Kind: String, Codable, Sendable { - case thinking, answer - } - public var kind: Kind - public var text: String - - public init(kind: Kind, text: String) { - self.kind = kind - self.text = text - } -} - -/// The role of a message in a conversation. Mirrors the standard chat roles; `.tool` carries a tool -/// step's outcome back into the thread. -public enum AgentRole: String, Codable, Sendable { - case user, assistant, system, tool -} - -/// One message in a conversation — the atom of a thread. -/// -/// `text` is the committed user/response content (the contract for *what is re-fed*). `thinking` is the -/// model's reasoning (the contract for *what is shown but never re-fed*). See the file-level invariant. -public struct AgentMessage: Codable, Equatable, Identifiable, Sendable { - public let id: UUID - public var role: AgentRole - /// The committed user/response text — NEVER the thinking. This is the only content re-fed on a - /// later turn (assembly reads this field exclusively). - public var text: String - /// The model's reasoning, retained for DISPLAY only; never re-fed verbatim as ground truth. - public var thinking: String? - /// The turn's ordered thinking/answer TIMELINE (`notch-timeline-and-tuning`), retained for DISPLAY - /// only like `thinking` — never re-fed, structurally excluded from assembly. Optional and decode-safe: - /// a message persisted before this field renders via the `displaySegments` legacy fallback. - public var segments: [TurnSegment]? - /// The turn's images (each PNG); a SINGLE turn may carry MULTIPLE images (design D2 — a multi-image - /// turn). Empty for a text-only turn. `LLMChatRequest.images` forwards the latest turn's full array. - public var images: [Data] - /// An assistant turn that routed to tools (type owned by `ai-tool-routing`; see `ToolPlaceholders`). - public var toolCalls: [ToolRoute]? - /// For `role == .tool`: the executed step's outcome (owned by `ai-tool-routing`). - public var toolResult: ToolStepResult? - public var createdAt: Date - - /// The turn's FIRST image, or nil when it carries none — the single-image convenience kept for callers - /// and assembly that only need one image (design D2: `images` is the source of truth; this is derived). - public var image: Data? { images.first } - - /// The message's display TIMELINE: the persisted `segments` when present, else the legacy fallback — - /// the flat `thinking` (a message stored before segments existed) as one leading thinking block, - /// followed by the committed `text` as one answer block. The single read every transcript renderer - /// uses, so old and new messages draw through the same path. - public var displaySegments: [TurnSegment] { - if let segments, !segments.isEmpty { return segments } - var out: [TurnSegment] = [] - if let thinking, !thinking.isEmpty { out.append(TurnSegment(kind: .thinking, text: thinking)) } - out.append(TurnSegment(kind: .answer, text: text)) - return out - } - - /// Designated init. `images` defaults to `[]`; the `image:` convenience init folds a single image in. - public init(id: UUID = UUID(), - role: AgentRole, - text: String, - thinking: String? = nil, - segments: [TurnSegment]? = nil, - images: [Data] = [], - toolCalls: [ToolRoute]? = nil, - toolResult: ToolStepResult? = nil, - createdAt: Date = Date()) { - self.id = id - self.role = role - self.text = text - self.thinking = thinking - self.segments = segments - self.images = images - self.toolCalls = toolCalls - self.toolResult = toolResult - self.createdAt = createdAt - } - - /// Single-image convenience (design D2): folds one optional image into the `images` array so existing - /// callers (`AgentMessage(... image: png)`) keep compiling unchanged. A nil image yields `[]`. - public init(id: UUID = UUID(), - role: AgentRole, - text: String, - thinking: String? = nil, - image: Data?, - toolCalls: [ToolRoute]? = nil, - toolResult: ToolStepResult? = nil, - createdAt: Date = Date()) { - self.init(id: id, role: role, text: text, thinking: thinking, segments: nil, - images: image.map { [$0] } ?? [], - toolCalls: toolCalls, toolResult: toolResult, createdAt: createdAt) - } - - // MARK: Codable (tolerant decode) - // - // The image field migrated from a single `image: Data?` to `images: [Data]` (design D2). A custom - // decoder accepts EITHER the new `images` array OR a legacy singular `image` value (folded into the - // array), defaulting to `[]` — so a conversation persisted under the old shape still decodes. Encode - // always writes the new `images` array (the singular key is decode-only legacy tolerance). - private enum CodingKeys: String, CodingKey { - case id, role, text, thinking, segments, images, image, toolCalls, toolResult, createdAt - } - - public init(from decoder: Decoder) throws { - let c = try decoder.container(keyedBy: CodingKeys.self) - self.id = try c.decode(UUID.self, forKey: .id) - self.role = try c.decode(AgentRole.self, forKey: .role) - self.text = try c.decode(String.self, forKey: .text) - self.thinking = try c.decodeIfPresent(String.self, forKey: .thinking) - self.segments = try c.decodeIfPresent([TurnSegment].self, forKey: .segments) - if let arr = try c.decodeIfPresent([Data].self, forKey: .images) { - self.images = arr - } else if let single = try c.decodeIfPresent(Data.self, forKey: .image) { - self.images = [single] - } else { - self.images = [] - } - self.toolCalls = try c.decodeIfPresent([ToolRoute].self, forKey: .toolCalls) - self.toolResult = try c.decodeIfPresent(ToolStepResult.self, forKey: .toolResult) - self.createdAt = try c.decode(Date.self, forKey: .createdAt) - } - - public func encode(to encoder: Encoder) throws { - var c = encoder.container(keyedBy: CodingKeys.self) - try c.encode(id, forKey: .id) - try c.encode(role, forKey: .role) - try c.encode(text, forKey: .text) - try c.encodeIfPresent(thinking, forKey: .thinking) - try c.encodeIfPresent(segments, forKey: .segments) - try c.encode(images, forKey: .images) - try c.encodeIfPresent(toolCalls, forKey: .toolCalls) - try c.encodeIfPresent(toolResult, forKey: .toolResult) - try c.encode(createdAt, forKey: .createdAt) - } -} - -/// The session identity threaded through EVERY slice — one opaque value, stable across park/restore (the -/// parked slice persists a conversation under this id; the batched runtime keys its streams by it). -public struct AgentSessionID: Hashable, Codable, Sendable { - public let raw: UUID - public init(raw: UUID = UUID()) { self.raw = raw } -} - -/// An ordered conversation: a message list with a title and timestamps. `Codable` so the parked-sessions -/// slice can persist it — but THIS slice owns only the type, not the durable store (no duplication). -public struct AgentConversation: Codable, Equatable, Identifiable, Sendable { - public let id: AgentSessionID - /// Short, model- or first-turn-derived; shown on the rail/badge by the parked slice. - public var title: String - public var messages: [AgentMessage] - public var createdAt: Date - public var updatedAt: Date - /// Compaction output: a prefix summary that replaces dropped older turns (see `ConversationCompactor`). - public var compactedSummary: String? - /// The active skill (file id) driving this session, if any (owned by `ai-skills-as-files`). - public var skillID: String? - /// The per-conversation auto-approve grant (`add-voice-computer-use-agent`, design D7): while - /// true, `.confirm`-tier acts execute without a per-step pause (narrated). Granted only through - /// the gated `enable_auto_mode` / a surface toggle / a parsed initial-command intent; revocation - /// is instant. Persisted with the conversation. Optional so rows stored by older builds decode - /// cleanly (nil reads as false). - public var autoApprove: Bool? - /// BORN-WITH tuning (`notch-timeline-and-tuning`): the reasoning flag this conversation was created - /// under. Stamped at birth from the notch tuning slider and carried for the conversation's whole - /// life — a later slider change never retunes an existing session. Optional and decode-safe: nil - /// (a pre-change conversation) falls back to the surface's global reasoning default at bind time. - public var reasoningOverride: Bool? - /// BORN-WITH tuning: the context-token budget this conversation was created under (already clamped - /// to the model max at birth). Feeds compaction for this conversation only. Optional and - /// decode-safe: nil falls back to the engine's injected budget. - public var contextTokens: Int? - - /// The effective grant (nil-safe read). - public var isAutoApproveGranted: Bool { autoApprove ?? false } - - public init(id: AgentSessionID = AgentSessionID(), - title: String, - messages: [AgentMessage], - createdAt: Date = Date(), - updatedAt: Date = Date(), - compactedSummary: String? = nil, - skillID: String? = nil, - autoApprove: Bool? = nil, - reasoningOverride: Bool? = nil, - contextTokens: Int? = nil) { - self.id = id - self.title = title - self.messages = messages - self.createdAt = createdAt - self.updatedAt = updatedAt - self.compactedSummary = compactedSummary - self.skillID = skillID - self.autoApprove = autoApprove - self.reasoningOverride = reasoningOverride - self.contextTokens = contextTokens - } -} - -/// A single conversational turn unit fed to the runtime (AFTER compaction is applied) — the transient, -/// in-memory product of "windowed/compacted messages + the latest image + per-turn reasoning + -/// parameters". Not `Codable` (only `AgentConversation` persists). -public struct AgentTurn: Sendable { - public var messages: [AgentMessage] - public var image: Data? - public var reasoning: Bool - public var parameters: GenerationParameters - - public init(messages: [AgentMessage], - image: Data? = nil, - reasoning: Bool = false, - parameters: GenerationParameters = .default) { - self.messages = messages - self.image = image - self.reasoning = reasoning - self.parameters = parameters - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Agent/AgentLoop.swift b/Sources/ThreeFingerSwitcher/AI/Agent/AgentLoop.swift deleted file mode 100644 index caf074f..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Agent/AgentLoop.swift +++ /dev/null @@ -1,270 +0,0 @@ -import Foundation - -/// Why a bounded loop stopped short of a plain answer. -enum AgentStopReason: Equatable, Sendable { - case repeatedStep // a byte-identical route to the immediately preceding executed call (spinning) - case noProgress // re-routed to a tool that already declined/failed more than once - case cancelled // the user discarded the canvas (a discard, never a failure) -} - -/// The terminal outcome of the bounded agent loop (design D7). Every text-carrying outcome holds the -/// best-effort final answer — the loop NEVER ends with a bare halt. `.pausedAwaitingUser` is the one -/// deliberate exception (`refactor-park-and-background-agents`): a background approval pause ends the -/// turn with NO fabricated answer — the pending step is observable state, not a completion. There is no -/// "task complete" notion here anymore: the old terminal classification let a docked chat's settled -/// answer auto-dismiss (delete) its session. -enum AgentLoopOutcome: Equatable, Sendable { - case answered(text: String) - case capReached(text: String) - case stopped(reason: AgentStopReason, text: String) - /// A step is pending on the user (background waitParked/escalate): the loop pauses honestly — the - /// step is neither run nor skipped, and no final answer is synthesized. - case pausedAwaitingUser - case failed(headline: String) -} - -/// The loop's full result: the outcome plus the ordered tool steps it ran (the canvas binds these as -/// observable state; tests assert them). -struct AgentLoopResult: Sendable { - var outcome: AgentLoopOutcome - var steps: [ToolStepResult] -} - -/// The loop's wall-clock bounds (`add-voice-computer-use-agent`, design D8): a per-step timeout and -/// a per-turn deadline. Both are BUDGET errors, not runtime errors — a timed-out step settles -/// `.failed("… timed out")` (never a misleading network headline) and an exceeded deadline -/// terminates via the existing cap-reached fallback. Human time is exempt: a step that pauses at -/// the approval gate is never step-timed, and the deadline re-baselines after it (a user thinking -/// about an approval must not spend the agent's budget). -struct LoopBudget: Sendable { - /// Per-step wall clock for NON-GATED dispatches (auto tier / auto-granted). Seconds. - var stepTimeout: TimeInterval - /// Total active wall clock for the whole turn. Seconds. - var turnDeadline: TimeInterval - - static let `default` = LoopBudget(stepTimeout: 30, turnDeadline: 180) -} - -/// The bounded route → execute → continue loop (design D7), pure Core, owns no UI. It drives an injected -/// `LLMRuntime` (the batched conformer in production, `StubLLMRuntime` in tests), routes via -/// `structured()`, dispatches through the `ToolRegistry`, and is bounded by a hard step cap plus two -/// loop-guards — and, since `add-voice-computer-use-agent`, the wall-clock `LoopBudget`. The running -/// plan rides the existing `.thinking` channel via `onThinking`; the committed answer is -/// `.response`-channel only. -struct AgentLoop: Sendable { - let runtime: LLMRuntime - let registry: ToolRegistry - let candidateSource: ToolCandidateSource - let gate: ApprovalGate - let reasoning: Bool - let maxToolSteps: Int - let source: TaskSource - /// Live plan sink: each route rationale + tool-step summary is emitted here (the canvas renders it in - /// its collapsible Thinking section). Default no-op. - let onThinking: @Sendable (String) -> Void - /// Live RESPONSE-token sink (`add-voice-computer-use-agent`): the final answer's `.response` - /// tokens stream here AS they generate — the voice surface feeds its sentence chunker from this - /// so the first sentence speaks before the loop returns. Default no-op (the canvas keeps reading - /// the returned text). - let onResponseToken: @Sendable (String) -> Void - /// The session this loop advances — audited + park-state-keyed by `ai-background-autonomy`. Defaults - /// to a fresh id for the foreground one-shot loop. - let sessionID: AgentSessionID - /// Optional background-policy runner (`ai-background-autonomy`, 6.1–6.4): when present, each tool step - /// routes through it (whitelist-aware tier → `BackgroundGate` decision → run/escalate/wait + audit) - /// instead of calling `registry.run` directly. nil = the plain foreground path (existing behavior). - let backgroundRunner: BackgroundToolRunner? - /// The wall-clock bounds (design D8). Defaults keep pre-change behavior practically unchanged. - let budget: LoopBudget - /// Whether the conversation's auto-approve grant is live (the same signal the gate wrapper reads): - /// a `.confirm` step under the grant cannot pause on a human, so it IS step-timed. - let isAutoGranted: @Sendable () -> Bool - /// Injected clock so deadline behavior is deterministically testable. - let clock: @Sendable () -> Date - - init(runtime: LLMRuntime, registry: ToolRegistry, candidateSource: ToolCandidateSource, - gate: ApprovalGate, reasoning: Bool = false, maxToolSteps: Int = 8, - source: TaskSource = TaskSource(), onThinking: @escaping @Sendable (String) -> Void = { _ in }, - sessionID: AgentSessionID = AgentSessionID(), backgroundRunner: BackgroundToolRunner? = nil, - budget: LoopBudget = .default, - isAutoGranted: @escaping @Sendable () -> Bool = { false }, - clock: @escaping @Sendable () -> Date = { Date() }, - onResponseToken: @escaping @Sendable (String) -> Void = { _ in }) { - self.runtime = runtime - self.registry = registry - self.candidateSource = candidateSource - self.gate = gate - self.reasoning = reasoning - self.maxToolSteps = maxToolSteps - self.source = source - self.onThinking = onThinking - self.sessionID = sessionID - self.backgroundRunner = backgroundRunner - self.budget = budget - self.isAutoGranted = isAutoGranted - self.clock = clock - self.onResponseToken = onResponseToken - } - - func run(context initial: RouteContext) async -> AgentLoopResult { - var context = initial - var steps: [ToolStepResult] = [] - var lastRoute: ToolRoute? - var candidateLimit = 5 - var declineCount: [String: Int] = [:] - // The turn deadline's baseline. Re-baselined after any step that could pause on a human - // (approval wait is the USER's time, never the agent's budget). - var deadlineStart = clock() - - for _ in 0 ..< max(1, maxToolSteps) { - if Task.isCancelled { return AgentLoopResult(outcome: .stopped(reason: .cancelled, text: ""), steps: steps) } - // The turn deadline (design D8): exceeded → terminate through the existing cap-reached - // fallback with an honest partial summary. - if budget.turnDeadline > 0, clock().timeIntervalSince(deadlineStart) > budget.turnDeadline { - onThinking("Ran out of time for this turn.\n") - return await answer(context, steps: steps, terminator: .cap) - } - - // Retrieve candidates (~5) and always offer the widen tool so the model can ask for more. - var candidates = candidateSource.candidates(for: context, limit: candidateLimit) - candidates.append(.widenCandidates) - - switch await ToolRouter.route(context: context, candidates: candidates, - runtime: runtime, reasoning: reasoning) { - case .cancelled: - return AgentLoopResult(outcome: .stopped(reason: .cancelled, text: ""), steps: steps) - case let .failed(headline): - return AgentLoopResult(outcome: .failed(headline: headline), steps: steps) - case let .route(route): - if let rationale = route.rationale, !rationale.isEmpty { onThinking(rationale + "\n") } - - if route.isPlainAnswer { - return await answer(context, steps: steps, terminator: .plain) - } - if route.tool == ToolDescriptor.widenCandidates.name { - candidateLimit = min(candidateLimit + 5, 20) - onThinking("Looking for more tools…\n") - continue - } - guard let descriptor = registry.descriptor(named: route.tool) else { - return await answer(context, steps: steps, terminator: .plain) // defensive (router normalizes) - } - // Loop-guard: a byte-identical consecutive route means the model is spinning. - if let last = lastRoute, last.tool == route.tool, last.argumentsJSON == route.argumentsJSON { - return await answer(context, steps: steps, terminator: .stopped(.repeatedStep)) - } - - let call = RoutedCall(descriptor: descriptor, route: route, - userText: context.latestUserText, source: source) - // A step that may PAUSE ON A HUMAN (confirm/dangerous without the auto grant) is - // never step-timed, and the deadline re-baselines after it — approval wait is user - // time. Everything else races the step timeout (design D8). - let mayPauseOnHuman = descriptor.writePolicy != .auto && !isAutoGranted() - // Background-autonomy path (6.1–6.4): when a runner is injected, each step routes through - // it (whitelist-aware tier → BackgroundGate → run/escalate/wait + audit); otherwise the - // plain foreground run is unchanged. - let result = await dispatch(call, timed: !mayPauseOnHuman) - if mayPauseOnHuman { deadlineStart = clock() } - steps.append(result) - if !result.summary.isEmpty { onThinking(result.summary + "\n") } - context.messages.append(AgentMessage(role: .tool, text: result.summary, toolResult: result)) - lastRoute = route - - switch result.status { - case let .failed(headline): - // A side effect that didn't land — surface it and stop (never a false "Done"). - return AgentLoopResult(outcome: .failed(headline: headline), steps: steps) - case let .declined(reason): - if reason == TaskKindToolContributor.cancelledReason { - return AgentLoopResult(outcome: .stopped(reason: .cancelled, text: ""), steps: steps) - } - // No-progress guard: re-declining the same tool more than once terminates. - declineCount[route.tool, default: 0] += 1 - if declineCount[route.tool, default: 0] > 1 { - return await answer(context, steps: steps, terminator: .stopped(.noProgress)) - } - continue // the model sees the decline and may pivot - case .done: - continue - case .awaitingApproval: - // Only the BACKGROUND runner returns this status as a result (the foreground gate - // resolves inside the contributor's await, yielding done/declined): the step is - // pending on the user — pause the turn honestly. Continuing here would neither run - // nor suspend the step and would fabricate a final answer over phantom work. - return AgentLoopResult(outcome: .pausedAwaitingUser, steps: steps) - } - } - } - // The hard cap is the backstop regardless of progress. - return await answer(context, steps: steps, terminator: .cap) - } - - // MARK: - Dispatch with the step timeout - - /// Run one routed call, racing the step timeout when `timed`. A timed-out step CANCELS its work - /// (tools are cancellation-safe by contract) and settles `.failed("… timed out")` — a clean - /// budget headline, never a fabricated success and never a misleading network error. - private func dispatch(_ call: RoutedCall, timed: Bool) async -> ToolStepResult { - let runStep: @Sendable () async -> ToolStepResult = { [self] in - if let runner = backgroundRunner { - return await runner.run(call, sessionID: sessionID, registry: registry, gate: gate) - } - return await registry.run(call, gate: gate) - } - guard timed, budget.stepTimeout > 0 else { return await runStep() } - - return await withTaskGroup(of: ToolStepResult?.self) { group in - group.addTask { await runStep() } - group.addTask { [stepTimeout = budget.stepTimeout] in - try? await Task.sleep(nanoseconds: UInt64(stepTimeout * 1_000_000_000)) - return nil // the timeout sentinel - } - let first = await group.next() ?? nil - group.cancelAll() - if let result = first { return result } - return ToolStepResult( - tool: call.descriptor.name, - status: .failed(headline: "“\(call.descriptor.name)” timed out."), - summary: "“\(call.descriptor.name)” didn't finish within \(Int(budget.stepTimeout))s and was stopped.") - } - } - - // MARK: - Final answer - - private enum Terminator { - case plain - case cap - case stopped(AgentStopReason) - } - - /// Stream a best-effort final text answer over the (possibly tool-augmented) context, then wrap it in - /// the right terminal outcome. Cancellation during this is a quiet `.cancelled` (a discard). - private func answer(_ context: RouteContext, steps: [ToolStepResult], terminator: Terminator) async -> AgentLoopResult { - var text = "" - do { - for try await token in runtime.chat(LLMChatRequest(messages: context.messages, reasoning: reasoning)) { - if Task.isCancelled { return AgentLoopResult(outcome: .stopped(reason: .cancelled, text: ""), steps: steps) } - switch token.channel { - case .response: - text += token.text - onResponseToken(token.text) - case .thinking: - onThinking(token.text) - } - } - } catch let error as RuntimeError where error == .cancelled { - return AgentLoopResult(outcome: .stopped(reason: .cancelled, text: ""), steps: steps) - } catch is CancellationError { - return AgentLoopResult(outcome: .stopped(reason: .cancelled, text: ""), steps: steps) - } catch { - return AgentLoopResult(outcome: .failed(headline: AIError.message(for: error).headline), steps: steps) - } - if Task.isCancelled { return AgentLoopResult(outcome: .stopped(reason: .cancelled, text: ""), steps: steps) } - - switch terminator { - case .plain: return AgentLoopResult(outcome: .answered(text: text), steps: steps) - case .cap: return AgentLoopResult(outcome: .capReached(text: text), steps: steps) - case let .stopped(reason): return AgentLoopResult(outcome: .stopped(reason: reason, text: text), steps: steps) - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Agent/AutoApprovingGate.swift b/Sources/ThreeFingerSwitcher/AI/Agent/AutoApprovingGate.swift deleted file mode 100644 index 64091c5..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Agent/AutoApprovingGate.swift +++ /dev/null @@ -1,70 +0,0 @@ -import Foundation - -/// The auto-mode gate wrapper (`add-voice-computer-use-agent`, design D7 / spec "Acts respect the -/// write-policy gate, with a per-conversation auto-approve mode"): wraps the real foreground gate; -/// while the conversation's grant is live, a `.confirm` pause resolves `.approve` IMMEDIATELY and -/// the act is narrated (spoken when voice is active, always visible) — silence never hides an act. -/// Without the grant it is a transparent pass-through. The `enable_auto_mode` tool itself always -/// reaches the BASE gate through its own `.confirm` tier before any grant exists, so turning auto -/// mode on is the one approval that can't be skipped. -/// A lock-guarded boolean readable from any thread — the auto-grant's live storage. The route loop -/// reads it OFF the main actor (same reason `ParkScheduler.parkState(of:)` is lock-guarded), while -/// the engine mutates it on the main actor. -public final class LockedBool: @unchecked Sendable { - private let lock = NSLock() - private var flag = false - - public init(_ initial: Bool = false) { flag = initial } - - public var value: Bool { - get { lock.lock(); defer { lock.unlock() }; return flag } - set { lock.lock(); flag = newValue; lock.unlock() } - } -} - -/// The VOICE surface's base gate (`add-voice-computer-use-agent`): a voice conversation has no -/// approval canvas, so a `.confirm` step that reaches here SPEAKS one line of guidance and SKIPS — -/// never a silent decline, never a fabricated approval. Acts by voice therefore require the -/// auto-mode grant (given explicitly in the spoken command or on the surface toggle), which the -/// wrapping `AutoApprovingGate` resolves before this gate is ever consulted. -final class SpokenGuidanceGate: ApprovalGate { - private let speak: @Sendable (String) -> Void - - init(speak: @escaping @Sendable (String) -> Void) { - self.speak = speak - } - - func awaitDecision(for review: TaskReview) async -> ApprovalDecision { - if case let .action(title, _, _) = review { - speak("\(title) needs approval. Say “enable auto mode” to let me act, or use the chat surface.") - } - return .skip - } -} - -final class AutoApprovingGate: ApprovalGate { - private let base: ApprovalGate - /// Live read of the conversation's grant (thread-safe — the loop calls off-main). - private let isGranted: @Sendable () -> Bool - /// Narration sink for auto-approved acts (title + preview fields, one line). - private let narrate: @Sendable (String) -> Void - - init(base: ApprovalGate, - isGranted: @escaping @Sendable () -> Bool, - narrate: @escaping @Sendable (String) -> Void = { _ in }) { - self.base = base - self.isGranted = isGranted - self.narrate = narrate - } - - func awaitDecision(for review: TaskReview) async -> ApprovalDecision { - guard isGranted() else { - return await base.awaitDecision(for: review) - } - if case let .action(title, fields, _) = review { - let detail = fields.map { "\($0.label): \($0.value)" }.joined(separator: ", ") - narrate("Auto: \(title)\(detail.isEmpty ? "" : " — \(detail)")") - } - return .approve - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Agent/CanvasApprovalGate.swift b/Sources/ThreeFingerSwitcher/AI/Agent/CanvasApprovalGate.swift deleted file mode 100644 index e4a72df..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Agent/CanvasApprovalGate.swift +++ /dev/null @@ -1,68 +0,0 @@ -import Foundation - -/// The `ApprovalGate` the conversational canvas drives via its canonical compass (design D7, -/// `ai-conversational-canvas` task 2.8): when the bounded route loop reaches a `.confirm`/`.dangerous` -/// tool step, the owning contributor `await`s a decision here, which suspends the loop and surfaces the -/// backing `TaskReview` to the canvas (DOWN = approve, RIGHT = skip — identical mnemonic to commit / -/// discard). The recognizer is UNTOUCHED; interpretation lives at the `AppCoordinator` seam, which calls -/// `approve()` / `skip()` on the executor, which forwards here. -/// -/// It is a standalone `Sendable` class (NOT `@MainActor`) because `ApprovalGate.awaitDecision` is invoked -/// from `ToolRegistry.run` OFF the main actor inside the loop's task; it bridges back to the executor's -/// main-actor observable state through the injected `onAwait` closure (which sets `.awaitingApproval`). -/// A single in-flight pause at a time (the loop is sequential); a second `awaitDecision` before the first -/// resolves is defensive-resolved to `.skip` so the loop can never deadlock. -final class CanvasApprovalGate: ApprovalGate, @unchecked Sendable { - /// Called (on the main actor) the instant a step needs approval, so the executor can transition to - /// `.awaitingApproval(review)` and the canvas can render the review card. - private let onAwait: @MainActor (TaskReview) -> Void - /// Called (on the main actor) when a pending pause resolves (approve/skip/cancel), so the executor can - /// leave `.awaitingApproval` (the loop resumes and re-publishes its next state). - private let onResolve: @MainActor (ApprovalDecision) -> Void - - private let lock = NSLock() - private var pending: CheckedContinuation? - - init(onAwait: @escaping @MainActor (TaskReview) -> Void, - onResolve: @escaping @MainActor (ApprovalDecision) -> Void = { _ in }) { - self.onAwait = onAwait - self.onResolve = onResolve - } - - func awaitDecision(for review: TaskReview) async -> ApprovalDecision { - if Task.isCancelled { return .cancel } - return await withCheckedContinuation { continuation in - lock.lock() - if pending != nil { - // Defensive: a second pause before the first resolved — decline it so nothing deadlocks. - lock.unlock() - continuation.resume(returning: .skip) - return - } - pending = continuation - lock.unlock() - Task { @MainActor in self.onAwait(review) } - } - } - - /// Resolve the pending pause (DOWN = `.approve`, RIGHT = `.skip`, a whole-canvas discard = `.cancel`). - /// A no-op when nothing is awaiting. Returns whether a pause was actually resolved (so the seam can - /// fall through to the normal commit/discard path when no step is pending). - @discardableResult - func resolve(_ decision: ApprovalDecision) -> Bool { - lock.lock() - let continuation = pending - pending = nil - lock.unlock() - guard let continuation else { return false } - Task { @MainActor in self.onResolve(decision) } - continuation.resume(returning: decision) - return true - } - - /// Whether a tool step is currently awaiting a decision. - var isAwaiting: Bool { - lock.lock(); defer { lock.unlock() } - return pending != nil - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Agent/ChatTemplate.swift b/Sources/ThreeFingerSwitcher/AI/Agent/ChatTemplate.swift deleted file mode 100644 index 9ad82e7..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Agent/ChatTemplate.swift +++ /dev/null @@ -1,34 +0,0 @@ -import Foundation - -/// The Core, model-agnostic message-list→prompt assembler (design D3). The default `LLMRuntime.chat(_:)` -/// needs *a* prompt from a message list; this produces a deterministic, role-labeled transcript. -/// -/// Reads `message.text` ONLY — never `thinking` — so the default conversation path cannot leak reasoning -/// into the prompt (the load-bearing invariant of `AgentMessage`). -/// -/// FLAGGED: GemmaRuntime — the REAL model-specific chat template (Gemma's `` turn markers -/// and the `enable_thinking` reasoning flag) is owned by the on-device Gemma conformer in the -/// `ai-batched-runtime-and-context` / GemmaRuntime slice, which OVERRIDES `chat(_:)` to build the native -/// template instead of calling `flatten`. This Core assembler is the model-agnostic default + the basis -/// for deterministic tests; it is intentionally NOT Gemma's real template. -public enum ChatTemplate { - - /// Build a deterministic, role-labeled transcript from `messages`, reading committed `text` only. - /// A `.tool` message renders its `toolResult?.summary` (the outcome fed back into the loop). A - /// trailing `Assistant:` cue invites the next turn. An empty list yields the cue alone. - public static func flatten(_ messages: [AgentMessage]) -> String { - let lines: [String] = messages.map { message in - switch message.role { - case .system: return "System: \(message.text)" - case .user: return "User: \(message.text)" - case .assistant: return "Assistant: \(message.text)" - case .tool: return "Tool: \(message.toolResult?.summary ?? message.text)" - } - } - guard !lines.isEmpty else { return assistantCue } - return lines.joined(separator: "\n\n") + "\n\n" + assistantCue - } - - /// The trailing cue that invites the model to produce the next assistant turn. - static let assistantCue = "Assistant:" -} diff --git a/Sources/ThreeFingerSwitcher/AI/Agent/ConcurrencyBudget.swift b/Sources/ThreeFingerSwitcher/AI/Agent/ConcurrencyBudget.swift deleted file mode 100644 index 6b65300..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Agent/ConcurrencyBudget.swift +++ /dev/null @@ -1,62 +0,0 @@ -import Foundation - -/// The per-stream KV-cache cost model (design D3/D4). Gemma 4 interleaves **local sliding-window** layers -/// (most layers; their KV is bounded by the window) with a few **global** layers (full-context KV), so -/// per-token KV bytes are NOT uniform across layers — treating every layer as global over-estimates KV -/// and needlessly throttles concurrency; treating every layer as local under-estimates and risks OOM. -/// Pure value type (no MLX) so it is unit-testable with fixed inputs. `public` so the GemmaRuntime -/// conformer can build it from the live model config. -public struct KVCacheCost: Equatable, Sendable { - public var slidingLayers: Int - public var globalLayers: Int - public var slidingWindow: Int // tokens the sliding-window layers retain - public var kvBytesPerTokenPerLayer: Double // a function of head dim × num-kv-heads × kv-quant bits - - public init(slidingLayers: Int, globalLayers: Int, slidingWindow: Int, kvBytesPerTokenPerLayer: Double) { - self.slidingLayers = slidingLayers - self.globalLayers = globalLayers - self.slidingWindow = slidingWindow - self.kvBytesPerTokenPerLayer = kvBytesPerTokenPerLayer - } - - /// The interleaved-attention KV byte sum for a context length: sliding layers retain at most - /// `slidingWindow` tokens each; global layers retain the full context. - public func kvBytes(forContext ctx: Int) -> Int64 { - let perToken = kvBytesPerTokenPerLayer - let slidingTokens = Double(slidingLayers) * Double(min(max(0, ctx), slidingWindow)) - let globalTokens = Double(globalLayers) * Double(max(0, ctx)) - return Int64((slidingTokens + globalTokens) * perToken) - } -} - -/// RAM-is-the-ceiling concurrency math (design D4). Pure: the live free-memory probe happens at the -/// GemmaRuntime boundary and is INJECTED here, so the math is unit-testable without Metal. -public struct ConcurrencyBudget: Equatable, Sendable { - public var unifiedMemoryBytes: Int64 // total unified memory (probed + injected) - public var weightBytes: Int64 // resident weights, read once (shared across all streams) - public var reservedBytes: Int64 // OS + app + graph-activation headroom - public var kv: KVCacheCost - - public init(unifiedMemoryBytes: Int64, weightBytes: Int64, reservedBytes: Int64, kv: KVCacheCost) { - self.unifiedMemoryBytes = unifiedMemoryBytes - self.weightBytes = weightBytes - self.reservedBytes = reservedBytes - self.kv = kv - } - - /// How many concurrent streams fit at this context length. CLAMPED ≥ 1 — the foreground session - /// always fits even if a chosen context is so large only one stream is affordable (then background - /// sessions wait; never an OOM). This is the honest "growing context trades concurrency for length." - public func maxStreams(contextTokens: Int) -> Int { - let perStream = kv.kvBytes(forContext: contextTokens) - guard perStream > 0 else { return 1 } - let free = unifiedMemoryBytes - weightBytes - reservedBytes - guard free > 0 else { return 1 } - return max(1, Int(free / perStream)) - } - - /// Estimated resident RAM at a given (streams, context) — for the Hub cost surface (never silent OOM). - public func estimatedRAM(streams: Int, contextTokens: Int) -> Int64 { - weightBytes + reservedBytes + Int64(max(1, streams)) * kv.kvBytes(forContext: contextTokens) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Agent/ContextBudget.swift b/Sources/ThreeFingerSwitcher/AI/Agent/ContextBudget.swift deleted file mode 100644 index 515e854..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Agent/ContextBudget.swift +++ /dev/null @@ -1,44 +0,0 @@ -import Foundation - -/// The INJECTED context-budget seam (design D6, integration fix C3). The executor's compaction logic -/// depends on THIS protocol, NEVER on the concrete `agentContextTokens` user slider — so this slice -/// (`ai-conversation-runtime`) builds and runs without `ai-batched-runtime-and-context` landing first. -/// -/// The app wires a real provider backed by `ModelDescriptor.maxContextTokens` ∩ the user's -/// `agentContextTokens` (both owned by `ai-batched-runtime-and-context`); tests pass a fixed-budget stub; -/// this slice ships `DefaultContextBudget` as a standalone constant fallback. -public protocol ContextBudgetProviding: Sendable { - /// The maximum number of tokens the assembled context for a turn may occupy. - var maxContextTokens: Int { get } -} - -/// A standalone constant budget so this slice builds + runs before the real provider exists. The default -/// is a conservative Gemma-class context; the real model max is supplied later through this same seam. -public struct DefaultContextBudget: ContextBudgetProviding { - public let maxContextTokens: Int - public init(maxContextTokens: Int = 8192) { - self.maxContextTokens = maxContextTokens - } -} - -/// A pure, deterministic estimate of an `[AgentMessage]`'s token cost. Honest about being an estimate -/// (a character/token ratio, not a real tokenizer) — which is why compaction keeps a safety margin -/// (`ConversationCompactor.marginFraction`) and compacts BELOW the budget rather than at it. Estimates -/// `text` ONLY (never `thinking`), since thinking is never re-fed. -public enum TokenEstimator { - - /// Rough characters-per-token ratio for a Gemma-class tokenizer. Deliberately low (so we - /// over-estimate and compact early) rather than risk an overflow. - static let charsPerToken = 4 - - /// Estimate the token cost of a single string (its committed text). - public static func estimate(_ text: String) -> Int { - guard !text.isEmpty else { return 0 } - return max(1, text.count / charsPerToken) - } - - /// Estimate the token cost of a message list, reading committed `text` only (never `thinking`). - public static func estimate(_ messages: [AgentMessage]) -> Int { - messages.reduce(0) { $0 + estimate($1.text) } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Agent/ConversationCompactor.swift b/Sources/ThreeFingerSwitcher/AI/Agent/ConversationCompactor.swift deleted file mode 100644 index 152ca6f..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Agent/ConversationCompactor.swift +++ /dev/null @@ -1,104 +0,0 @@ -import Foundation - -/// Context compaction (design D6) — this slice OWNS it. A long thread will exceed the model's context -/// window; when the assembled-context estimate approaches the budget, the older turns collapse into a -/// single compact summary (a model call) so the conversation can continue without overflow, and so -/// `.thinking` never bloats the window (the summary input reads committed `text` only). -/// -/// Split into a PURE decision (`needsCompaction`/`plan`/`applied`) and an IMPURE pass (`summarize`, a -/// single `LLMRuntime` call). The decision is deterministic and unit-testable against a fixed-budget -/// `ContextBudgetProviding` stub. - -/// The plan for one compaction: which prefix to collapse, which recent tail to keep verbatim, and the -/// prior summary (if any) that is folded INTO the new summary's input (not kept separately). -public struct CompactionPlan: Equatable, Sendable { - /// The older messages to collapse into the new summary (their committed text is the summary input). - public var toSummarize: [AgentMessage] - /// The most-recent messages kept verbatim. - public var keptTail: [AgentMessage] - /// Any existing summary, folded into the new summary's input so it is not lost. - public var priorSummary: String? - - public init(toSummarize: [AgentMessage], keptTail: [AgentMessage], priorSummary: String?) { - self.toSummarize = toSummarize - self.keptTail = keptTail - self.priorSummary = priorSummary - } - - /// Nothing to collapse (e.g. the thread is already at/below the kept-tail size). - public var isEmpty: Bool { toSummarize.isEmpty } -} - -public enum ConversationCompactor { - - /// Compact BELOW the budget, not at it — the estimator is approximate, so leave headroom. - public static let marginFraction = 0.8 - - /// The number of most-recent messages kept verbatim through a compaction (tuning constant — its - /// final value is a run-verify decision on the M5 build; the windowing logic is independent of it). - public static let keepRecentTurns = 6 - - /// The margin-adjusted threshold at which compaction triggers. - static func threshold(_ maxContextTokens: Int) -> Int { - Int(Double(maxContextTokens) * marginFraction) - } - - /// The estimated assembled-context cost of a conversation: its messages' committed text plus any - /// existing summary prefix (both re-fed; thinking is excluded by construction). - public static func assembledEstimate(_ conversation: AgentConversation) -> Int { - TokenEstimator.estimate(conversation.messages) - + (conversation.compactedSummary.map(TokenEstimator.estimate) ?? 0) - } - - /// PURE: true when the assembled estimate (incl. an existing summary) crosses the margin-adjusted - /// budget read through the injected provider. - public static func needsCompaction(_ conversation: AgentConversation, - budget: ContextBudgetProviding) -> Bool { - assembledEstimate(conversation) > threshold(budget.maxContextTokens) - } - - /// PURE: keep the most-recent `keepRecentTurns` messages verbatim; collapse everything older — and - /// any prior summary — into the to-summarize input. Deterministic and budget-agnostic in its - /// windowing (the budget decides WHETHER to compact, not the window shape). - public static func plan(_ conversation: AgentConversation, - keepRecentTurns: Int = keepRecentTurns) -> CompactionPlan { - let messages = conversation.messages - let keep = min(max(0, keepRecentTurns), messages.count) - let keptTail = Array(messages.suffix(keep)) - let toSummarize = Array(messages.prefix(messages.count - keep)) - return CompactionPlan(toSummarize: toSummarize, - keptTail: keptTail, - priorSummary: conversation.compactedSummary) - } - - /// IMPURE: a single `runtime.generate` call (reasoning OFF) condensing the to-summarize slice — and - /// the prior summary — into a compact factual summary. The summary input reads committed `text` ONLY - /// (never `thinking`). Errors propagate as `RuntimeError` (the caller maps them via `AIError`); a - /// failure here MUST NOT drop history (the caller applies the plan only on success). - public static func summarize(_ plan: CompactionPlan, runtime: LLMRuntime) async throws -> String { - let priorPart = plan.priorSummary.map { "Previous summary:\n\($0)\n\n" } ?? "" - let transcript = plan.toSummarize - .map { "\($0.role.rawValue): \($0.text)" } // committed text only — never `thinking` - .joined(separator: "\n") - let prompt = """ - Condense the following conversation excerpt into a concise factual summary that preserves the key \ - facts, decisions, names, and context needed to continue the conversation. Output only the summary. - - \(priorPart)\(transcript) - """ - return try await runtime.generateText(LLMRequest(prompt: prompt, reasoning: false)) - } - - /// PURE: apply a completed compaction — set the summary as the new prefix and drop the collapsed - /// turns, keeping only the recent tail. Called by the executor ONLY after `summarize` succeeds. - public static func applied(_ plan: CompactionPlan, - summary: String, - to conversation: AgentConversation, - now: Date = Date()) -> AgentConversation { - var updated = conversation - updated.compactedSummary = summary - updated.messages = plan.keptTail - updated.updatedAt = now - return updated - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Agent/RouteSchema.swift b/Sources/ThreeFingerSwitcher/AI/Agent/RouteSchema.swift deleted file mode 100644 index c8b4900..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Agent/RouteSchema.swift +++ /dev/null @@ -1,46 +0,0 @@ -import Foundation - -/// The fixed route schema + the route-prompt builder (design D1). The router uses the EXISTING -/// `structured()` seam against this single `StructuredSchema` — the reliability mechanism is -/// `structured()`'s repair/retry/decline, never native function-call tokens. -enum RouteSchema { - - /// The decision shape the model fills: a chosen `tool` (or "" to answer directly), its - /// `argumentsJSON`, and a one-sentence `rationale`. Only `tool` is required (a plain answer needs - /// nothing else), mirroring `ParsedActions` schema style. - static let schema = StructuredSchema( - name: "tool_route", - json: """ - { - "type": "object", - "required": ["tool"], - "properties": { - "tool": { "type": "string", "description": "the chosen tool name, or \\"\\" to answer directly" }, - "argumentsJSON": { "type": "string", "description": "JSON object of arguments for the chosen tool" }, - "rationale": { "type": "string", "description": "one short sentence: why this choice" } - } - } - """) - - /// Build the route prompt: the conversation tail + the candidate descriptors (name, summary, args - /// schema) + the instruction to choose one OR answer directly. Pure + deterministic. - static func prompt(context: RouteContext, candidates: [ToolDescriptor]) -> String { - let transcript = ChatTemplate.flatten(context.messages) - let toolLines = candidates.map { d in - "- \(d.name): \(d.summary)\n args schema: \(d.argsSchema.json)" - }.joined(separator: "\n") - return """ - You are deciding whether to call a tool to help with the user's request, or to answer directly. - - Conversation so far: - \(transcript) - - Available tools: - \(toolLines) - - Choose EXACTLY ONE tool by its exact name and provide "argumentsJSON" (a JSON object matching that \ - tool's args schema), OR set "tool" to "" to answer the user directly without any tool. Always give \ - a one-sentence "rationale". - """ - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Agent/Subagent.swift b/Sources/ThreeFingerSwitcher/AI/Agent/Subagent.swift deleted file mode 100644 index 437d6d0..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Agent/Subagent.swift +++ /dev/null @@ -1,59 +0,0 @@ -import Foundation - -/// A fixed-pattern context-hygiene primitive (design D7) — NOT concurrency, NOT dynamic spawning. A -/// subagent runs a bounded sub-task in a FRESH `AgentConversation` (empty history, its own system prompt) -/// and returns only a SUMMARY to the orchestrator, so the orchestrator's context never absorbs the -/// sub-task's intermediate turns. This is the honest justification for "subagents" on a single-GPU -/// machine: they are context hygiene (a lean orchestrator), not parallelism. -/// -/// It is a NAMED, bounded template from a small registered set — there is deliberately NO open-ended, -/// model-decided recursive spawning (a small model orchestrates unbounded recursion poorly). A subagent -/// may itself be a routed tool step (`ai-tool-routing`); the loop appends only the returned summary. -struct Subagent: Equatable, Sendable { - let name: String // a named, fixed template (not model-invented) - let systemPrompt: String - let maxTurns: Int // bounded - - init(name: String, systemPrompt: String, maxTurns: Int = 4) { - self.name = name - self.systemPrompt = systemPrompt - self.maxTurns = max(1, maxTurns) - } - - /// Open the subagent's FRESH conversation seeded with its system prompt + the orchestrator's input — - /// a brand-new `AgentSessionID`, empty of the orchestrator's history. (The actual turn loop runs on - /// the injected runtime; this models the isolation contract.) - func freshConversation(input: String, now: Date = Date()) -> AgentConversation { - AgentConversation( - id: AgentSessionID(), - title: name, - messages: [ - AgentMessage(role: .system, text: systemPrompt, createdAt: now), - AgentMessage(role: .user, text: input, createdAt: now), - ], - createdAt: now, updatedAt: now, - skillID: name) - } -} - -/// What a subagent returns to the orchestrator: ONLY a summary (never the raw intermediate turns) plus -/// its session id (so it can ride a batch slot like any other stream — concurrency-cheap + context-cheap). -struct SubagentResult: Equatable, Sendable { - let summary: String - let sessionID: AgentSessionID -} - -extension Subagent { - /// Expose "run subagent " as a routable tool (design D7 / blueprint §3.4): `ai-tool-routing` - /// registers this so the model can invoke the subagent as one routed step, and only the returned - /// summary re-enters the orchestrator's thread as a `.tool` message. Read-only to the orchestrator's - /// world (`.auto`) — the subagent's own steps are gated within its fresh session. - var toolDescriptor: ToolDescriptor { - ToolDescriptor( - name: "subagent:\(name)", - summary: "Run the \(name) sub-task in a fresh context and return only its summary.", - argsSchema: StructuredSchema(name: "subagent_\(name)", json: "{\"type\":\"object\",\"properties\":{\"input\":{\"type\":\"string\"}}}"), - writePolicy: .auto, - keywords: ["subagent", name]) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Agent/SubagentToolContributor.swift b/Sources/ThreeFingerSwitcher/AI/Agent/SubagentToolContributor.swift deleted file mode 100644 index 25a7969..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Agent/SubagentToolContributor.swift +++ /dev/null @@ -1,93 +0,0 @@ -import Foundation - -/// Subagents as routable tools (`refactor-park-and-background-agents` / `ai-subagents`): each REGISTERED -/// `Subagent` template is exposed as a `subagent:` tool the route loop can invoke. Running one -/// opens the template's FRESH conversation (its own system prompt + the routed input — none of the -/// orchestrator's history) and streams ONE prompt-only chat over the injected runtime; ONLY the final -/// text re-enters the orchestrator thread as the step's summary (the context-hygiene contract). The -/// sub-task runs WITHIN the orchestrator's in-flight turn — same task, same cancellation — and offers no -/// tools of its own, so there is structurally no recursion. MLX-free Core (tests drive a stub runtime). -struct SubagentToolContributor: ToolContributor { - let templates: [Subagent] - /// Resolves the runtime per invocation (app-side: `modelManager.runtime(requiring: [.text])`) so the - /// contributor is built once at registry time but always runs on the CURRENT model. - let runtimeProvider: @Sendable () async throws -> LLMRuntime - - init(templates: [Subagent], runtimeProvider: @escaping @Sendable () async throws -> LLMRuntime) { - self.templates = templates - self.runtimeProvider = runtimeProvider - } - - func descriptors() -> [ToolDescriptor] { templates.map(\.toolDescriptor) } - - func canHandle(_ tool: String) -> Bool { - templates.contains { $0.toolDescriptor.name == tool } - } - - func run(_ call: RoutedCall, gate: ApprovalGate) async -> ToolStepResult { - guard let template = templates.first(where: { $0.toolDescriptor.name == call.descriptor.name }) else { - return ToolStepResult(tool: call.descriptor.name, - status: .failed(headline: "That tool isn't available."), - summary: "Unknown subagent: \(call.descriptor.name).") - } - let fresh = template.freshConversation(input: Self.input(from: call)) - do { - let runtime = try await runtimeProvider() - var text = "" - for try await token in runtime.chat(LLMChatRequest(messages: fresh.messages)) { - if Task.isCancelled { return Self.cancelled(call) } - if token.channel == .response { text += token.text } - } - if Task.isCancelled { return Self.cancelled(call) } - let summary = text.trimmingCharacters(in: .whitespacesAndNewlines) - return ToolStepResult(tool: call.descriptor.name, status: .done, - summary: summary.isEmpty ? "(no output)" : summary) - } catch let error as RuntimeError where error == .cancelled { - return Self.cancelled(call) - } catch is CancellationError { - return Self.cancelled(call) - } catch { - // A subagent failure is a clean failed step (one taxonomy, one translator) — never a - // fabricated summary, never raw error text. - return ToolStepResult(tool: call.descriptor.name, - status: .failed(headline: AIError.message(for: error).headline), - summary: "") - } - } - - /// The sub-task input: the routed `{"input": …}` argument, falling back to the orchestrator's - /// latest user text when the model routed with empty arguments. - private static func input(from call: RoutedCall) -> String { - struct Args: Decodable { var input: String? } - let parsed = call.route.argumentsJSON.data(using: .utf8) - .flatMap { try? JSONDecoder().decode(Args.self, from: $0) } - let input = parsed?.input?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" - return input.isEmpty ? call.userText : input - } - - /// A discarded turn cancels the subagent WITH it — the loop's cancel sentinel, never a failure. - private static func cancelled(_ call: RoutedCall) -> ToolStepResult { - ToolStepResult(tool: call.descriptor.name, - status: .declined(reason: TaskKindToolContributor.cancelledReason), - summary: "Cancelled.") - } -} - -extension Subagent { - /// The built-in template set (v1): deliberately small, prompt-only, single-turn. The set is - /// injectable at the contributor, so a skills-derived roster is a later drop-in. - static let builtIns: [Subagent] = [ - Subagent( - name: "summarize", - systemPrompt: "You are a focused summarizer running as an isolated sub-task. Condense the " - + "provided input to its essential points, preserving concrete facts, names, and numbers. " - + "Reply with ONLY the summary — no preamble, no commentary.", - maxTurns: 1), - Subagent( - name: "draft", - systemPrompt: "You are a focused writing assistant running as an isolated sub-task. Compose " - + "the text the brief asks for — complete and ready to use. Reply with ONLY the drafted " - + "text — no preamble, no commentary.", - maxTurns: 1), - ] -} diff --git a/Sources/ThreeFingerSwitcher/AI/Agent/TaskKindToolContributor.swift b/Sources/ThreeFingerSwitcher/AI/Agent/TaskKindToolContributor.swift deleted file mode 100644 index 348562c..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Agent/TaskKindToolContributor.swift +++ /dev/null @@ -1,133 +0,0 @@ -import Foundation - -/// The load-bearing reuse (design D4): a thin adapter that turns the existing `TaskKind`s into routable -/// tools and bridges a routed call back into the UNCHANGED `TaskDispatcher.prepare`/`execute`. The model -/// only chooses the menu item (and pre-fills it); the kind's own `ParsedActions` schema stays the -/// authority — so `TaskDispatcher`/`ParsedActions`/`TaskSinks` are byte-unchanged. -struct TaskKindToolContributor: ToolContributor { - let dispatcher: TaskDispatching - let resolver: WritePolicyResolving - /// The authored task kinds available as tools (with their bound config — never invented by the router). - let kinds: [TaskKind] - - init(dispatcher: TaskDispatching, resolver: WritePolicyResolving = DescriptorWritePolicy(), kinds: [TaskKind]) { - self.dispatcher = dispatcher - self.resolver = resolver - self.kinds = kinds - } - - /// A sentinel decline reason the loop recognizes as "the user cancelled the whole canvas" (not a skip). - static let cancelledReason = "__cancelled__" - - func descriptors() -> [ToolDescriptor] { kinds.map(Self.descriptor(for:)) } - - func canHandle(_ tool: String) -> Bool { kinds.contains { Self.name(for: $0) == tool } } - - func run(_ call: RoutedCall, gate: ApprovalGate) async -> ToolStepResult { - let toolName = call.descriptor.name - guard let kind = kinds.first(where: { Self.name(for: $0) == toolName }) else { - return ToolStepResult(tool: toolName, status: .failed(headline: "That tool isn't available."), - summary: "Unknown tool: \(toolName).") - } - - // Fold the route's argumentsJSON into the prompt the dispatcher consumes — the args are a HINT; - // the kind's ParsedActions schema re-validates them (so the router's JSON never reaches a sink - // unvalidated). Carry the user's content so the parse has its source text. - let resolvedPrompt = Self.fold(userText: call.userText, argumentsJSON: call.route.argumentsJSON) - let review = await dispatcher.prepare(kind, resolvedPrompt: resolvedPrompt, source: call.source, - reasoning: false) - - switch review { - case let .declined(reason): - return ToolStepResult(tool: toolName, status: .declined(reason: reason), summary: reason) - case let .unavailable(reason): - return ToolStepResult(tool: toolName, status: .failed(headline: reason), summary: reason) - case let .action(title, _, _): - switch resolver.effectiveTier(for: call.descriptor) { - case .auto: - return await fire(review, tool: toolName, title: title) - case .confirm, .dangerous: - switch await gate.awaitDecision(for: review) { - case .approve: - return await fire(review, tool: toolName, title: title) - case .skip: - return ToolStepResult(tool: toolName, status: .declined(reason: "skipped"), - summary: "Skipped “\(title)”.") - case .cancel: - return ToolStepResult(tool: toolName, status: .declined(reason: Self.cancelledReason), - summary: "Cancelled.") - } - } - } - } - - /// Execute a reviewed action; a sink that throws becomes `.failed` with a CLEAN headline — never a - /// false "Done" (the executor's existing honesty rule). - private func fire(_ review: TaskReview, tool: String, title: String) async -> ToolStepResult { - do { - try await dispatcher.execute(review) - return ToolStepResult(tool: tool, status: .done, summary: "Done: \(title).") - } catch { - return ToolStepResult(tool: tool, status: .failed(headline: AIError.message(for: error).headline), - summary: "Couldn't complete “\(title)”.") - } - } - - // MARK: - Descriptor identity (Decision Q2: config bound in the name) - - static func fold(userText: String, argumentsJSON: String) -> String { - var parts: [String] = [] - if !userText.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty { parts.append(userText) } - if !argumentsJSON.isEmpty, argumentsJSON != "{}" { - parts.append("Use these arguments where they fit: \(argumentsJSON)") - } - return parts.joined(separator: "\n\n") - } - - static func name(for kind: TaskKind) -> String { - switch kind { - case .addToCalendar: return "add_to_calendar" - case .addToReminder: return "add_to_reminder" - case .newContact: return "new_contact" - case let .saveToProject(project): return "save_to_project:\(project)" - case let .openToolWithPayload(tool): return "open_tool:\(tool)" - case let .sendTo(destination): return "send_to:\(destinationKey(destination))" - } - } - - static func destinationKey(_ destination: Destination) -> String { - switch destination { - case let .shortcut(name): return "shortcut:\(name)" - case let .urlScheme(scheme): return "url:\(scheme)" - case let .shell(command): return "shell:\(command)" - } - } - - static func descriptor(for kind: TaskKind) -> ToolDescriptor { - ToolDescriptor(name: name(for: kind), summary: summary(for: kind), - argsSchema: StructuredSchema(name: name(for: kind), json: "{\"type\":\"object\"}"), - writePolicy: .confirm, keywords: keywords(for: kind)) - } - - static func summary(for kind: TaskKind) -> String { - switch kind { - case .addToCalendar: return "Create a calendar event from the request." - case .addToReminder: return "Create a reminder / to-do from the request." - case .newContact: return "Create a contact card from the request." - case let .saveToProject(project): return "Append the content to the “\(project)” project note." - case let .openToolWithPayload(tool): return "Generate a payload and open “\(tool)” with it." - case let .sendTo(destination): return "Send the content to \(destinationKey(destination))." - } - } - - static func keywords(for kind: TaskKind) -> [String] { - switch kind { - case .addToCalendar: return ["calendar", "event", "meeting", "schedule", "appointment"] - case .addToReminder: return ["reminder", "todo", "task", "remind"] - case .newContact: return ["contact", "person", "card", "address"] - case .saveToProject: return ["save", "note", "project", "append"] - case .openToolWithPayload: return ["open", "tool", "payload"] - case .sendTo: return ["send", "share", "route", "shortcut"] - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Agent/ToolCandidateSource.swift b/Sources/ThreeFingerSwitcher/AI/Agent/ToolCandidateSource.swift deleted file mode 100644 index 2b5f18a..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Agent/ToolCandidateSource.swift +++ /dev/null @@ -1,79 +0,0 @@ -import Foundation - -/// Surfaces ~3–5 candidate tools per route turn — never the full registry (design D6). The shape -/// deliberately mirrors `DocIndex.retrieve(query:limit:)` (blueprint §3.4) so that when -/// `ai-skills-as-files`/`ai-agent-memory` land, a `DocIndex`-backed source ranks skill/memory tools -/// through the SAME retriever — no second ranking path. -protocol ToolCandidateSource: Sendable { - func candidates(for context: RouteContext, limit: Int) -> [ToolDescriptor] -} - -/// The v1 default: a cheap lexical match of the latest user turn against each descriptor's -/// `name`/`summary`/`keywords` (token overlap + keyword substring), top-`limit`, always including the -/// active skill's allowed tools, capped at `additiveCap` total. -struct KeywordToolCandidateSource: ToolCandidateSource { - /// LIVE provider of the full tool set — re-queried EVERY turn (never frozen). A tool gated behind a - /// flag the user toggles mid-session (e.g. Media generation → `generate_image`) appears the instant the - /// flag flips. Snapshotting this once (the old `[ToolDescriptor]` field) silently dropped such tools: - /// the registry's `descriptors()` are live, but freezing them here defeated that. - let all: @Sendable () -> [ToolDescriptor] - let additiveCap: Int - - /// Live source: `all` is re-evaluated on each `candidates(...)` call. - init(all: @escaping @Sendable () -> [ToolDescriptor], additiveCap: Int = 8) { - self.all = all - self.additiveCap = additiveCap - } - - /// Convenience for a STATIC tool set (tests / fixed registries): wraps the array in a constant provider. - init(all: [ToolDescriptor], additiveCap: Int = 8) { - self.init(all: { all }, additiveCap: additiveCap) - } - - func candidates(for context: RouteContext, limit: Int) -> [ToolDescriptor] { - let all = self.all() // LIVE snapshot for THIS turn — honors flags toggled since construction - let queryTokens = Set(Self.tokenize(context.latestUserText)) - let queryLower = context.latestUserText.lowercased() - - func score(_ d: ToolDescriptor) -> Int { - let hay = Set(Self.tokenize(d.name) + Self.tokenize(d.summary) + d.keywords.flatMap(Self.tokenize)) - let overlap = queryTokens.intersection(hay).count - let substringBoost = d.keywords.contains { !$0.isEmpty && queryLower.contains($0.lowercased()) } ? 1 : 0 - return overlap + substringBoost - } - - let allowed = all.filter { context.allowedTools.contains($0.name) } - let ranked = all - .filter { !context.allowedTools.contains($0.name) } - .map { ($0, score($0)) } - .sorted { $0.1 > $1.1 } - .prefix(max(0, limit)) - .map { $0.0 } - - // Ranked first (so the best match leads), allowed tools merged, deduped, capped. - var seen = Set() - var out: [ToolDescriptor] = [] - for d in ranked + allowed where !seen.contains(d.name) { - seen.insert(d.name) - out.append(d) - if out.count >= additiveCap { break } - } - return out - } - - static func tokenize(_ s: String) -> [String] { - s.lowercased().split { !$0.isLetter && !$0.isNumber }.map(String.init).filter { $0.count > 1 } - } -} - -extension ToolDescriptor { - /// The retrieval-as-a-routed-step tool (blueprint §3.4 / Decision 6): the model calls this to ask for - /// a broader candidate set when none of the current candidates fit. The loop widens the next turn's - /// candidates in response. Read-only (`.auto`) — it changes nothing in the world. - static let widenCandidates = ToolDescriptor( - name: "widen_candidates", - summary: "Request a broader set of tools when none of the offered candidates fit the request.", - argsSchema: StructuredSchema(name: "widen_candidates", json: "{\"type\":\"object\"}"), - writePolicy: .auto, - keywords: ["other", "more", "different", "another", "tool"]) -} diff --git a/Sources/ThreeFingerSwitcher/AI/Agent/ToolContracts.swift b/Sources/ThreeFingerSwitcher/AI/Agent/ToolContracts.swift deleted file mode 100644 index 052c34f..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Agent/ToolContracts.swift +++ /dev/null @@ -1,123 +0,0 @@ -import Foundation - -/// The canonical tool-routing contracts (blueprint §3.3 / design D2). This slice (`ai-tool-routing`, -/// Wave 2) OWNS them — it took ownership from the Wave-1 `ToolPlaceholders.swift` (now deleted), adding -/// the `keywords` retrieval signal to `ToolDescriptor`. The five public types are referenced by the -/// Wave-1 `AgentMessage`/`LLMChatRequest`; the routing-internal types (`RoutedCall`/`RouteContext`) -/// reference internal task types and so stay internal. MLX-free Core (`swift test`-verified). -/// -/// Integration fix C1: the bare `WritePolicyTier` enum lives HERE (with `ToolDescriptor`) so every -/// descriptor is self-describing without a DAG back-edge to `ai-background-autonomy` (which owns only -/// the user whitelist, effective-tier resolution, the audit log, and escalation — never the enum). - -/// Write-policy tier for a tool — drives auto / confirm / escalate. -public enum WritePolicyTier: String, Codable, Equatable, Sendable { - /// Whitelisted/safe: runs without confirmation, even when parked (still audited downstream). - case auto - /// Default: needs foreground approval (DOWN=approve / RIGHT=skip). - case confirm - /// Always escalates to the foreground via the needs-you badge, even if parked. - case dangerous -} - -/// A tool the model may call. Describes itself to the router: a stable id, a one-line summary, the -/// arguments schema, its write-policy tier, and cheap retrieval `keywords` (the candidate-source ranking -/// signal, Decision 6). -public struct ToolDescriptor: Codable, Equatable, Sendable { - public let name: String - public let summary: String - public let argsSchema: StructuredSchema - public let writePolicy: WritePolicyTier - public var keywords: [String] - - public init(name: String, summary: String, argsSchema: StructuredSchema, - writePolicy: WritePolicyTier, keywords: [String] = []) { - self.name = name - self.summary = summary - self.argsSchema = argsSchema - self.writePolicy = writePolicy - self.keywords = keywords - } -} - -/// The model's routing decision for one step (produced via `runtime.structured(RouteSchema, as: -/// ToolRoute.self)`). `tool == ""` is a plain text answer (the first-class "just talk" case). Decoding -/// is tolerant: a model that omits `argumentsJSON`/`rationale` decodes cleanly (defaults), so a plain -/// answer never trips the repair loop. -public struct ToolRoute: Codable, Equatable, Sendable { - /// Matches a `ToolDescriptor.name`, or "" for a plain text answer. - public let tool: String - /// JSON object string of arguments; "" when `tool == ""`. - public let argumentsJSON: String - /// One short sentence — why this choice; rides the `.thinking` channel + the audit log. - public let rationale: String? - - /// True when the model chose to answer directly rather than call a tool. - public var isPlainAnswer: Bool { tool.isEmpty } - - public init(tool: String, argumentsJSON: String = "", rationale: String? = nil) { - self.tool = tool - self.argumentsJSON = argumentsJSON - self.rationale = rationale - } - - enum CodingKeys: String, CodingKey { case tool, argumentsJSON, rationale } - - public init(from decoder: Decoder) throws { - let c = try decoder.container(keyedBy: CodingKeys.self) - tool = try c.decodeIfPresent(String.self, forKey: .tool) ?? "" - argumentsJSON = try c.decodeIfPresent(String.self, forKey: .argumentsJSON) ?? "" - rationale = try c.decodeIfPresent(String.self, forKey: .rationale) - } -} - -/// The outcome of executing one routed step. Its `summary` is fed back into the loop as a `.tool` -/// message and is what `ChatTemplate.flatten` renders for a `.tool` turn. -public struct ToolStepResult: Codable, Equatable, Sendable { - public let tool: String - public let status: ToolStepStatus - public let summary: String - - public init(tool: String, status: ToolStepStatus, summary: String) { - self.tool = tool - self.status = status - self.summary = summary - } -} - -/// The status of an executed tool step. `.failed` carries a CLEAN headline only -/// (`AIPresentedError.headline`); raw error text goes to logs, never here. -public enum ToolStepStatus: Codable, Equatable, Sendable { - case done - case awaitingApproval - case declined(reason: String) - case failed(headline: String) -} - -// MARK: - Routing-internal value types (reference internal task types → internal) - -/// One routed call ready to run: the chosen descriptor, the model's route, the latest user text (the -/// content the bridge folds into the dispatcher's resolved prompt), and the fire-time provenance. -struct RoutedCall: Sendable { - var descriptor: ToolDescriptor - var route: ToolRoute - var userText: String - var source: TaskSource -} - -/// The context a route turn / candidate retrieval reads: the conversation tail plus the active skill's -/// allowed tools (always offered as candidates). -struct RouteContext: Sendable { - var messages: [AgentMessage] - var activeSkillID: String? - var allowedTools: [String] - - init(messages: [AgentMessage], activeSkillID: String? = nil, allowedTools: [String] = []) { - self.messages = messages - self.activeSkillID = activeSkillID - self.allowedTools = allowedTools - } - - /// The latest user turn's committed text — the query the candidate source ranks against. - var latestUserText: String { messages.last(where: { $0.role == .user })?.text ?? "" } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Agent/ToolRegistry.swift b/Sources/ThreeFingerSwitcher/AI/Agent/ToolRegistry.swift deleted file mode 100644 index aac5f67..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Agent/ToolRegistry.swift +++ /dev/null @@ -1,62 +0,0 @@ -import Foundation - -/// The user's resolution of an approval pause, fed by the canvas's canonical compass (DOWN=approve / -/// RIGHT=skip), or `.cancel` when the whole canvas is discarded. -enum ApprovalDecision: Equatable, Sendable { - case approve - case skip - case cancel -} - -/// The async seam the canvas drives to resolve a `.confirm`/`.dangerous` tool step (design D5). A -/// contributor that needs approval surfaces the backing `TaskReview` and awaits a decision here. Tests -/// inject a scripted gate. -protocol ApprovalGate: Sendable { - func awaitDecision(for review: TaskReview) async -> ApprovalDecision -} - -/// One source of tools: advertises its descriptors and runs a routed call (design D3). v1 ships a single -/// `TaskKindToolContributor`; later waves add `MemoryToolContributor`, `SkillToolContributor`, -/// `ClaudeHandoffContributor` — each just adds descriptors + a `run`, with NO loop change. -protocol ToolContributor: Sendable { - func descriptors() -> [ToolDescriptor] - func canHandle(_ tool: String) -> Bool - func run(_ call: RoutedCall, gate: ApprovalGate) async -> ToolStepResult -} - -/// Aggregates contributors (design D3): the union of their descriptors (deduped by `name`, first wins) -/// and dispatch of a routed call to the owning contributor. Pure aggregation — no loop coupling. -struct ToolRegistry: Sendable { - private let contributors: [ToolContributor] - - init(_ contributors: [ToolContributor]) { - self.contributors = contributors - } - - func allDescriptors() -> [ToolDescriptor] { - var seen = Set() - var out: [ToolDescriptor] = [] - for contributor in contributors { - for descriptor in contributor.descriptors() where !seen.contains(descriptor.name) { - seen.insert(descriptor.name) - out.append(descriptor) - } - } - return out - } - - func descriptor(named name: String) -> ToolDescriptor? { - allDescriptors().first { $0.name == name } - } - - func run(_ call: RoutedCall, gate: ApprovalGate) async -> ToolStepResult { - guard let contributor = contributors.first(where: { $0.canHandle(call.descriptor.name) }) else { - // Defensive: a routed tool no contributor owns never dispatches — a clean failed step the - // model sees and can re-route around. - return ToolStepResult(tool: call.descriptor.name, - status: .failed(headline: "That tool isn't available."), - summary: "Unknown tool: \(call.descriptor.name).") - } - return await contributor.run(call, gate: gate) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Agent/ToolRouter.swift b/Sources/ThreeFingerSwitcher/AI/Agent/ToolRouter.swift deleted file mode 100644 index 041a374..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Agent/ToolRouter.swift +++ /dev/null @@ -1,52 +0,0 @@ -import Foundation - -/// The result of one route turn. `.route` carries the model's decision (including a plain answer when -/// `tool == ""`); `.failed` carries a clean headline for an unexpected runtime failure; `.cancelled` is -/// a discard (not a failure). -enum RouteResult: Sendable { - case route(ToolRoute) - case failed(headline: String) - case cancelled -} - -/// Maps the `structured()` route turn's outcomes into a `RouteResult` (design D1). The router is the -/// reliability mechanism: a malformed or unknown route degrades to a plain answer (never a fabricated -/// tool call), a decline IS a first-class plain answer, and only a genuine runtime failure is `.failed`. -enum ToolRouter { - - static func route(context: RouteContext, candidates: [ToolDescriptor], - runtime: LLMRuntime, reasoning: Bool) async -> RouteResult { - let prompt = RouteSchema.prompt(context: context, candidates: candidates) - do { - let outcome = try await runtime.structured( - LLMRequest(prompt: prompt, reasoning: reasoning), - schema: RouteSchema.schema, as: ToolRoute.self) - switch outcome { - case let .value(route): - if route.isPlainAnswer { return .route(route) } - if candidates.contains(where: { $0.name == route.tool }) { return .route(route) } - // The model named a tool not in the candidate set (hallucinated) — degrade to a plain - // answer rather than dispatch an unknown tool. The `widen_candidates` tool is the - // legitimate path to more tools. - return .route(ToolRoute(tool: "", rationale: route.rationale)) - case let .declined(reason): - // The model judged that no tool fits — exactly the first-class "just talk" case. - return .route(ToolRoute(tool: "", rationale: reason)) - } - } catch let error as RuntimeError { - switch error { - case .cancelled: - return .cancelled - case .couldNotProduceValid: - // A malformed route is never a fabricated tool call — fall back to a plain answer. - return .route(ToolRoute(tool: "")) - default: - return .failed(headline: AIError.message(for: error).headline) - } - } catch is CancellationError { - return .cancelled - } catch { - return .failed(headline: AIError.message(for: error).headline) - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Agent/WritePolicyResolving.swift b/Sources/ThreeFingerSwitcher/AI/Agent/WritePolicyResolving.swift deleted file mode 100644 index 76bdb62..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Agent/WritePolicyResolving.swift +++ /dev/null @@ -1,16 +0,0 @@ -import Foundation - -/// Resolves a descriptor's effective write-policy tier (design D5). `ai-background-autonomy` (Wave 4) -/// supplies the production conformer that intersects the descriptor's tier with the user whitelist -/// (descriptor default ∩ whitelist → effective tier). This slice ships a stand-alone default so it -/// compiles + tests without that slice. -protocol WritePolicyResolving: Sendable { - func effectiveTier(for descriptor: ToolDescriptor) -> WritePolicyTier -} - -/// The stand-alone default: the descriptor's own tier, unmodified. Replaced by the whitelist-aware -/// resolver in `ai-background-autonomy`. -struct DescriptorWritePolicy: WritePolicyResolving { - init() {} - func effectiveTier(for descriptor: ToolDescriptor) -> WritePolicyTier { descriptor.writePolicy } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Audit/AuditError.swift b/Sources/ThreeFingerSwitcher/AI/Audit/AuditError.swift deleted file mode 100644 index c6b5477..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Audit/AuditError.swift +++ /dev/null @@ -1,35 +0,0 @@ -import Foundation - -/// Audit-store persistence failures (`ai-background-autonomy`, design Decision 7) — a new taxonomy ONLY -/// for the cases `RuntimeError`/`TaskError`/`SkillError`/`MemoryError`/`ParkError` cannot carry: the -/// append-only audit log's durable-store IO. Each case has a clean, user-facing `errorDescription`; raw -/// OS/coding text stays in opt-in details / logs (never the headline). `AIError.message(for:)` (the -/// single translator) is extended to translate this. -/// -/// `FileManager`/JSON-coding throws map into this AT the `DiskAuditLog` IO boundary; Core stays MLX-free. -/// Crucially, `AuditLog.record(_:)` NEVER throws this into the route loop — a persistence failure is -/// observed bounded on the Hub viewer; the in-memory ring is unaffected (auditing must not break the -/// agent). -enum AuditError: Error, Equatable, LocalizedError { - /// The durable audit store directory could not be created / opened (permission, read-only volume). - case storeUnavailable(detail: String) - /// A record could not be appended / the log could not be read back (disk full, permission, corruption). - case persistFailed(detail: String) - - var errorDescription: String? { - switch self { - case .storeUnavailable: - return "The audit log couldn't be opened on disk." - case .persistFailed: - return "The audit log couldn't be saved." - } - } - - /// The copyable raw detail kept OUT of the headline. - var rawDetail: String? { - switch self { - case let .storeUnavailable(detail), let .persistFailed(detail): - return detail - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Audit/AuditLog.swift b/Sources/ThreeFingerSwitcher/AI/Audit/AuditLog.swift deleted file mode 100644 index d89ba9c..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Audit/AuditLog.swift +++ /dev/null @@ -1,220 +0,0 @@ -import Foundation -import os - -private let auditLog = Logger(subsystem: "ThreeFingerSwitcher", category: "AuditLog") - -/// The append-only audit log seam (`ai-background-autonomy`, design Decision 5). `record(_:)` is -/// NON-BLOCKING and infallible from the caller's view — auditing must never break the agent (design -/// rejected-alternative 6). Viewers (the notch rail drop-down + the Hub AI page) read `recent(limit:)` -/// synchronously. MLX-free Core. -public protocol AuditLog: Sendable { - /// Append one record (never throws — a persistence failure is observed on the viewer, not here). - func record(_ r: AuditRecord) - /// The most-recent records, reverse-chronological, capped at `limit`. - func recent(limit: Int) -> [AuditRecord] - /// The ledger's SINGLE removal operation (`notch-conversation-gestures`): erase every record - /// attributed to `sessionID` — the user's explicit purge-delete gesture, and nothing else, invokes - /// it (no agent/tool/background path reaches it, so the model can never erase its own tracks). - /// Non-blocking like a write; a durable-store rewrite failure surfaces on the same bounded channel - /// as a write failure while the in-memory ring stays purged. - func purge(sessionID: AgentSessionID) -} - -/// A bounded in-memory ring (`ai-background-autonomy`). The synchronous read seam every viewer hits; the -/// disk store wraps this for the durable path. Cap-trims the oldest on overflow. -public final class InMemoryAuditLog: AuditLog, @unchecked Sendable { - private let lock = NSLock() - private var ring: [AuditRecord] = [] - /// Retain the most-recent N records. - public let cap: Int - - public init(cap: Int = 500) { - self.cap = max(1, cap) - } - - public func record(_ r: AuditRecord) { - lock.lock(); defer { lock.unlock() } - ring.append(r) - if ring.count > cap { ring.removeFirst(ring.count - cap) } - } - - public func recent(limit: Int) -> [AuditRecord] { - lock.lock(); defer { lock.unlock() } - guard limit > 0 else { return [] } - return Array(ring.suffix(limit).reversed()) - } - - public func purge(sessionID: AgentSessionID) { - lock.lock(); defer { lock.unlock() } - ring.removeAll { $0.sessionID == sessionID } - } - - /// Seed the ring from a persisted slice (newest-last order, as stored). Used by `DiskAuditLog` at init. - func seed(_ records: [AuditRecord]) { - lock.lock(); defer { lock.unlock() } - ring = records.suffix(cap) - } - - /// The current ring in stored (oldest-first) order — the durable store's write source. - func snapshot() -> [AuditRecord] { - lock.lock(); defer { lock.unlock() } - return ring - } -} - -/// The durable, append-only audit log (`ai-background-autonomy`, design Decision 5). A JSON-lines file -/// under Application Support (mirroring `ParkedSessionStore`/`ClipboardStore`), capped + trimmed on write. -/// `record(_:)` appends to the in-memory ring synchronously (so viewers read it immediately) and bridges -/// the disk write OFF-MAIN on a serialized queue — it NEVER throws into the caller. A persistence failure -/// maps to `AuditError` at the IO boundary and is published on `lastPersistError` for a bounded, -/// non-blocking banner on the Hub viewer (the in-memory ring still has the record). -public final class DiskAuditLog: AuditLog, @unchecked Sendable { - private let fileURL: URL - private let ring: InMemoryAuditLog - private let writeQueue = DispatchQueue(label: "ThreeFingerSwitcher.AuditLog.writer") - private let lock = NSLock() - private var _lastPersistError: AuditError? - - private let encoder: JSONEncoder = { - let e = JSONEncoder(); e.outputFormatting = [.sortedKeys]; e.dateEncodingStrategy = .iso8601; return e - }() - private let decoder: JSONDecoder = { - let d = JSONDecoder(); d.dateDecodingStrategy = .iso8601; return d - }() - - public let cap: Int - - public init(fileURL: URL = DiskAuditLog.defaultFileURL(), cap: Int = 500) { - self.fileURL = fileURL - self.cap = max(1, cap) - self.ring = InMemoryAuditLog(cap: self.cap) - loadIntoRing() - } - - /// `~/Library/Application Support/ThreeFingerSwitcher/audit/audit.jsonl`. - public static func defaultFileURL() -> URL { - let base = (try? FileManager.default.url(for: .applicationSupportDirectory, in: .userDomainMask, - appropriateFor: nil, create: true)) - ?? URL(fileURLWithPath: NSTemporaryDirectory()) - return base - .appendingPathComponent("ThreeFingerSwitcher/audit", isDirectory: true) - .appendingPathComponent("audit.jsonl") - } - - /// The last persistence failure, if any — surfaced bounded + non-blocking on the Hub viewer. `nil` - /// once a subsequent write succeeds. Internal because `AuditError` is internal (like `ParkError`); the - /// app-target viewer reaches it within the Core module. - var lastPersistError: AuditError? { - lock.lock(); defer { lock.unlock() } - return _lastPersistError - } - - public func record(_ r: AuditRecord) { - ring.record(r) // synchronous: viewers see it immediately - let snapshot = ring.snapshot() - writeQueue.async { [weak self] in - self?.persist(snapshot) // off-main; never throws to the caller - } - } - - public func recent(limit: Int) -> [AuditRecord] { - ring.recent(limit: limit) - } - - /// Purge one session's records: the ring filters SYNCHRONOUSLY (viewers are instantly consistent), - /// then the durable JSON-lines file is rewritten through the existing atomic `persist` on the - /// off-main writer queue. A rewrite failure surfaces on `lastPersistError` (bounded, non-blocking) - /// exactly like a write failure — and the ring is already purged either way. Deliberately writes - /// no log line referencing the purged session. - public func purge(sessionID: AgentSessionID) { - ring.purge(sessionID: sessionID) - let snapshot = ring.snapshot() - writeQueue.async { [weak self] in - self?.persist(snapshot) - } - } - - // MARK: - IO boundary (maps every throw to AuditError; logged raw) - - /// Read the JSON-lines file into the ring at init. A single corrupt line is skipped (logged); the - /// rest load — a relaunch never fails wholesale because one line went bad. - private func loadIntoRing() { - guard FileManager.default.fileExists(atPath: fileURL.path), - let data = try? Data(contentsOf: fileURL), - let text = String(data: data, encoding: .utf8) else { return } - var records: [AuditRecord] = [] - for line in text.split(separator: "\n", omittingEmptySubsequences: true) { - guard let lineData = line.data(using: .utf8), - let rec = try? decoder.decode(AuditRecord.self, from: lineData) else { - auditLog.error("skipping unreadable audit line") - continue - } - records.append(rec) - } - ring.seed(records) - } - - /// Rewrite the (capped) ring as JSON-lines, atomically. Trim-on-write is implicit (the ring is - /// already capped). Maps every throw to `AuditError` at the boundary; records it for the viewer - /// banner; NEVER rethrows to the caller. - private func persist(_ records: [AuditRecord]) { - do { - try FileManager.default.createDirectory(at: fileURL.deletingLastPathComponent(), - withIntermediateDirectories: true) - } catch { - auditLog.error("audit store dir create failed: \(String(describing: error), privacy: .public)") - setError(.storeUnavailable(detail: String(describing: error))) - return - } - do { - var lines = "" - for rec in records.suffix(cap) { - let data = try encoder.encode(rec) - if let s = String(data: data, encoding: .utf8) { lines += s + "\n" } - } - try lines.data(using: .utf8)?.write(to: fileURL, options: .atomic) - setError(nil) - } catch { - auditLog.error("audit persist failed: \(String(describing: error), privacy: .public)") - setError(.persistFailed(detail: String(describing: error))) - } - } - - private func setError(_ e: AuditError?) { - lock.lock(); _lastPersistError = e; lock.unlock() - } -} - -/// A test/no-disk double whose write path can be forced to "fail" — the failure surfaces on -/// `lastPersistError` (mirroring `DiskAuditLog`) WITHOUT ever throwing into the caller, so a test can pin -/// "failure is observable, never silent, never thrown into the loop." -public final class FailableInMemoryAuditLog: AuditLog, @unchecked Sendable { - private let ring: InMemoryAuditLog - private let lock = NSLock() - private var _lastPersistError: AuditError? - /// When true, every `record` marks a persist failure (but still keeps the in-memory record). - public var failPersist = false - - public init(cap: Int = 500) { self.ring = InMemoryAuditLog(cap: cap) } - - public func record(_ r: AuditRecord) { - ring.record(r) // never lost - lock.lock() - _lastPersistError = failPersist ? .persistFailed(detail: "forced") : nil - lock.unlock() - } - - public func recent(limit: Int) -> [AuditRecord] { ring.recent(limit: limit) } - - public func purge(sessionID: AgentSessionID) { - ring.purge(sessionID: sessionID) // never blocked — the ring is always purged - lock.lock() - _lastPersistError = failPersist ? .persistFailed(detail: "forced") : nil - lock.unlock() - } - - var lastPersistError: AuditError? { - lock.lock(); defer { lock.unlock() } - return _lastPersistError - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Audit/AuditRecord.swift b/Sources/ThreeFingerSwitcher/AI/Audit/AuditRecord.swift deleted file mode 100644 index 779220b..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Audit/AuditRecord.swift +++ /dev/null @@ -1,123 +0,0 @@ -import Foundation - -/// One append-only audit log entry (`ai-background-autonomy`, blueprint §3.7 / design Decision 5). Every -/// agent tool step writes one — auto, confirmed, declined, escalated, or failed — so the ledger is a -/// complete "what did my agents do while I was away." CONSUMES `AgentSessionID` (`ai-conversation-runtime`) -/// + `WritePolicyTier`/`ToolStepStatus` (`ai-tool-routing`) verbatim. MLX-free Core. -public struct AuditRecord: Codable, Equatable, Identifiable, Sendable { - public let id: UUID - /// Attribution (the session that ran the step). - public let sessionID: AgentSessionID - /// The descriptor name. - public let tool: String - /// The EFFECTIVE tier this step ran at (post-resolution), not the descriptor default. - public let policy: WritePolicyTier - /// Redacted / short single-line summary — NEVER raw secrets or full bodies. - public let argumentsSummary: String - /// The outcome (a `.failed` carries a CLEAN headline only — `AIPresentedError.headline`). - public let outcome: ToolStepStatus - /// True if applied while the session was parked (background). - public let wasBackground: Bool - public let timestamp: Date - - public init(id: UUID = UUID(), - sessionID: AgentSessionID, - tool: String, - policy: WritePolicyTier, - argumentsSummary: String, - outcome: ToolStepStatus, - wasBackground: Bool, - timestamp: Date = Date()) { - self.id = id - self.sessionID = sessionID - self.tool = tool - self.policy = policy - self.argumentsSummary = argumentsSummary - self.outcome = outcome - self.wasBackground = wasBackground - self.timestamp = timestamp - } -} - -/// The pure `argumentsSummary` redaction builder (design Decision 5). Produces a bounded, single-line -/// summary safe to persist + show: the path's last component(s), a command name, a middle-truncated -/// content preview — never the full shell line with embedded secrets, never a raw body. Raw args go -/// nowhere near the record (they live only in the os.Logger breadcrumb at the sink boundary). -public enum AuditRedaction { - /// The bounded length of a redacted summary (characters). Past this, the middle is elided with `…`. - public static let maxSummaryLength = 80 - - /// Redact a `PolicyTarget` into a short, safe summary. - public static func summary(for target: PolicyTarget) -> String { - switch target { - case let .path(p): return "→ " + lastComponents(p) - case let .command(c): return "$ " + commandName(c) - case let .both(command, path): return "$ " + commandName(command) + " → " + lastComponents(path) - case .none: return "" - } - } - - /// Redact a free-form arguments string (e.g. a JSON args blob or a content preview): collapse - /// whitespace to single lines, strip obvious secret-looking key=value pairs, and middle-truncate to - /// the bounded length. - public static func summary(forRawArguments raw: String) -> String { - let singleLine = collapseWhitespace(raw) - let scrubbed = scrubSecrets(singleLine) - return middleTruncate(scrubbed, to: maxSummaryLength) - } - - // MARK: - Helpers (pure) - - /// The command name only — `argv[0]`'s last path component, never the full line (which may carry an - /// embedded token / secret as a later argument). - static func commandName(_ command: String) -> String { - let argv0 = collapseWhitespace(command).split(separator: " ").first.map(String.init) ?? command - let name = (argv0 as NSString).lastPathComponent - return middleTruncate(name, to: maxSummaryLength) - } - - /// The last 1–2 path components of a path, so the record says *where* without leaking the full tree. - static func lastComponents(_ path: String) -> String { - let std = Whitelist.standardize(path) - let parts = std.split(separator: "/").map(String.init) - let tail = parts.suffix(2).joined(separator: "/") - let shown = tail.isEmpty ? std : ".../" + tail - return middleTruncate(shown, to: maxSummaryLength) - } - - /// Collapse all runs of whitespace (incl. newlines) to a single space, trimmed. - static func collapseWhitespace(_ s: String) -> String { - s.split(whereSeparator: { $0.isWhitespace }).joined(separator: " ") - } - - /// Strip obvious secret-looking `key=value`/`key: value` pairs where the key smells sensitive, - /// replacing the value with `***`. A best-effort scrub for the summary — the real defense is that we - /// never emit raw bodies, only short summaries. - static func scrubSecrets(_ s: String) -> String { - let sensitive = ["token", "secret", "password", "passwd", "apikey", "api_key", "key", "auth", "bearer"] - var out = s - for word in sensitive { - // key=VALUE / key: VALUE / --key VALUE (VALUE = a run of non-space chars) - let patterns = ["\(word)=\\S+", "\(word):\\s*\\S+", "--\(word)\\s+\\S+", "\(word)\\s+[A-Za-z0-9._\\-]{8,}"] - for pat in patterns { - if let re = try? NSRegularExpression(pattern: pat, options: [.caseInsensitive]) { - let range = NSRange(out.startIndex..., in: out) - out = re.stringByReplacingMatches(in: out, options: [], range: range, withTemplate: "\(word)=***") - } - } - } - return out - } - - /// Middle-truncate with `…` so both the start and the end remain legible. - static func middleTruncate(_ s: String, to limit: Int) -> String { - guard s.count > limit else { return s } - guard limit > 1 else { return "…" } - let keep = limit - 1 - let head = keep - keep / 2 - let tail = keep / 2 - let start = s.prefix(head) - let end = s.suffix(tail) - return start + "…" + end - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Audit/BackgroundGate.swift b/Sources/ThreeFingerSwitcher/AI/Audit/BackgroundGate.swift deleted file mode 100644 index 3a84d8d..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Audit/BackgroundGate.swift +++ /dev/null @@ -1,63 +0,0 @@ -import Foundation - -/// The per-step auto-vs-escalate decision (`ai-background-autonomy`, design Decision 6). Pure: a function -/// of the step's EFFECTIVE tier and the session's `ParkState` (CONSUMED verbatim from -/// `ai-parked-sessions`). The wiring (App/integration) computes the effective tier (`BackgroundPolicyResolver`), -/// calls `BackgroundGate.decide`, then runs/escalates/waits and records an `AuditRecord`. MLX-free Core. -public enum BackgroundDecision: Equatable, Sendable { - /// Run now, in the background, then audit. - case auto - /// `.confirm` while parked: stay parked, resolve on restore via the routing approval gate - /// (DOWN=approve / RIGHT=skip) — no escalation, no glow. - case waitParked - /// `.dangerous` while parked: → `ParkScheduler.escalate` → `.needsYou` + badge + notch glow. - case escalate(reason: String) - /// The session is active (foreground): the existing routing approval gate owns it (this slice no-ops). - case foreground -} - -/// The pure auto-vs-escalate brain. -public enum BackgroundGate { - - /// The decision table (design Decision 6): - /// - /// | effectiveTier | parkState | → BackgroundDecision | - /// |---|---|---| - /// | `.auto` | `.parked`/`.idle`/`.active` | `.auto` | - /// | `.confirm` | `.parked`/`.idle` | `.waitParked` | - /// | `.confirm` | `.active` | `.foreground` | - /// | `.dangerous` | `.parked`/`.idle` | `.escalate(reason)` | - /// | `.dangerous` | `.active` | `.foreground` (already in front; the gate owns it, no glow) | - /// | any | `.needsYou` | `.waitParked` (already escalated; do not double-escalate) | - public static func decide(effectiveTier: WritePolicyTier, - parkState: ParkState, - tool: String = "") -> BackgroundDecision { - // Already escalated → never double-escalate; the step is recorded awaiting approval and waits. - if parkState == .needsYou { return .waitParked } - - switch effectiveTier { - case .auto: - // A foreground auto-run still runs + audits; the background path is the point. - return .auto - case .confirm: - switch parkState { - case .active: return .foreground - case .parked, .idle: return .waitParked - case .needsYou: return .waitParked // handled above; exhaustive for the compiler - } - case .dangerous: - switch parkState { - case .active: return .foreground // already in front; the canvas gate owns it - case .parked, .idle: return .escalate(reason: escalationReason(tool: tool)) - case .needsYou: return .waitParked // handled above - } - } - } - - /// A clean, headline-grade one-liner for the needs-you reason (never raw error text). The glow itself - /// is rendered by `ai-parked-sessions`. - public static func escalationReason(tool: String) -> String { - let name = tool.isEmpty ? "An action" : "\u{201C}\(tool)\u{201D}" - return "\(name) needs your approval before it can run." - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Audit/BackgroundToolRunner.swift b/Sources/ThreeFingerSwitcher/AI/Audit/BackgroundToolRunner.swift deleted file mode 100644 index 2aa75bc..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Audit/BackgroundToolRunner.swift +++ /dev/null @@ -1,81 +0,0 @@ -import Foundation - -/// Wraps a routed tool step with the background-autonomy policy (`ai-background-autonomy`, design -/// Decision 5/6 — tasks 6.1–6.4). For each step it: resolves the EFFECTIVE (whitelist-aware) tier via -/// `BackgroundPolicyResolver`, decides auto/foreground/waitParked/escalate by the session's `ParkState` -/// via `BackgroundGate`, runs / escalates / waits accordingly, and records an `AuditRecord` for EVERY -/// step. This is the App/integration seam `BackgroundGate` references; `AgentLoop` routes each tool step -/// through it when a session may run in the background (nil = the plain foreground path). MLX-free Core. -struct BackgroundToolRunner: Sendable { - let resolver: BackgroundPolicyResolver - let audit: AuditLog - /// The parked-sessions escalation seam (`.dangerous` while parked → `.needsYou` + badge). Routed - /// through the CONTROLLER (`ParkController.escalate` — persist + repaint), never a raw scheduler - /// mutation, so a background needs-you survives relaunch and repaints the rail - /// (`refactor-park-and-background-agents`). nil in a pure foreground context. Called OFF the main - /// actor — the injector hops as needed. - let onEscalate: (@Sendable (AgentSessionID, String) -> Void)? - /// The session's current park state (keyed lookup; `.active` foreground by default). - let parkStateOf: @Sendable (AgentSessionID) -> ParkState - /// The routed call's whitelist target (per-tool; `.none` by default — calendar/contacts are never - /// whitelist-lowerable). - let targetOf: @Sendable (RoutedCall) -> PolicyTarget - - init(resolver: BackgroundPolicyResolver, audit: AuditLog, - onEscalate: (@Sendable (AgentSessionID, String) -> Void)? = nil, - parkStateOf: @escaping @Sendable (AgentSessionID) -> ParkState = { _ in .active }, - targetOf: @escaping @Sendable (RoutedCall) -> PolicyTarget = { _ in .none }) { - self.resolver = resolver - self.audit = audit - self.onEscalate = onEscalate - self.parkStateOf = parkStateOf - self.targetOf = targetOf - } - - /// Run one routed step under the policy, recording an audit row regardless of outcome (6.3/6.4). - func run(_ call: RoutedCall, sessionID: AgentSessionID, - registry: ToolRegistry, gate: ApprovalGate) async -> ToolStepResult { - let target = targetOf(call) - let tier = resolver.effectiveTier(for: call.descriptor, target: target) - let parkState = parkStateOf(sessionID) - let decision = BackgroundGate.decide(effectiveTier: tier, parkState: parkState, tool: call.descriptor.name) - - let result: ToolStepResult - switch decision { - case .auto: - // Policy already cleared it — run without prompting. The auto-approving gate bridges a - // contributor that still resolves `.confirm` on the descriptor-only fast path. - result = await registry.run(call, gate: AutoApproveGate()) - case .foreground: - result = await registry.run(call, gate: gate) // the canvas approval gate owns confirm here - case .waitParked: - // The loop PAUSES on this status (`.pausedAwaitingUser`) — the step is genuinely pending, - // resolved when the user brings the session back; never silently skipped-and-continued. - result = ToolStepResult(tool: call.descriptor.name, status: .awaitingApproval, - summary: "Waiting for your approval.") - case let .escalate(reason): - onEscalate?(sessionID, reason) - result = ToolStepResult(tool: call.descriptor.name, status: .awaitingApproval, summary: reason) - } - - // EVERY step is audited (6.4): the EFFECTIVE tier, a redacted args summary, the outcome, and the - // background flag. record(_:) is non-blocking + infallible — auditing never breaks the agent. - let targetSummary = AuditRedaction.summary(for: target) - audit.record(AuditRecord( - sessionID: sessionID, - tool: call.descriptor.name, - policy: tier, - argumentsSummary: targetSummary.isEmpty - ? AuditRedaction.summary(forRawArguments: call.route.argumentsJSON) - : targetSummary, - outcome: result.status, - wasBackground: parkState != .active)) - return result - } -} - -/// An approval gate that auto-approves — used by `BackgroundToolRunner` for an `.auto`-tier background -/// step the policy has already cleared (so a contributor never prompts for an already-authorized run). -struct AutoApproveGate: ApprovalGate { - func awaitDecision(for review: TaskReview) async -> ApprovalDecision { .approve } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Audit/BlastRadius.swift b/Sources/ThreeFingerSwitcher/AI/Audit/BlastRadius.swift deleted file mode 100644 index a8bd47c..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Audit/BlastRadius.swift +++ /dev/null @@ -1,58 +0,0 @@ -import Foundation - -/// The blast-radius classification of a tool/sink (`ai-background-autonomy`, Wave 4, design Decision 2). -/// This slice OWNS only resolution / whitelist / audit / escalation — `WritePolicyTier` and -/// `ToolDescriptor` are OWNED by `ai-tool-routing` (integration fix C1) and CONSUMED here verbatim -/// (never redefined). MLX-free Core (`swift test`-verified). -/// -/// The three blast-radius tiers are the *conceptual* classification; they collapse onto the existing -/// `WritePolicyTier` (`auto`/`confirm`/`dangerous`) so no new descriptor enum is born: -/// - `.contained` — the app's own stores (agent memory, project notes) + read-only retrieval; the -/// descriptor ships `.auto` and runs in the background even when parked (still audited). -/// - `.external` — touches the user's wider world but bounded (calendar / contacts / launch / send); -/// the descriptor ships `.confirm`, lowerable to `.auto` only by a whitelist match. -/// - `.dangerous` — delete / overwrite-existing / arbitrary-shell / off-list / Claude-handoff cost; the -/// descriptor ships `.dangerous` and is foreground-only (never lowered by the whitelist). -public enum BlastRadius: Equatable, Sendable { - /// The app's own stores; descriptor ships `.auto`. - case contained - /// Touches the user's world but bounded (calendar / contacts / launch / send). - case external - /// Delete / overwrite-existing / arbitrary-shell / off-list / handoff-cost. - case dangerous - - /// The stable CONTAINED name-prefix set OWNED here (design Decision 2) so the policy layer recognizes - /// the app's own stores WITHOUT importing the memory slice. A descriptor whose name begins with one of - /// these AND that ships `.auto` is `.contained`. Read-only retrieval (`retrieve`/`widen_candidates`) - /// is contained-by-read. - public static let containedNamePrefixes: [String] = [ - "memory.", - "save_to_project", - "retrieve", - "widen_candidates", - ] - - /// Whether a tool name names one of the app's own (CONTAINED) stores / read-only retrieval. - public static func isContainedName(_ name: String) -> Bool { - containedNamePrefixes.contains { name.hasPrefix($0) } - } - - /// The pure descriptor → blast-radius classifier (design Decision 2). Reads the descriptor's stable - /// `name` + `writePolicy`: - /// - `.dangerous` → `.dangerous` (unconditional; the name is ignored — danger is intrinsic). - /// - a CONTAINED name → `.contained`. - /// - otherwise → `.external`. - /// - /// Defensive: a `.auto` descriptor whose name is NOT in the CONTAINED set is treated as `.external` - /// (a non-contained tool should never ship `.auto`) and flagged in debug — the CONTAINED set is the - /// authority, an accidental `.auto` is never silently trusted. - public static func of(_ descriptor: ToolDescriptor) -> BlastRadius { - if descriptor.writePolicy == .dangerous { return .dangerous } - if isContainedName(descriptor.name) { return .contained } - if descriptor.writePolicy == .auto { - assertionFailure("Non-contained tool '\(descriptor.name)' ships .auto — treated as .external (CONTAINED set is the authority).") - return .external - } - return .external - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Audit/Whitelist.swift b/Sources/ThreeFingerSwitcher/AI/Audit/Whitelist.swift deleted file mode 100644 index aa5252a..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Audit/Whitelist.swift +++ /dev/null @@ -1,101 +0,0 @@ -import Foundation - -/// The user-visible, user-editable trust boundary (`ai-background-autonomy`, design Decision 3). A pure -/// value type + pure matching rules, persisted in `AppSettings`. Default-empty for arbitrary entries — a -/// fresh install trusts nothing on the wider filesystem; the app's own memory + project stores are -/// CONTAINED (`BlastRadius`), so they are auto WITHOUT any whitelist row. MLX-free Core. -public struct Whitelist: Codable, Equatable, Sendable { - /// Standardized absolute path prefixes the user trusts for writes (component-boundary match). - public var trustedPathPrefixes: [String] - /// Glob patterns (`*`/`?`, anchored full-string) matched against a command / tool / Shortcut name or - /// a shell command's `argv[0]`. - public var trustedCommandPatterns: [String] - - public init(trustedPathPrefixes: [String] = [], trustedCommandPatterns: [String] = []) { - self.trustedPathPrefixes = trustedPathPrefixes - self.trustedCommandPatterns = trustedCommandPatterns - } - - /// The default: trusts nothing arbitrary. - public static let empty = Whitelist(trustedPathPrefixes: [], trustedCommandPatterns: []) - - // MARK: - Matching (pure, unit-tested) - - /// A **path target** matches iff its standardized absolute path has one of `trustedPathPrefixes` as a - /// **path-component prefix** (design Decision 3): `/Users/me/Notes` matches `/Users/me/Notes/x.md` - /// but NOT `/Users/me/Notes2` (component boundary, never a bare string prefix). `..`/symlinks are - /// resolved (`standardizedFileURL`/`resolvingSymlinksInPath`) BEFORE matching, so - /// `/trusted/../etc/passwd` cannot sneak past `/trusted`. - public func matchesPath(_ rawPath: String) -> Bool { - let target = Whitelist.standardize(rawPath) - guard !target.isEmpty else { return false } - for prefix in trustedPathPrefixes { - let std = Whitelist.standardize(prefix) - guard !std.isEmpty else { continue } - if Whitelist.isComponentPrefix(std, of: target) { return true } - } - return false - } - - /// A **command target** matches iff it matches one of `trustedCommandPatterns` as an anchored - /// `fnmatch`-style glob (`*`/`?`), full-string (design Decision 3): `git*` matches `git`, not - /// `forgit`. - public func matchesCommand(_ command: String) -> Bool { - trustedCommandPatterns.contains { Whitelist.globMatch(pattern: $0, name: command) } - } - - /// The **both-rule** (design Decision 3): a target that is BOTH a command and a path must match a - /// command pattern AND a path prefix — the stricter wins. A whitelisted command aimed at an off-list - /// path does NOT match. - public func matchesBoth(command: String, path: String) -> Bool { - matchesCommand(command) && matchesPath(path) - } - - // MARK: - Pure helpers - - /// Standardize a path: expand `~`, resolve symlinks + `..`, and drop a trailing slash. An absolute - /// path standardizes in place; a relative one resolves against the working dir (rare for a write - /// target, but never crashes). - static func standardize(_ raw: String) -> String { - let expanded = (raw as NSString).expandingTildeInPath - let url = URL(fileURLWithPath: expanded) - .standardizedFileURL - .resolvingSymlinksInPath() - var path = url.path - // `resolvingSymlinksInPath` on macOS prefixes `/private` for some temp dirs; keep it — both the - // prefix and the target standardize the same way, so a consistent representation is what matters. - if path.count > 1, path.hasSuffix("/") { path.removeLast() } - return path - } - - /// True iff `prefix` is `target` or an ancestor directory of `target` at a path-component boundary. - static func isComponentPrefix(_ prefix: String, of target: String) -> Bool { - if prefix == target { return true } - // Component boundary: `target` must continue with a `/` right after the full prefix. - let boundary = prefix.hasSuffix("/") ? prefix : prefix + "/" - return target.hasPrefix(boundary) - } - - /// Anchored `fnmatch`-style glob (`*` = any run, `?` = one char), full-string. A small pure matcher - /// (no `fnmatch(3)` dependency / locale surprises) so it is deterministic and testable. - static func globMatch(pattern: String, name: String) -> Bool { - let p = Array(pattern) - let s = Array(name) - // Classic DP / two-pointer glob with backtracking. - var pi = 0, si = 0 - var star = -1, mark = 0 - while si < s.count { - if pi < p.count, p[pi] == "?" || p[pi] == s[si] { - pi += 1; si += 1 - } else if pi < p.count, p[pi] == "*" { - star = pi; mark = si; pi += 1 - } else if star != -1 { - pi = star + 1; mark += 1; si = mark - } else { - return false - } - } - while pi < p.count, p[pi] == "*" { pi += 1 } - return pi == p.count - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Audit/WritePolicyResolution.swift b/Sources/ThreeFingerSwitcher/AI/Audit/WritePolicyResolution.swift deleted file mode 100644 index c4d7417..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Audit/WritePolicyResolution.swift +++ /dev/null @@ -1,71 +0,0 @@ -import Foundation - -/// The argument target a routed call exposes for whitelist matching (design Decision 4). The per-tool -/// arg → target extraction lives next to each `ToolContributor` (it owns the args shape); this slice -/// supplies the resolver + the matching, not the per-tool parsing. -public enum PolicyTarget: Equatable, Sendable { - /// A write destination path. - case path(String) - /// A tool / Shortcut / shell command name (or `argv[0]`). - case command(String) - /// A shell command writing to a path (the both-rule applies). - case both(command: String, path: String) - /// No path/command (calendar / contacts) → never whitelist-lowerable. - case none -} - -/// The production `WritePolicyResolving` conformer (`ai-background-autonomy`, design Decision 4) that -/// REPLACES the routing slice's stand-alone `DescriptorWritePolicy` default. It intersects the -/// descriptor's tier with the user `Whitelist`: effective tier = `descriptor.writePolicy` lowered to -/// `.auto` ONLY when the tool is CONTAINED or its target matches the whitelist; NEVER lowered when the -/// descriptor is `.dangerous`. -/// -/// `WritePolicyResolving` (CONSUMED verbatim from `ai-tool-routing`) carries only the descriptor, so the -/// protocol method is the descriptor-only fast path. The richer `effectiveTier(for:target:)` overload — -/// which the contributor/loop calls when it has the routed call's target — adds the whitelist lowering -/// WITHOUT a routing protocol change (design rejected-alternative 2). MLX-free Core. -struct BackgroundPolicyResolver: WritePolicyResolving, Sendable { - let whitelist: Whitelist - - init(whitelist: Whitelist = .empty) { - self.whitelist = whitelist - } - - /// Protocol requirement (descriptor-only): used when no target is available. - /// CONTAINED → `.auto`; `.dangerous` → `.dangerous`; otherwise the descriptor's own tier (no - /// whitelist lowering possible without a target). - func effectiveTier(for descriptor: ToolDescriptor) -> WritePolicyTier { - switch BlastRadius.of(descriptor) { - case .contained: return .auto - case .dangerous: return .dangerous - case .external: return descriptor.writePolicy - } - } - - /// The richer overload: lower `.confirm` → `.auto` ONLY when the blast radius is `.external` AND the - /// `target` matches the whitelist. NEVER lowers `.dangerous`; a `.none` target is never lowerable; - /// CONTAINED stays `.auto`. - func effectiveTier(for descriptor: ToolDescriptor, target: PolicyTarget?) -> WritePolicyTier { - let radius = BlastRadius.of(descriptor) - switch radius { - case .contained: - return .auto - case .dangerous: - return .dangerous // never lowered, regardless of target / whitelist - case .external: - guard descriptor.writePolicy == .confirm else { return descriptor.writePolicy } - if let target, matches(target) { return .auto } - return .confirm - } - } - - /// Whether a `PolicyTarget` matches the whitelist (the both-rule for a `.both`). - private func matches(_ target: PolicyTarget) -> Bool { - switch target { - case let .path(p): return whitelist.matchesPath(p) - case let .command(c): return whitelist.matchesCommand(c) - case let .both(command, path): return whitelist.matchesBoth(command: command, path: path) - case .none: return false - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Ax/AXActionError.swift b/Sources/ThreeFingerSwitcher/AI/Ax/AXActionError.swift deleted file mode 100644 index 588318a..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Ax/AXActionError.swift +++ /dev/null @@ -1,56 +0,0 @@ -import Foundation - -/// The computer-use tool layer's error taxonomy (`add-voice-computer-use-agent`, spec "Computer-use -/// failures join the single error taxonomy"). Parallel to `FileActionError`: a Core `LocalizedError` -/// with clean per-case headlines, populated at the `AXUIElement` boundary. Raw AX/OS error text rides -/// ONLY in `copyableDetails`; surfaces are bounded + non-blocking; `AIError.message(for:)` translates. -public enum AXActionError: Error, Equatable { - /// The Accessibility grant is missing or was revoked mid-session. - case notPermitted - /// The target app's AX server didn't answer in time (beach-balling app). The step timeout also - /// covers this; the boundary maps an explicit AX timeout here so the headline is honest. - case appNotResponding(appName: String) - /// The target window disappeared between snapshot and act. - case windowGone - /// The element ID doesn't resolve against the CURRENT snapshot epoch — the window changed since - /// the last read. The loop's recovery is a fresh `read_window` (constrained-ID rule: an act can - /// only target what currently exists; a stale target NEVER degrades to a guess). - case staleElement - /// The element exists but does not support the requested action (not pressable / not settable). - case elementNotActionable(role: String) - /// The act was posted but the re-read could not observe the expected change (verify-after-act): - /// an unverified side effect is a failure, never a false "Done". - case verifyFailed(expectation: String) - /// The window exposes no usable accessibility content (an AX desert — Electron/canvas surfaces). - case unreadableWindow(appName: String) - /// Raw diagnostic for the opt-in disclosure, populated at the boundary where available. - public var copyableDetails: String? { - switch self { - case let .verifyFailed(expectation): return "Expected: \(expectation)" - case let .appNotResponding(app): return "AX request to \(app) timed out" - case .notPermitted, .windowGone, .staleElement, .elementNotActionable, .unreadableWindow: - return nil - } - } -} - -extension AXActionError: LocalizedError { - public var errorDescription: String? { - switch self { - case .notPermitted: - return "Accessibility access is off. Allow it in System Settings so the assistant can read and act on windows." - case let .appNotResponding(appName): - return "\(appName) isn't responding." - case .windowGone: - return "That window is gone." - case .staleElement: - return "The window changed — it needs to be read again." - case let .elementNotActionable(role): - return "That element (\(role)) can't be acted on." - case .verifyFailed: - return "The action didn't take effect." - case let .unreadableWindow(appName): - return "Can't read this \(appName) window." - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Ax/AXActionPerformer.swift b/Sources/ThreeFingerSwitcher/AI/Ax/AXActionPerformer.swift deleted file mode 100644 index 747c9bf..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Ax/AXActionPerformer.swift +++ /dev/null @@ -1,189 +0,0 @@ -import Foundation -import AppKit -import ApplicationServices - -/// The read service + act primitives (`add-voice-computer-use-agent`, design D5): `snapshot` reads a -/// window through the boundary reader into the pure builder and registers it with the epoch store; -/// `press`/`typeText` resolve a CONSTRAINED element ID against the current epoch, act, and -/// **verify-after-act** — an act whose expected change is not observable throws `verifyFailed` -/// (never a false "Done"). All OS failures cross as `AXActionError`. -@MainActor -public final class AXActionPerformer { - - /// The verification outcome carried in a step summary. - public struct Verification: Equatable, Sendable { - public var verified: Bool - public var detail: String - } - - private let reader: AXWindowReader - private let store: AXSnapshotStore - /// The tagged source for synthetic keystrokes (`AgentActionArbiter.eventSource`). nil = tests / - /// no-posting environments (the AXValue fast path still works). - private let eventSource: CGEventSource? - - public init(reader: AXWindowReader, store: AXSnapshotStore, eventSource: CGEventSource?) { - self.reader = reader - self.store = store - self.eventSource = eventSource - } - - /// Convenience with fresh collaborators (the coordinator passes the arbiter's tagged source). - public convenience init(eventSource: CGEventSource? = nil) { - self.init(reader: AXWindowReader(), store: AXSnapshotStore(), eventSource: eventSource) - } - - public var snapshotStore: AXSnapshotStore { store } - - // MARK: - Read - - /// Read + register a fresh snapshot for the target window (the `read_window` tool's engine). - public func snapshot(pid: pid_t, titleHint: String?) async throws -> AXWindowSnapshot { - let (root, title, appName) = try await reader.readWindow(pid: pid, titleHint: titleHint) - let built = AXSnapshotBuilder.build(pid: pid, appName: appName, title: title, - root: root, epoch: 0) - let registered = store.register(built) - // An AX desert is an HONEST failure, not an empty success (spec: "An AX desert is honest"). - if registered.textBlocks.isEmpty && registered.elements.isEmpty { - throw AXActionError.unreadableWindow(appName: appName) - } - return registered - } - - // MARK: - Acts (constrained IDs + verify-after-act) - - /// Press an enumerated element. Verify: the window's content changed, a new window/sheet - /// appeared, or the frontmost app changed — SOME observable consequence; none → `verifyFailed`. - public func press(pid: pid_t, titleHint: String?, elementID: String) async throws -> Verification { - let ref = try store.resolve(elementID, pid: pid) - guard ref.isPressable else { throw AXActionError.elementNotActionable(role: ref.role) } - let preHash = store.current(for: pid)?.contentHash ?? "" - let preFront = NSWorkspace.shared.frontmostApplication?.processIdentifier - - let element = try await reader.resolveElement(pid: pid, titleHint: titleHint, path: ref.path) - try await performOffMain(element: element, appPID: pid) { el in - AXUIElementPerformAction(el, kAXPressAction as CFString) - } - - // Settle briefly, then re-read: the post-act snapshot ALSO advances the epoch, so pre-act - // element IDs go stale by construction (the window changed; the model must re-read). - try? await Task.sleep(nanoseconds: 250_000_000) - let post = try? await snapshot(pid: pid, titleHint: titleHint) - let postFront = NSWorkspace.shared.frontmostApplication?.processIdentifier - - if let post, post.contentHash != preHash { - return Verification(verified: true, detail: "the window updated") - } - if postFront != preFront { - return Verification(verified: true, detail: "focus moved to another app/window") - } - throw AXActionError.verifyFailed(expectation: "pressing “\(ref.label.isEmpty ? ref.role : ref.label)” changes the window") - } - - /// Type into an element (or the window's focused element when `elementID` is nil). Prefers the - /// `AXValue` fast path for settable fields (verifiable exactly); falls back to tagged synthetic - /// keystrokes for terminal-like surfaces. `submit` appends Return. - public func typeText(pid: pid_t, titleHint: String?, elementID: String?, - text: String, submit: Bool) async throws -> Verification { - if let elementID { - let ref = try store.resolve(elementID, pid: pid) - let element = try await reader.resolveElement(pid: pid, titleHint: titleHint, path: ref.path) - // Focus the target first so a keystroke fallback (and the user's mental model) agree - // about where the text lands. - _ = try? await performOffMain(element: element, appPID: pid) { el in - AXUIElementSetAttributeValue(el, kAXFocusedAttribute as CFString, kCFBooleanTrue) - } - if ref.isSettable { - let existing = try await currentValue(of: element, appPID: pid) ?? "" - let newValue = existing + text - try await performOffMain(element: element, appPID: pid) { el in - AXUIElementSetAttributeValue(el, kAXValueAttribute as CFString, newValue as CFString) - } - let readBack = try await currentValue(of: element, appPID: pid) ?? "" - guard readBack.contains(text) else { - throw AXActionError.verifyFailed(expectation: "the field contains the typed text") - } - if submit { postKeystroke(keyCode: 36) } // Return - _ = try? await snapshot(pid: pid, titleHint: titleHint) // advance the epoch honestly - return Verification(verified: true, detail: "field now contains the text") - } - } - // Keystroke path (no element / not settable): synthetic tagged unicode events. - guard eventSource != nil else { - throw AXActionError.elementNotActionable(role: "keyboard") - } - let preHash = store.current(for: pid)?.contentHash - postUnicode(text) - if submit { postKeystroke(keyCode: 36) } - try? await Task.sleep(nanoseconds: 250_000_000) - let post = try? await snapshot(pid: pid, titleHint: titleHint) - if let preHash, let post, post.contentHash == preHash { - throw AXActionError.verifyFailed(expectation: "typing changes the window content") - } - return Verification(verified: true, detail: "typed \(text.count) characters") - } - - // MARK: - Internals - - /// Run one AX call off-main with the boundary error map. - @discardableResult - private func performOffMain(element: AXUIElement, appPID: pid_t, - _ call: @escaping @Sendable (AXUIElement) -> AXError) async throws -> AXError { - let appName = NSRunningApplication(processIdentifier: appPID)?.localizedName ?? "App" - let result = await Task.detached(priority: .userInitiated) { call(element) }.value - switch result { - case .success: return result - case .cannotComplete: throw AXActionError.appNotResponding(appName: appName) - case .apiDisabled, .notImplemented: throw AXActionError.notPermitted - case .invalidUIElement: throw AXActionError.staleElement - case .actionUnsupported, .attributeUnsupported: - throw AXActionError.elementNotActionable(role: "element") - default: throw AXActionError.staleElement - } - } - - private func currentValue(of element: AXUIElement, appPID: pid_t) async throws -> String? { - try await Task.detached(priority: .userInitiated) { - var value: CFTypeRef? - guard AXUIElementCopyAttributeValue(element, kAXValueAttribute as CFString, &value) == .success else { - return nil - } - return value as? String - }.value - } - - /// Post `text` as tagged synthetic unicode keyboard events (chunked — CGEvent's unicode payload - /// is bounded). - private func postUnicode(_ text: String) { - guard let source = eventSource else { return } - for chunk in text.chunked(into: 16) { - let utf16 = Array(chunk.utf16) - if let down = CGEvent(keyboardEventSource: source, virtualKey: 0, keyDown: true) { - down.keyboardSetUnicodeString(stringLength: utf16.count, unicodeString: utf16) - down.post(tap: .cghidEventTap) - } - if let up = CGEvent(keyboardEventSource: source, virtualKey: 0, keyDown: false) { - up.post(tap: .cghidEventTap) - } - } - } - - private func postKeystroke(keyCode: CGKeyCode) { - guard let source = eventSource else { return } - CGEvent(keyboardEventSource: source, virtualKey: keyCode, keyDown: true)?.post(tap: .cghidEventTap) - CGEvent(keyboardEventSource: source, virtualKey: keyCode, keyDown: false)?.post(tap: .cghidEventTap) - } -} - -private extension String { - func chunked(into size: Int) -> [String] { - var result: [String] = [] - var index = startIndex - while index < endIndex { - let end = self.index(index, offsetBy: size, limitedBy: endIndex) ?? endIndex - result.append(String(self[index.. (root: AXNodeData, title: String, appName: String) { - let appName = NSRunningApplication(processIdentifier: pid)?.localizedName ?? "App" - return try await Task.detached(priority: .userInitiated) { - guard AXIsProcessTrusted() else { throw AXActionError.notPermitted } - let appElement = AXUIElementCreateApplication(pid) - AXUIElementSetMessagingTimeout(appElement, Self.messagingTimeout) - - guard let window = try Self.pickWindow(appElement, titleHint: titleHint, appName: appName) else { - throw AXActionError.windowGone - } - let title = Self.stringAttribute(window, kAXTitleAttribute) ?? "" - let root = try Self.node(from: window, appName: appName, depth: 0) - return (root, title, appName) - }.value - } - - /// Re-resolve an element by its recorded child-index path (act time). Throws `staleElement` when - /// the walk falls off the tree (the window changed since the snapshot). - public func resolveElement(pid: pid_t, titleHint: String?, path: [Int]) async throws -> AXUIElement { - let appName = NSRunningApplication(processIdentifier: pid)?.localizedName ?? "App" - return try await Task.detached(priority: .userInitiated) { - guard AXIsProcessTrusted() else { throw AXActionError.notPermitted } - let appElement = AXUIElementCreateApplication(pid) - AXUIElementSetMessagingTimeout(appElement, Self.messagingTimeout) - guard let window = try Self.pickWindow(appElement, titleHint: titleHint, appName: appName) else { - throw AXActionError.windowGone - } - var current = window - for index in path { - let children = Self.children(of: current) - guard index < children.count else { throw AXActionError.staleElement } - current = children[index] - } - return current - }.value - } - - // MARK: - Tree walking (background thread) - - private static func pickWindow(_ appElement: AXUIElement, titleHint: String?, - appName: String) throws -> AXUIElement? { - var value: CFTypeRef? - let result = AXUIElementCopyAttributeValue(appElement, kAXWindowsAttribute as CFString, &value) - switch result { - case .success: break - case .cannotComplete: throw AXActionError.appNotResponding(appName: appName) - case .apiDisabled, .notImplemented: throw AXActionError.notPermitted - default: throw AXActionError.windowGone - } - guard let windows = value as? [AXUIElement], !windows.isEmpty else { return nil } - guard let hint = titleHint?.lowercased(), !hint.isEmpty else { - // The app's focused window when available, else the first. - var focused: CFTypeRef? - if AXUIElementCopyAttributeValue(appElement, kAXFocusedWindowAttribute as CFString, &focused) == .success, - let focusedWindow = focused, CFGetTypeID(focusedWindow) == AXUIElementGetTypeID() { - return (focusedWindow as! AXUIElement) - } - return windows.first - } - return windows.first { (stringAttribute($0, kAXTitleAttribute) ?? "").lowercased().contains(hint) } - ?? windows.first - } - - private static func node(from element: AXUIElement, appName: String, depth: Int) throws -> AXNodeData { - // Hard depth cut here mirrors the builder's limit — the builder is the honest reporter; this - // is just the walk's own runaway guard. - let role = stringAttribute(element, kAXRoleAttribute) ?? "AXUnknown" - let label = stringAttribute(element, kAXTitleAttribute) - ?? stringAttribute(element, kAXDescriptionAttribute) - ?? stringAttribute(element, "AXLabel") - ?? "" - let value = stringAttribute(element, kAXValueAttribute) ?? "" - - var actions: CFArray? - AXUIElementCopyActionNames(element, &actions) - let actionNames = (actions as? [String]) ?? [] - let pressable = actionNames.contains(kAXPressAction as String) - - var settable = DarwinBoolean(false) - AXUIElementIsAttributeSettable(element, kAXValueAttribute as CFString, &settable) - - var focusableFlag = DarwinBoolean(false) - AXUIElementIsAttributeSettable(element, kAXFocusedAttribute as CFString, &focusableFlag) - - var childNodes: [AXNodeData] = [] - if depth < 20 { - for child in children(of: element) { - childNodes.append(try node(from: child, appName: appName, depth: depth + 1)) - } - } - return AXNodeData(role: role, label: label, value: value, - isPressable: pressable, isSettable: settable.boolValue, - isFocusable: focusableFlag.boolValue, children: childNodes) - } - - private static func children(of element: AXUIElement) -> [AXUIElement] { - var value: CFTypeRef? - guard AXUIElementCopyAttributeValue(element, kAXChildrenAttribute as CFString, &value) == .success, - let children = value as? [AXUIElement] else { return [] } - return children - } - - private static func stringAttribute(_ element: AXUIElement, _ attribute: String) -> String? { - var value: CFTypeRef? - guard AXUIElementCopyAttributeValue(element, attribute as CFString, &value) == .success else { - return nil - } - return value as? String - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Ax/AXWindowSnapshot.swift b/Sources/ThreeFingerSwitcher/AI/Ax/AXWindowSnapshot.swift deleted file mode 100644 index d5c966a..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Ax/AXWindowSnapshot.swift +++ /dev/null @@ -1,176 +0,0 @@ -import Foundation -import CryptoKit - -/// The computer-use read layer's PURE half (`add-voice-computer-use-agent`, design D5): value models -/// for a window's semantic snapshot, the bounded tree→snapshot builder, and the per-window epoch -/// store that enforces the constrained-ID rule. The live `AXUIElement` boundary (`AXWindowReader`) -/// only produces `AXNodeData` trees; everything below is deterministic and unit-tested over fakes. - -/// One node of a window's accessibility tree, as read at the boundary (or faked in tests). -public struct AXNodeData: Equatable, Sendable { - public var role: String - public var label: String - public var value: String - /// Whether the node supports AXPress / a settable AXValue / keyboard focus. - public var isPressable: Bool - public var isSettable: Bool - public var isFocusable: Bool - public var children: [AXNodeData] - - public init(role: String, label: String = "", value: String = "", - isPressable: Bool = false, isSettable: Bool = false, isFocusable: Bool = false, - children: [AXNodeData] = []) { - self.role = role - self.label = label - self.value = value - self.isPressable = isPressable - self.isSettable = isSettable - self.isFocusable = isFocusable - self.children = children - } -} - -/// One enumerated, actionable element with its STABLE ID — the only currency the act tools accept -/// (never coordinates, never free-form names). `path` is the child-index walk from the window root -/// used to re-resolve the live element at act time. -public struct AXElementRef: Equatable, Sendable, Codable { - public var id: String - public var role: String - public var label: String - public var valuePreview: String - public var isPressable: Bool - public var isSettable: Bool - public var path: [Int] -} - -/// A window's semantic snapshot: extracted text + the enumerated element list, bounded and honest -/// (`truncated` reports a depth/count cut — never a silently-partial tree presented as complete). -public struct AXWindowSnapshot: Equatable, Sendable { - public var pid: pid_t - public var appName: String - public var title: String - public var textBlocks: [String] - public var elements: [AXElementRef] - public var truncated: Bool - /// The constrained-ID epoch: an act must name an element from THIS snapshot generation; a newer - /// read invalidates older IDs' epoch (see `AXSnapshotStore`). - public var epoch: Int - - /// All extracted text joined for the tool summary / extraction prompts. - public var joinedText: String { textBlocks.joined(separator: "\n") } - - /// A cheap content fingerprint (verify-after-act compares before/after). - public var contentHash: String { - var hasher = SHA256() - for block in textBlocks { hasher.update(data: Data(block.utf8)) } - for element in elements { hasher.update(data: Data("\(element.role)|\(element.label)|\(element.valuePreview)".utf8)) } - return hasher.finalize().map { String(format: "%02x", $0) }.joined().prefix(16).lowercased() - } -} - -/// Builds a bounded snapshot from a node tree. Pure — the entire behavior is unit-tested with fakes. -public enum AXSnapshotBuilder { - - public struct Limits: Sendable { - public var maxDepth: Int - public var maxNodes: Int - public var maxTextBlocks: Int - public var maxValuePreview: Int - public init(maxDepth: Int = 14, maxNodes: Int = 1200, maxTextBlocks: Int = 400, - maxValuePreview: Int = 120) { - self.maxDepth = maxDepth - self.maxNodes = maxNodes - self.maxTextBlocks = maxTextBlocks - self.maxValuePreview = maxValuePreview - } - } - - /// Roles whose value is content worth extracting as text. - private static let textRoles: Set = [ - "AXStaticText", "AXTextArea", "AXTextField", "AXLink", "AXHeading", "AXCell", - ] - - public static func build(pid: pid_t, appName: String, title: String, - root: AXNodeData, epoch: Int, - limits: Limits = Limits()) -> AXWindowSnapshot { - var textBlocks: [String] = [] - var elements: [AXElementRef] = [] - var visited = 0 - var truncated = false - - func walk(_ node: AXNodeData, path: [Int], depth: Int) { - visited += 1 - if visited > limits.maxNodes || depth > limits.maxDepth { - truncated = true - return - } - let trimmedValue = node.value.trimmingCharacters(in: .whitespacesAndNewlines) - if textRoles.contains(node.role), !trimmedValue.isEmpty, - textBlocks.count < limits.maxTextBlocks { - textBlocks.append(trimmedValue) - } else if textRoles.contains(node.role), !trimmedValue.isEmpty { - truncated = true - } - if node.isPressable || node.isSettable { - elements.append(AXElementRef( - id: stableID(role: node.role, label: node.label, path: path), - role: node.role, - label: node.label, - valuePreview: String(trimmedValue.prefix(limits.maxValuePreview)), - isPressable: node.isPressable, - isSettable: node.isSettable, - path: path)) - } - for (index, child) in node.children.enumerated() { - walk(child, path: path + [index], depth: depth + 1) - } - } - - walk(root, path: [], depth: 0) - return AXWindowSnapshot(pid: pid, appName: appName, title: title, - textBlocks: textBlocks, elements: elements, - truncated: truncated, epoch: epoch) - } - - /// The stable element ID: role + label + hierarchical path, hashed. Stable across re-reads of an - /// unchanged window (same walk → same path); a moved/renamed element gets a NEW id, which is - /// exactly the staleness signal the constrained-ID rule wants. - public static func stableID(role: String, label: String, path: [Int]) -> String { - let raw = "\(role)|\(label)|\(path.map(String.init).joined(separator: "."))" - let digest = SHA256.hash(data: Data(raw.utf8)) - return "el-" + digest.map { String(format: "%02x", $0) }.joined().prefix(10) - } -} - -/// The per-pid latest-snapshot store enforcing the constrained-ID epoch (design D5): an act resolves -/// its element ID against the MOST RECENT snapshot for that window's pid; an ID from an older epoch -/// (or a different window) is `staleElement` — the model must `read_window` again. `@MainActor` like -/// every stateful controller. -@MainActor -public final class AXSnapshotStore { - private var latest: [pid_t: AXWindowSnapshot] = [:] - private var nextEpoch = 1 - - public init() {} - - /// Register a fresh snapshot, assigning it the next epoch. Returns the stamped snapshot. - public func register(_ snapshot: AXWindowSnapshot) -> AXWindowSnapshot { - var stamped = snapshot - stamped.epoch = nextEpoch - nextEpoch += 1 - latest[snapshot.pid] = stamped - return stamped - } - - public func current(for pid: pid_t) -> AXWindowSnapshot? { latest[pid] } - - /// Resolve an element ID against the CURRENT epoch for `pid`. Throws `staleElement` when the ID - /// isn't in the latest snapshot — the only recovery is a fresh read. - public func resolve(_ elementID: String, pid: pid_t) throws -> AXElementRef { - guard let snapshot = latest[pid], - let element = snapshot.elements.first(where: { $0.id == elementID }) else { - throw AXActionError.staleElement - } - return element - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Ax/AgentActionArbiter.swift b/Sources/ThreeFingerSwitcher/AI/Ax/AgentActionArbiter.swift deleted file mode 100644 index 33faf6c..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Ax/AgentActionArbiter.swift +++ /dev/null @@ -1,67 +0,0 @@ -import Foundation -import CoreGraphics - -/// The agent-vs-human input arbitration (`add-voice-computer-use-agent`, design D9 / spec "Any human -/// trackpad touch aborts agent action instantly"). Two jobs: -/// -/// 1. **Tagged synthetic input**: every synthetic keyboard event the agent posts comes from THIS -/// arbiter's `CGEventSource`, whose `userData` carries a magic tag. The app's own event taps call -/// `isSyntheticAgentEvent(_:)` and IGNORE tagged events — the agent's typing can never be misread -/// as human gestures or trip the ⌘-Tab/scroll taps. -/// 2. **The kill switch**: while `isActing`, a HUMAN trackpad contact (fed from the raw touch -/// stream) triggers `abort()` — cancelling the in-flight act/turn as a DISCARD. The human always -/// wins the input; the visible "agent has the wheel" indicator binds to `isActing`. -@MainActor -public final class AgentActionArbiter: ObservableObject { - - /// The magic tag on every agent-posted event (`CGEventSource.userData`). - public static let syntheticTag: Int64 = 0x7F53_A6E7 - - /// True while an acting tool (click/type) is executing — drives the on-screen indicator and arms - /// the touch kill switch. - @Published public private(set) var isActing = false - - /// Called on abort (human touch during an act): the coordinator wires this to cancel the - /// in-flight turn (engine discard + voice abort). The abort is an acknowledgment path, not an - /// error path. - public var onAbort: (@MainActor () -> Void)? - - /// The tagged source for ALL synthetic agent input. - public let eventSource: CGEventSource? - - private var actingDepth = 0 - private var abortedCurrentAct = false - - public init() { - let source = CGEventSource(stateID: .privateState) - source?.userData = Self.syntheticTag - self.eventSource = source - } - - /// Whether `event` was posted by the agent itself (the taps' ignore rule). - public nonisolated static func isSyntheticAgentEvent(_ event: CGEvent) -> Bool { - event.getIntegerValueField(.eventSourceUserData) == syntheticTag - } - - /// Run one acting scope. Nesting-safe (a tool that acts twice keeps the indicator up); the abort - /// flag resets per outermost scope. - public func acting(_ body: () async throws -> T) async rethrows -> T { - actingDepth += 1 - if actingDepth == 1 { - abortedCurrentAct = false - isActing = true - } - defer { - actingDepth -= 1 - if actingDepth == 0 { isActing = false } - } - return try await body() - } - - /// A HUMAN trackpad contact arrived. Only bites while acting — normal app gestures are untouched. - public func humanTouchDetected() { - guard isActing, !abortedCurrentAct else { return } - abortedCurrentAct = true - onAbort?() - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Ax/ComputerUseToolContributor.swift b/Sources/ThreeFingerSwitcher/AI/Ax/ComputerUseToolContributor.swift deleted file mode 100644 index a9f101d..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Ax/ComputerUseToolContributor.swift +++ /dev/null @@ -1,242 +0,0 @@ -import Foundation - -/// A resolved window target for the computer-use tools: produced by the coordinator's resolver over -/// the SWITCHER'S OWN enumeration (`WindowService.snapshot()` matching) — the tools never enumerate -/// windows themselves (switcher-as-API, design D6). -public struct ComputerUseWindowTarget: Equatable, Sendable { - public var pid: pid_t - public var title: String - public var appName: String - public init(pid: pid_t, title: String, appName: String) { - self.pid = pid - self.title = title - self.appName = appName - } -} - -/// The computer-use tools (`add-voice-computer-use-agent`, design D6): `read_window` / -/// `focus_window` (.auto) + `click_element` / `type_text` (.confirm, auto-mode liftable via the -/// gate). AX-first, constrained element IDs only — the schemas contain NO coordinate parameter, and -/// an unresolvable target is a clean failure, never a guess. Acts run inside the arbiter's acting -/// scope (human-touch kill switch + tagged synthetic input) and verify-after-act. -struct ComputerUseToolContributor: ToolContributor { - - /// Live opt-in (thread-safe read — the registry re-queries per turn, so toggling is immediate). - let enabled: @Sendable () -> Bool - /// Resolve app/title hints against the switcher's enumeration (main-actor). - let resolveWindow: @MainActor (String?, String?) -> ComputerUseWindowTarget? - /// Raise through the EXISTING hardened commit path (`raiseCommitted` — the third caller). - let focusWindow: @MainActor (ComputerUseWindowTarget) -> Bool - let performer: AXActionPerformer - let arbiter: AgentActionArbiter - /// Narration hook: spoken when voice is active, always visible (spec: silence never hides an act). - let narrate: @MainActor (String) -> Void - - // MARK: - Descriptors - - static let readWindowName = "read_window" - static let focusWindowName = "focus_window" - static let clickElementName = "click_element" - static let typeTextName = "type_text" - - private static let windowArgs = """ - {"type":"object","properties":{"app":{"type":"string","description":"App name (fuzzy)"},\ - "title":{"type":"string","description":"Window title hint (optional)"}}} - """ - private static let clickArgs = """ - {"type":"object","required":["element_id"],"properties":{"app":{"type":"string"},\ - "title":{"type":"string"},"element_id":{"type":"string","description":"An id from read_window"}}} - """ - private static let typeArgs = """ - {"type":"object","required":["text"],"properties":{"app":{"type":"string"},"title":{"type":"string"},\ - "element_id":{"type":"string"},"text":{"type":"string"},"submit":{"type":"boolean"}}} - """ - - func descriptors() -> [ToolDescriptor] { - guard enabled() else { return [] } // off = ABSENT (spec: "Off means absent") - return [ - ToolDescriptor(name: Self.readWindowName, - summary: "Read a window's visible text and interactive elements (accessibility, no screenshot).", - argsSchema: StructuredSchema(name: Self.readWindowName, json: Self.windowArgs), - writePolicy: .auto, - keywords: ["read", "window", "screen", "look", "see", "text", "terminal", "content"]), - ToolDescriptor(name: Self.focusWindowName, - summary: "Bring an app's window to the front (the switcher's own raise).", - argsSchema: StructuredSchema(name: Self.focusWindowName, json: Self.windowArgs), - writePolicy: .auto, - keywords: ["focus", "switch", "window", "front", "go", "open", "move"]), - ToolDescriptor(name: Self.clickElementName, - summary: "Press a button/link in a window, by an element id from read_window.", - argsSchema: StructuredSchema(name: Self.clickElementName, json: Self.clickArgs), - writePolicy: .confirm, - keywords: ["click", "press", "button", "tap", "select"]), - ToolDescriptor(name: Self.typeTextName, - summary: "Type text into a window (optionally into a specific field, optionally press Return).", - argsSchema: StructuredSchema(name: Self.typeTextName, json: Self.typeArgs), - writePolicy: .confirm, - keywords: ["type", "write", "enter", "send", "reply", "input", "text"]), - ] - } - - func canHandle(_ tool: String) -> Bool { - [Self.readWindowName, Self.focusWindowName, Self.clickElementName, Self.typeTextName] - .contains(tool) - } - - // MARK: - Args - - private struct Args: Decodable { - var app: String? - var title: String? - var element_id: String? - var text: String? - var submit: Bool? - } - - private func parseArgs(_ json: String) -> Args { - guard let data = json.data(using: .utf8), - let args = try? JSONDecoder().decode(Args.self, from: data) else { return Args() } - return args - } - - // MARK: - Run - - func run(_ call: RoutedCall, gate: ApprovalGate) async -> ToolStepResult { - guard enabled() else { - return ToolStepResult(tool: call.descriptor.name, - status: .failed(headline: "Computer use is turned off."), - summary: "Computer use is turned off.") - } - let args = parseArgs(call.route.argumentsJSON) - let tool = call.descriptor.name - - guard let target = await resolveTarget(args) else { - let headline = "Couldn't find that window." - return ToolStepResult(tool: tool, status: .failed(headline: headline), - summary: "\(headline) Asked for app “\(args.app ?? "frontmost")”.") - } - - do { - switch tool { - case Self.readWindowName: - return try await readWindow(target) - case Self.focusWindowName: - return await focusWindowStep(target) - case Self.clickElementName: - return try await clickElement(target, args: args, gate: gate, call: call) - case Self.typeTextName: - return try await typeText(target, args: args, gate: gate, call: call) - default: - return ToolStepResult(tool: tool, status: .failed(headline: "That tool isn't available."), - summary: "Unknown tool: \(tool).") - } - } catch { - let presented = AIError.message(for: error) - return ToolStepResult(tool: tool, status: .failed(headline: presented.headline), - summary: presented.headline) - } - } - - private func resolveTarget(_ args: Args) async -> ComputerUseWindowTarget? { - await MainActor.run { resolveWindow(args.app, args.title) } - } - - // MARK: - The four tools - - private func readWindow(_ target: ComputerUseWindowTarget) async throws -> ToolStepResult { - let snapshot = try await performer.snapshot(pid: target.pid, titleHint: target.title) - var lines: [String] = [] - lines.append("Window: \(snapshot.appName) — “\(snapshot.title)”\(snapshot.truncated ? " (content truncated)" : "")") - let text = snapshot.joinedText - if !text.isEmpty { - lines.append("Text:\n\(String(text.prefix(4_000)))") - } - if !snapshot.elements.isEmpty { - lines.append("Interactive elements (use these ids with click_element/type_text):") - for element in snapshot.elements.prefix(60) { - let label = element.label.isEmpty ? element.valuePreview : element.label - lines.append("- [\(element.id)] \(element.role) “\(String(label.prefix(60)))”") - } - } - return ToolStepResult(tool: Self.readWindowName, status: .done, - summary: lines.joined(separator: "\n")) - } - - private func focusWindowStep(_ target: ComputerUseWindowTarget) async -> ToolStepResult { - let raised = await MainActor.run { focusWindow(target) } - if raised { - return ToolStepResult(tool: Self.focusWindowName, status: .done, - summary: "Focused \(target.appName) — “\(target.title)”.") - } - return ToolStepResult(tool: Self.focusWindowName, - status: .failed(headline: "Couldn't raise that window."), - summary: "Couldn't raise \(target.appName) — “\(target.title)”.") - } - - private func clickElement(_ target: ComputerUseWindowTarget, args: Args, - gate: ApprovalGate, call: RoutedCall) async throws -> ToolStepResult { - guard let elementID = args.element_id else { - return ToolStepResult(tool: Self.clickElementName, - status: .failed(headline: "No element was named."), - summary: "click_element needs an element_id from read_window.") - } - let review = TaskReview.action( - title: "Click in \(target.appName)", - fields: [ReviewField("Window", target.title), - ReviewField("Element", elementID)], - payload: .openTool(tool: Self.clickElementName, - action: ParsedOpenTool(applicable: true, reason: nil, payload: elementID))) - switch await gate.awaitDecision(for: review) { - case .approve: - let verification = try await arbiter.acting { - try await performer.press(pid: target.pid, titleHint: target.title, elementID: elementID) - } - let summary = "Clicked in \(target.appName): \(verification.detail)." - await MainActor.run { narrate(summary) } - return ToolStepResult(tool: Self.clickElementName, status: .done, summary: summary) - case .skip: - return ToolStepResult(tool: Self.clickElementName, status: .declined(reason: "skipped"), - summary: "Skipped the click.") - case .cancel: - return ToolStepResult(tool: Self.clickElementName, - status: .declined(reason: TaskKindToolContributor.cancelledReason), - summary: "Cancelled.") - } - } - - private func typeText(_ target: ComputerUseWindowTarget, args: Args, - gate: ApprovalGate, call: RoutedCall) async throws -> ToolStepResult { - guard let text = args.text, !text.isEmpty else { - return ToolStepResult(tool: Self.typeTextName, - status: .failed(headline: "There's no text to type."), - summary: "type_text needs non-empty text.") - } - let submit = args.submit ?? false - let review = TaskReview.action( - title: "Type in \(target.appName)", - fields: [ReviewField("Window", target.title), - ReviewField("Text", String(text.prefix(200))), - ReviewField("Press Return", submit ? "yes" : "no")], - payload: .openTool(tool: Self.typeTextName, - action: ParsedOpenTool(applicable: true, reason: nil, payload: text))) - switch await gate.awaitDecision(for: review) { - case .approve: - // Assert focus first so the text lands where the preview said it would. - _ = await MainActor.run { focusWindow(target) } - let verification = try await arbiter.acting { - try await performer.typeText(pid: target.pid, titleHint: target.title, - elementID: args.element_id, text: text, submit: submit) - } - let summary = "Typed into \(target.appName)\(submit ? " and pressed Return" : ""): \(verification.detail)." - await MainActor.run { narrate(summary) } - return ToolStepResult(tool: Self.typeTextName, status: .done, summary: summary) - case .skip: - return ToolStepResult(tool: Self.typeTextName, status: .declined(reason: "skipped"), - summary: "Skipped typing.") - case .cancel: - return ToolStepResult(tool: Self.typeTextName, - status: .declined(reason: TaskKindToolContributor.cancelledReason), - summary: "Cancelled.") - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Compute/ComputeLane.swift b/Sources/ThreeFingerSwitcher/AI/Compute/ComputeLane.swift deleted file mode 100644 index 535ab5f..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Compute/ComputeLane.swift +++ /dev/null @@ -1,105 +0,0 @@ -import Foundation - -/// The two-lane compute model (addendum §A1 — OWNED by `ai-compute-tiers`, written verbatim from the -/// pinned sketch). MLX-free Core: pure value types + a pure policy, `swift test`-verified. -/// -/// The V2 batched runtime (`ai-batched-runtime-and-context`) folds K conversation streams into ONE GPU -/// forward pass per decode step — the right answer for concurrency over one weight read, but single-lane: -/// the foreground reply, the router's `structured()` turn, every classify, every memory-TOC retrieval, -/// and every parked-subagent advance all queue behind the SAME GPU decode loop and the same ~153 GB/s -/// unified-memory bus. This slice adds a SECOND physical lane so the light, frequent, structured work -/// runs OFF the GPU, CONCURRENTLY with the heavy reply, on a small ternary/BitNet-class model whose -/// weights are ~32× smaller and therefore barely touch the shared bus. -/// -/// The physics this rests on (cited honestly; only the user's stable-signed build verifies live): -/// - **Prefill ≈ ~4× on the GPU neural accelerators** → prefill-heavy heavy generation + media diffusion -/// belong on the GPU. -/// - **Token-gen is bandwidth-bound at ~153 GB/s** → two heavy generations on one lane contend for one bus. -/// - **Ternary weights ≈ ~32× smaller** → a CPU-lane ternary decode runs concurrently with a GPU -/// generation with low bus contention (the whole reason a second lane is a win, not just a second queue). -/// - **CPU per-token is slower (the honest limit)** → the CPU lane is for SHORT structured bursts ONLY, -/// NEVER the long foreground reply. - -// MARK: - Compute lane - -/// Which physical lane a runtime / work-unit uses. The GPU does heavy generation + diffusion; the CPU -/// ternary lane does short/frequent/structured work (routing, classify, memory-index, parked subagents) -/// — concurrently, because ternary weights are bandwidth-frugal. -public enum ComputeLane: String, Codable, Sendable, CaseIterable { - case gpu - case cpuTernary -} - -// MARK: - Agent work role - -/// What KIND of agent work a unit is — the input to the role→lane policy. The lane is a deterministic -/// function of the role alone (never the model), so lane assignment can never drift between call sites. -public enum AgentWorkRole: String, Codable, Sendable, CaseIterable { - /// The main, visible reply → GPU (long, prefill-heavy; the user is watching it). - case foregroundGeneration - /// Image/video diffusion → GPU (evicts chat under the 48 GB budget; see the fleet slice §C1). - case mediaDiffusion - /// The router's `structured()` route turn → CPU ternary (short, frequent, bounded). - case toolRoute - /// Cheap decisions — should-park / needs-you / which-skill → CPU ternary. - case classify - /// Memory index/TOC retrieval → CPU ternary. - case memoryRetrieval - /// A background advance of a parked session → CPU ternary (runs while the GPU streams the foreground). - case parkedSubagent -} - -// MARK: - Role→lane policy - -/// The pure role→lane policy seam (addendum §A1, OWNED here). A total function: every defined role maps -/// to a lane, and the same role always maps to the same lane. This is the SINGLE source of truth every -/// consumer (scheduler, batched runtime, executor) reads — so lane assignment cannot drift between sites. -public protocol LaneRouting: Sendable { - func lane(for role: AgentWorkRole) -> ComputeLane -} - -/// The concrete, default role→lane map (design D1). Heavy generation → `.gpu`; the router turn, -/// classification, memory retrieval, and parked-subagent advances → `.cpuTernary`. No state, no time -/// input needed — a pure switch, trivially `swift test`-exhaustive (one assertion per case). -/// -/// Rationale (D1): a pure total map is the one source of truth; deriving the lane from the model -/// descriptor instead would be wrong because the SAME ternary model could in principle serve a GPU role -/// — so the ROLE, not the model, drives the lane. -public struct DefaultLaneRouting: LaneRouting { - public init() {} - - public func lane(for role: AgentWorkRole) -> ComputeLane { - switch role { - case .foregroundGeneration, .mediaDiffusion: - // Heavy, prefill-bound work → the GPU neural accelerators (~4× prefill). - return .gpu - case .toolRoute, .classify, .memoryRetrieval, .parkedSubagent: - // Short, frequent, structured bursts → the bandwidth-frugal CPU ternary lane. - return .cpuTernary - } - } -} - -// MARK: - Lane affinity hint - -/// The lane-affinity hint (design D4): the compute lane a runnable session PREFERS, derived from its -/// work role via the role→lane policy. Carried ADDITIVELY beside the parked scheduler's -/// `runnableSessions(now:maxSlots:)` returned IDs and read by the dispatcher — the pinned scheduler / -/// batched-runtime signatures are UNCHANGED; this is an attached value, not a signature change. -/// -/// Consumes `AgentSessionID` verbatim from `ai-conversation-runtime` (never redefined). -public struct LaneAffinity: Equatable, Sendable { - public let sessionID: AgentSessionID - public let lane: ComputeLane - - public init(sessionID: AgentSessionID, lane: ComputeLane) { - self.sessionID = sessionID - self.lane = lane - } - - /// Derive a session's affinity from its work role via a `LaneRouting` policy. A `parkedSubagent` - /// session yields `.cpuTernary`; a `foregroundGeneration` session yields `.gpu`. - public init(sessionID: AgentSessionID, role: AgentWorkRole, routing: LaneRouting) { - self.init(sessionID: sessionID, lane: routing.lane(for: role)) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Compute/GatedLaneRouting.swift b/Sources/ThreeFingerSwitcher/AI/Compute/GatedLaneRouting.swift deleted file mode 100644 index e130aa7..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Compute/GatedLaneRouting.swift +++ /dev/null @@ -1,36 +0,0 @@ -import Foundation - -/// The master-toggle gate over the role→lane policy (design D6). A `LaneRouting` DECORATOR: when the -/// CPU lane is disabled, it coerces EVERY role's lane to `.gpu`, so a one-lane / fleet-of-one build -/// stays valid and behaves exactly as today (no `TernaryCPURuntime` installed, all work on the GPU -/// batched runtime). When enabled, it delegates to the wrapped policy (`DefaultLaneRouting`) unchanged. -/// -/// The CPU lane is gated by `cpuLaneEnabled`, itself under the master `fullPotentialEnabled` — BOTH owned -/// by `ai-full-potential-toggle` (addendum §D1). This slice READS those flags; it does NOT define the -/// persisted keys. They are injected here as booleans (and recomputed by the owner whenever settings -/// change) so the gate LOGIC is pure and `swift test`-verified without depending on `AppSettings`. The -/// effective gate is `fullPotentialEnabled && cpuLaneEnabled` (a sub-flag never overrides the master OFF). -public struct GatedLaneRouting: LaneRouting { - /// The policy applied when the CPU lane is enabled (typically `DefaultLaneRouting`). - private let base: LaneRouting - /// `fullPotentialEnabled && cpuLaneEnabled` — the two-lane policy is in effect only when true. - public let cpuLaneActive: Bool - - /// - Parameters: - /// - base: the underlying role→lane policy (`DefaultLaneRouting`). - /// - fullPotentialEnabled: the master gate (`ai-full-potential-toggle`, §D1 — read, not owned). - /// - cpuLaneEnabled: the CPU-lane sub-flag (§D1 — read, not owned). - public init(base: LaneRouting = DefaultLaneRouting(), - fullPotentialEnabled: Bool, - cpuLaneEnabled: Bool) { - self.base = base - self.cpuLaneActive = fullPotentialEnabled && cpuLaneEnabled - } - - public func lane(for role: AgentWorkRole) -> ComputeLane { - // OFF (master off OR sub-flag off): coerce every role to the GPU lane — the single-lane build. - guard cpuLaneActive else { return .gpu } - // ON: the `DefaultLaneRouting` mapping holds (heavy → GPU, structured → CPU ternary). - return base.lane(for: role) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Compute/LaneArbiter.swift b/Sources/ThreeFingerSwitcher/AI/Compute/LaneArbiter.swift deleted file mode 100644 index 4423900..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Compute/LaneArbiter.swift +++ /dev/null @@ -1,91 +0,0 @@ -import Foundation - -/// Cross-lane concurrency admission (design D3). PURE and deterministic: `now:` and free memory are -/// INPUTS (mirrors `DockHoverModel` / `ConcurrencyBudget`), so the decision is `swift test`-able without -/// real GPU work. The arbiter admits CPU-lane work CONCURRENTLY with GPU work, bounds CPU-lane -/// concurrency on its OWN small cap (it does NOT borrow GPU batch slots), and enforces two honest -/// invariants: -/// 1. A heavy GPU generation is NEVER made to wait on CPU-lane work. -/// 2. CPU-lane bursts NEVER preempt or starve the foreground GPU reply. -/// -/// The two lanes are physically independent (GPU cores + accelerators vs CPU cores); the ternary -/// weights' tiny bandwidth footprint is exactly what makes true concurrency — not a second queue — -/// correct. A CPU-lane unit that cannot be admitted under its own cap or the residency budget in a given -/// step WAITS (stays runnable) — it is NOT a failure. -public struct LaneArbiter: Sendable { - - /// The CPU lane's OWN concurrency cap — small and fixed, independent of the GPU's RAM-derived K. - /// The CPU lane never borrows GPU batch slots, so this cap is the only bound on CPU-lane concurrency. - public let cpuLaneCap: Int - - /// The co-residency budget the ternary model must satisfy to be admitted at all. - public let budget: LaneResidencyBudget - - public init(cpuLaneCap: Int, budget: LaneResidencyBudget) { - self.cpuLaneCap = max(0, cpuLaneCap) - self.budget = budget - } - - /// The outcome of an admission tick — explicit, so a test can assert the two invariants directly. - public struct Admission: Equatable, Sendable { - /// GPU-lane requests admitted THIS tick. A heavy GPU generation is ALWAYS admitted (invariant 1): - /// it never waits on CPU-lane work and is never deferred behind it. - public var admittedGPU: [AgentSessionID] - /// CPU-lane requests admitted THIS tick, bounded by `cpuLaneCap` + the residency budget. - public var admittedCPU: [AgentSessionID] - /// CPU-lane requests that could not be admitted this tick — they WAIT (stay runnable), not fail. - public var waitingCPU: [AgentSessionID] - - public init(admittedGPU: [AgentSessionID] = [], - admittedCPU: [AgentSessionID] = [], - waitingCPU: [AgentSessionID] = []) { - self.admittedGPU = admittedGPU - self.admittedCPU = admittedCPU - self.waitingCPU = waitingCPU - } - } - - /// Admit GPU-lane and CPU-lane requests for one tick, given the free memory, current GPU stream - /// count, context length, and `now` (accepted for deterministic-input parity; admission here depends - /// only on the lane state + budget, not on wall-clock — but the signature mirrors the project's - /// `now:`-injected pure models so a later cadence rule slots in without a signature break). - /// - /// - `gpuRequests`: GPU-affined + foreground sessions wanting to advance. - /// - `cpuRequests`: CPU-ternary-affined sessions (router turns, classify, memory, parked subagents). - /// - `inflightCPU`: CPU-lane units ALREADY running this step (counted against `cpuLaneCap`). - public func admit(now: Date, - freeBytes: Int64, - gpuStreams: Int, - contextTokens: Int, - gpuRequests: [AgentSessionID], - cpuRequests: [AgentSessionID], - inflightCPU: Int = 0) -> Admission { - _ = now - - // Invariant 1: the GPU lane is admitted IN FULL, unconditionally — it never waits on CPU work. - // The GPU lane's own concurrency (K) is bounded upstream by `ConcurrencyBudget`/the batched - // runtime; the arbiter never defers a GPU request behind CPU-lane work. - var result = Admission(admittedGPU: gpuRequests) - - // The CPU lane co-resides only if the small ternary footprint fits beside the GPU batch + KV. - // If it cannot co-reside, every CPU burst WAITS (not fails) — invariant: unadmittable ⇒ wait. - let coResides = budget.ternaryCoResides(freeBytes: freeBytes, - gpuStreams: gpuStreams, - contextTokens: contextTokens) - guard coResides else { - result.waitingCPU = cpuRequests - return result - } - - // Bound CPU-lane concurrency on its OWN cap (invariant 2: it never borrows GPU slots, so even a - // flood of CPU bursts cannot grow past `cpuLaneCap` and cannot touch the foreground GPU reply). - let availableCPU = max(0, cpuLaneCap - max(0, inflightCPU)) - if cpuRequests.count <= availableCPU { - result.admittedCPU = cpuRequests - } else { - result.admittedCPU = Array(cpuRequests.prefix(availableCPU)) - result.waitingCPU = Array(cpuRequests.dropFirst(availableCPU)) - } - return result - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Compute/LaneResidencyBudget.swift b/Sources/ThreeFingerSwitcher/AI/Compute/LaneResidencyBudget.swift deleted file mode 100644 index 17050ee..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Compute/LaneResidencyBudget.swift +++ /dev/null @@ -1,52 +0,0 @@ -import Foundation - -/// Cross-lane co-residency math (design D3). PURE: the live free-memory probe happens at the runtime -/// boundary and is INJECTED here (mirrors `ConcurrencyBudget`), so the decision is unit-testable without -/// Metal. It answers ONE question: does the small ternary model co-reside with the current GPU batch -/// (resident chat weights, read once) + the GPU streams' KV under the 48 GB unified-memory budget? -/// -/// The ternary weights are ~32× smaller than an FP16/Q4 chat model's, so `ternaryResidencyBytes` is a -/// SMALL constant. The whole point: the ternary model fits in headroom where a SECOND full chat model -/// would not — that is what makes a second lane a co-resident win rather than an eviction. -/// -/// This slice supplies ONLY the ternary co-residency MATH; the fleet's eviction policy itself is owned by -/// `ai-model-fleet` (§C1) and is out of scope here. -public struct LaneResidencyBudget: Equatable, Sendable { - /// Resident GPU chat weights, read once and shared across all GPU streams (e.g. ~17 GB at 4-bit). - public var chatWeightBytes: Int64 - /// Per-GPU-stream KV-cache bytes at the current context length (the cost that grows with concurrency). - public var kvBytesPerGPUStream: Int64 - /// The ternary model's resident footprint — SMALL (~32× smaller weights). Read once; it does not grow - /// per CPU-lane burst (the CPU lane's per-burst working set is negligible beside the weight set). - public var ternaryResidencyBytes: Int64 - /// OS + app + graph-activation headroom held out of the budget. - public var reservedBytes: Int64 - - public init(chatWeightBytes: Int64, - kvBytesPerGPUStream: Int64, - ternaryResidencyBytes: Int64, - reservedBytes: Int64) { - self.chatWeightBytes = chatWeightBytes - self.kvBytesPerGPUStream = kvBytesPerGPUStream - self.ternaryResidencyBytes = ternaryResidencyBytes - self.reservedBytes = reservedBytes - } - - /// Bytes the resident GPU batch occupies at `gpuStreams` streams (weights read once + KV per stream). - public func gpuResidentBytes(gpuStreams: Int) -> Int64 { - chatWeightBytes + Int64(max(0, gpuStreams)) * kvBytesPerGPUStream - } - - /// Does the ternary model co-reside with the current GPU batch + KV under `freeBytes`? - /// - /// `freeBytes` is the total unified memory available to the AI feature (injected). The ternary model - /// is admitted iff, after the GPU batch's resident bytes and the reserved headroom, the small ternary - /// footprint still fits. `contextTokens` is accepted for signature parity with the fleet's call site - /// (the GPU per-stream KV already encodes the context cost in `kvBytesPerGPUStream`); it is not used - /// to grow the ternary footprint, which is context-independent here. - public func ternaryCoResides(freeBytes: Int64, gpuStreams: Int, contextTokens: Int) -> Bool { - _ = contextTokens - let remaining = freeBytes - gpuResidentBytes(gpuStreams: gpuStreams) - reservedBytes - return remaining >= ternaryResidencyBytes - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Compute/StubTernaryRuntime.swift b/Sources/ThreeFingerSwitcher/AI/Compute/StubTernaryRuntime.swift deleted file mode 100644 index 62f6213..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Compute/StubTernaryRuntime.swift +++ /dev/null @@ -1,110 +0,0 @@ -import Foundation - -/// A deterministic, scriptable CPU-ternary-lane `LLMRuntime` for `swift build` / `swift test` (the real -/// bitnet.cpp-class `TernaryCPURuntime` is native-linked and `xcodebuild` compile-verify only — see the -/// GemmaRuntime file). It conforms to the EXISTING `LLMRuntime` seam (design D2: a SECOND conformer, -/// NOT a new protocol), so feature code stays model-agnostic and selects the CPU lane BY LANE. -/// -/// It scripts the SHORT-BURST paths the CPU lane actually serves (design D5): a `structured()` route -/// turn, classify, memory-retrieval, and parked-subagent `generate`/`chat` — never the long foreground -/// reply (that is GPU-only). A scripted error fails the burst so the per-turn `.failed` path is -/// observable; a cancelled burst is NOT a failure (`RuntimeError.cancelled`, treated as benign upstream). -/// -/// `capabilities` advertises `.text` ONLY — a ternary text model carries no vision/audio, and the policy -/// keeps `mediaDiffusion`/vision roles on the GPU lane, so a vision request against this stub is a hard -/// `unsupportedModality` error (never a silent degrade). -public final class StubTernaryRuntime: LLMRuntime, @unchecked Sendable { - - /// CPU ternary lane carries TEXT only. A vision/media role is never routed here (the role→lane policy - /// keeps `mediaDiffusion` on `.gpu`); a vision request still hard-errors rather than degrading. - public let capabilities: Set - - /// Scripted response chunks for `generate`/`chat`. Empty → echo the prompt as one token. - private var scriptedTokens: [String] - /// Scripted `structured(...)` behavior. nil → a generic empty-object attempt. - private var structuredScript: StubLLMRuntime.StructuredScript? - /// A scripted load/prepare/decode failure for the next burst (simulates a CPU-lane failure mapped - /// into `RuntimeError` at the conformer boundary). Cleared after it fires once. - private var scriptedError: RuntimeError? - private var maxRepairAttempts: Int - private let interTokenDelayNanos: UInt64 - private let lock = NSLock() - - public init(capabilities: Set = [.text], - scriptedTokens: [String] = [], - structuredScript: StubLLMRuntime.StructuredScript? = nil, - scriptedError: RuntimeError? = nil, - maxRepairAttempts: Int = 3, - interTokenDelayNanos: UInt64 = 1_000_000) { - self.capabilities = capabilities - self.scriptedTokens = scriptedTokens - self.structuredScript = structuredScript - self.scriptedError = scriptedError - self.maxRepairAttempts = maxRepairAttempts - self.interTokenDelayNanos = interTokenDelayNanos - } - - // MARK: Streaming (short bursts only — never the long reply) - - public func generate(_ request: LLMRequest) -> AsyncThrowingStream { - let chunks = scriptedTokens.isEmpty ? [request.prompt] : scriptedTokens - let delay = interTokenDelayNanos - let needsVision = request.requiresVision - let caps = capabilities - let err = takeError() - - return AsyncThrowingStream { continuation in - let task = Task { - // A vision request against a text-only ternary lane is a hard error, never a degrade. - if needsVision && !caps.contains(.vision) { - continuation.finish(throwing: RuntimeError.unsupportedModality(.vision)) - return - } - // A simulated CPU-lane failure (load/prepare/decode), already mapped to RuntimeError. - if let err { - continuation.finish(throwing: err) - return - } - do { - for (i, chunk) in chunks.enumerated() { - try Task.checkCancellation() - if delay > 0 { try await Task.sleep(nanoseconds: delay) } - try Task.checkCancellation() - continuation.yield(Token(chunk, isFinal: i == chunks.count - 1)) - } - continuation.finish() - } catch is CancellationError { - // Cancellation is NOT a failure (the turn was discarded); a clean benign terminal. - continuation.finish(throwing: RuntimeError.cancelled) - } catch { - continuation.finish(throwing: error) - } - } - continuation.onTermination = { _ in task.cancel() } - } - } - - // MARK: Structured (the router's route turn — the CPU lane's bread and butter) - - public func structured( - _ request: LLMRequest, - schema: StructuredSchema, - as type: T.Type - ) async throws -> StructuredOutcome { - try Task.checkCancellation() - if let err = takeError() { throw err } - // Reuse the stub's proven validate → repair/retry → decode/decline pipeline so the CPU-lane - // route path exercises the SAME contract the GPU runtime does (no shortcut). - let proxy = StubLLMRuntime(capabilities: capabilities, - structuredScript: structuredScript, - maxRepairAttempts: maxRepairAttempts) - return try await proxy.structured(request, schema: schema, as: type) - } - - private func takeError() -> RuntimeError? { - lock.lock(); defer { lock.unlock() } - let e = scriptedError - scriptedError = nil - return e - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/DevAIRuntime.swift b/Sources/ThreeFingerSwitcher/AI/DevAIRuntime.swift deleted file mode 100644 index b6119eb..0000000 --- a/Sources/ThreeFingerSwitcher/AI/DevAIRuntime.swift +++ /dev/null @@ -1,64 +0,0 @@ -import Foundation - -/// A **development wiring** for the AI model layer used until the real MLX/Gemma runtime (phase 10) is -/// wired into the app's `xcodebuild` target. It lets the streaming preview canvas and the whole -/// command pipeline run in a signed build TODAY without a real multi-gigabyte download: -/// -/// - The standard `ModelCatalog` ships **placeholder** download URLs (`models.invalid`) and dummy -/// integrity SHAs, so a real download can't succeed and `downloadAndVerify` would always fail the -/// SHA check. This builds a **dev registry** whose `integritySHA` is the SHA of a small fabricated -/// payload, paired with a `ModelDownloading` that returns exactly that payload — so download + -/// verify + load succeed deterministically and instantly. -/// - The `runtimeFactory` resolves a `StubLLMRuntime`, which echoes its prompt and serves structured -/// output, so commands produce a visible (if canned) result on-device with no network. -/// -/// Swapping in the real runtime is a single `runtimeFactory` change + the real registry/downloader -/// (design D1/D7): feature code only ever sees `LLMRuntime`, so nothing here leaks upward. -enum DevAIRuntime { - - /// The fabricated dev "weights" payload. Tiny and deterministic; its SHA pins the dev descriptor. - private static let payload = Data("three-finger-switcher-dev-gemma-stub".utf8) - - /// A dev registry mirroring `ModelCatalog.standard`'s ids/display names/capabilities, but with an - /// integrity SHA that matches `payload` so verification passes. Keeps the real registry's selection - /// behavior (default-first, capability subset) intact for the band/executor. - @MainActor - static var devRegistry: ModelCatalog { - let standard = ModelCatalog.standard - let models = standard.models.map { d in - ModelDescriptor( - id: d.id, - displayName: d.displayName, - sizeBytes: Int64(payload.count), - integritySHA: ModelManager.sha256Hex(payload), - downloadURL: d.downloadURL, - capabilities: d.capabilities, - quantization: d.quantization - ) - } - return ModelCatalog(models: models, defaultModelID: standard.defaultModelID) - } - - /// Build a `ModelManager` wired to the dev stub: the dev registry, a downloader that returns the - /// pinned payload, and a `StubLLMRuntime` factory. `optedIn` seeds the opt-in from settings. - @MainActor - static func makeModelManager(optedIn: Bool) -> ModelManager { - ModelManager( - registry: devRegistry, - downloader: DevDownloader(payload: payload), - optedIn: optedIn, - runtimeFactory: { descriptor in StubLLMRuntime(capabilities: descriptor.capabilities) } - ) - } - - /// A `ModelDownloading` that returns the fabricated dev payload immediately (no network). Honors - /// the `ModelManager` contract: reports full progress, returns the bytes for the integrity check. - private struct DevDownloader: ModelDownloading { - let payload: Data - func download(_ descriptor: ModelDescriptor, to destination: URL, - progress: @Sendable (Double) -> Void) async throws -> Data { - progress(1.0) - return payload - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/EvictionPolicy.swift b/Sources/ThreeFingerSwitcher/AI/EvictionPolicy.swift deleted file mode 100644 index 15b5ca2..0000000 --- a/Sources/ThreeFingerSwitcher/AI/EvictionPolicy.swift +++ /dev/null @@ -1,107 +0,0 @@ -import Foundation - -/// Automatic model-weight eviction (`model-idle-ttl-and-memory-pressure`): the PURE decision core. -/// -/// The resident ~17 GB model previously had no automatic path back out of RAM — `evict()` fired only -/// on opt-in-off, a fleet plan, or the manual Settings button. This policy adds the two automatic -/// triggers the spec promises: memory pressure (fast path) and a quiescence-keyed idle TTL (backstop). -/// It is a pure function — time, activity, pressure, and quiescence are explicit inputs — so every -/// rule is deterministically testable without real weights or a real OS pressure source (the -/// `ResidencyPlanner`/`ConcurrencyBudget` idiom). `ModelManager` is the only executor of its verdicts. - -/// The OS memory-pressure level, as reported by the injected observer (a thin -/// `DispatchSource.makeMemoryPressureSource` wrapper in the app; a settable fake in tests). -public enum MemoryPressureLevel: Comparable, Sendable { - case nominal - case warning - case critical -} - -/// A point-in-time view of whether the AI system is busy, pulled (not pushed) from the scheduler at -/// decision time. Same-actor (`@MainActor`) evaluation makes the evict-vs-just-scheduled ordering -/// deterministic: work scheduled after the snapshot lands after the evict and simply lazy-reloads. -public struct QuiescenceSnapshot: Equatable, Sendable { - /// A generation turn is streaming right now (foreground or background). Blocks ALL eviction. - public var turnInFlight: Bool - /// Any live conversational surface is open — an `.active` chat session or an open voice - /// conversation. Blocks TTL and warning-pressure eviction (a user mid-dialogue must not pay a - /// reload between turns); only critical pressure overrides. - public var foregroundSessionActive: Bool - /// The earliest scheduled parked advance, if any (`ParkedSession.nextRunAt` minimum). Work due - /// within the horizon blocks TTL/warning eviction — evicting just before a scheduled advance - /// would thrash (evict → immediate reload). - public var nextScheduledWork: Date? - - public init(turnInFlight: Bool = false, - foregroundSessionActive: Bool = false, - nextScheduledWork: Date? = nil) { - self.turnInFlight = turnInFlight - self.foregroundSessionActive = foregroundSessionActive - self.nextScheduledWork = nextScheduledWork - } -} - -/// Why the policy chose to evict — carried into the log line (never user-facing UI; an automatic -/// evict is invisible-correct, the next request transparently reloads). -public enum EvictionReason: Equatable, Sendable { - case criticalPressure - case warningPressure - case idleTTL -} - -public enum EvictionVerdict: Equatable, Sendable { - case keep - case evict(EvictionReason) -} - -public enum EvictionPolicy { - - /// Scheduled work due within this window counts as "imminent" and blocks TTL/warning eviction - /// (an evict immediately followed by a scheduled reload is pure thrash). - public static let scheduledWorkHorizon: TimeInterval = 5 * 60 - - /// The warning-pressure IDLE FLOOR (`fix-evict-thrash-and-hot-path`): a resident large model - /// keeps the system at sustained `.warning` as its NORMAL state, so warning-level eviction also - /// requires no AI activity for this long. Without it, chronic warning + the coarse tick evicted - /// between active turns → a 17 GB reload per turn → the bus-saturating reload storm the user - /// felt as "late swipes and a laggy Mac". Reloads re-stamp activity, so every reload buys an - /// automatic grace window. Critical stays immediate — a real emergency outranks comfort. - public static let warningIdleFloor: TimeInterval = 5 * 60 - - /// The pure verdict. Rules (design D1): - /// - A turn or load in flight → ALWAYS keep (no trigger may interrupt work; re-evaluated next tick). - /// - `.critical` pressure → evict even with a foreground session open (system health outranks an - /// open chat; the next message pays a visible, ordinary reload). - /// - `.warning` pressure → evict only when FULLY quiescent AND idle ≥ `warningIdleFloor` - /// (chronic warning with a resident large model is normal operation, never a thrash trigger). - /// - Idle TTL (`ttl > 0`) → evict when fully quiescent AND nothing has stamped activity for `ttl`. - /// `ttl == 0` disables ONLY this trigger — pressure triggers stay armed. - /// - Fully quiescent = no turn, no foreground-active session, no scheduled work within - /// `scheduledWorkHorizon` (past-due counts as imminent: the driver just hasn't served it yet). - public static func verdict(now: Date, - lastActivity: Date, - pressure: MemoryPressureLevel, - quiescence: QuiescenceSnapshot, - ttl: TimeInterval, - loadInFlight: Bool) -> EvictionVerdict { - if quiescence.turnInFlight || loadInFlight { return .keep } - if pressure == .critical { return .evict(.criticalPressure) } - - let scheduledImminent: Bool - if let next = quiescence.nextScheduledWork { - scheduledImminent = next.timeIntervalSince(now) <= scheduledWorkHorizon - } else { - scheduledImminent = false - } - let fullyQuiescent = !quiescence.foregroundSessionActive && !scheduledImminent - - if pressure == .warning { - let idleEnough = now.timeIntervalSince(lastActivity) >= warningIdleFloor - return (fullyQuiescent && idleEnough) ? .evict(.warningPressure) : .keep - } - if ttl > 0, fullyQuiescent, now.timeIntervalSince(lastActivity) >= ttl { - return .evict(.idleTTL) - } - return .keep - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Fleet/FleetCloudEscalation.swift b/Sources/ThreeFingerSwitcher/AI/Fleet/FleetCloudEscalation.swift deleted file mode 100644 index 4a42118..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Fleet/FleetCloudEscalation.swift +++ /dev/null @@ -1,60 +0,0 @@ -import Foundation - -/// The cloud-escalation routing seam for fleet cloud members (design D5, task 6.2). A `.cloud` member -/// (Claude, GLM-5.2) is NEVER loaded; selecting one routes the turn through the EXISTING Claude-handoff -/// escalation surface (`ai-claude-handoff` §3.8 — confirm-by-default, budget-capped, audited). This slice -/// ROUTES, it does not reimplement the handoff: a production conformer bridges to the handoff launcher; -/// a recording spy keeps the routing `swift test`-verified. -/// -/// MLX-free Core. -public protocol FleetCloudEscalating: Sendable { - /// Route a turn to the cloud member `descriptor` through the handoff escalation surface. The - /// descriptor is always a `.cloud` / `.cloudEscalation` member; the conformer does the actual - /// confirm/budget/audit (this seam only carries the routing decision). - func escalate(to descriptor: ModelDescriptor) async throws -} - -/// The no-op default (a context with no handoff surface wired). Routes nowhere — a safe fallback. -public struct NoopFleetCloudEscalation: FleetCloudEscalating { - public init() {} - public func escalate(to descriptor: ModelDescriptor) async throws {} -} - -/// Gates + routes selection of a cloud member behind `fleetCloudEscalationEnabled` (design D5, task 6). -/// -/// The flag itself is OWNED by `ai-full-potential-toggle` (§D1) — this slice only CONSUMES it via an -/// injected closure (default treated as `false`, task 6.1). When off, a cloud member is not offered for -/// selection (`selectableDescriptors` filters it out) and selecting one yields `FleetError.cloudDisabled`; -/// when on, a cloud-member selection routes through the `FleetCloudEscalating` seam. -public struct FleetCloudGate: Sendable { - /// Reads the persisted `fleetCloudEscalationEnabled` (owned by `ai-full-potential-toggle`). Injected - /// so this slice never defines the flag; default treated as `false` (task 6.1). - public let isEnabled: @Sendable () -> Bool - private let escalator: FleetCloudEscalating - - public init(isEnabled: @escaping @Sendable () -> Bool = { false }, - escalator: FleetCloudEscalating = NoopFleetCloudEscalation()) { - self.isEnabled = isEnabled - self.escalator = escalator - } - - /// The members offered for selection in the Hub / executor: ALL on-device members, plus cloud members - /// ONLY when escalation is enabled. (The Hub still RENDERS cloud rows when off — see task 7.3 — but as - /// disabled/captioned; this is the set that is actually SELECTABLE.) - public func selectableDescriptors(from descriptors: [ModelDescriptor]) -> [ModelDescriptor] { - let cloudOK = isEnabled() - return descriptors.filter { $0.provider == .onDevice || cloudOK } - } - - /// Select `descriptor` as a command's model. On-device members are a no-op here (the manager handles - /// residency). A cloud member: - /// - while OFF → throws `FleetError.cloudDisabled` (an observable refusal, never silent), - /// - while ON → routes the turn through the handoff escalation seam (D5). - public func select(_ descriptor: ModelDescriptor) async throws { - guard descriptor.provider == .cloud else { return } - guard isEnabled() else { - throw FleetError.cloudDisabled(modelName: descriptor.displayName) - } - try await escalator.escalate(to: descriptor) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Fleet/FleetError.swift b/Sources/ThreeFingerSwitcher/AI/Fleet/FleetError.swift deleted file mode 100644 index b9fb05b..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Fleet/FleetError.swift +++ /dev/null @@ -1,43 +0,0 @@ -import Foundation - -/// The fleet-specific error taxonomy (design D7) — a clean `LocalizedError` for ONLY the two cases the -/// shared `RuntimeError` cannot carry. Capability-mismatch, unavailable-hardware, download/integrity -/// failures all stay `RuntimeError`; these two are genuinely fleet-specific: -/// - `.cannotAdmit` — the residency plan is INFEASIBLE (the target does not fit even after evicting every -/// evictable on-device model). The eviction list it tried rides in `evictedDetails` (opt-in copyable), -/// NEVER in the headline (raw interpolation in a headline is banned). -/// - `.cloudDisabled` — a `.cloud` member was selected while `fleetCloudEscalationEnabled` is off. -/// -/// Every case has a per-case, user-facing headline; raw lists/text ride ONLY in `details` and logs. -/// `AIError.message(for:)` (THE one translator) renders this taxonomy identically on every surface. -/// A failed admission is an observable `.failed`, never a false "loaded". MLX-free Core. -public enum FleetError: Error, Equatable { - /// The target could not be admitted under the budget even after evicting everything evictable. - /// `evictedDetails` is the (opt-in, copyable) list of what the planner tried to evict — kept OUT of - /// the headline. - case cannotAdmit(modelName: String, evictedDetails: String? = nil) - /// A cloud member was selected while cloud escalation is off (`fleetCloudEscalationEnabled` false). - case cloudDisabled(modelName: String) -} - -extension FleetError: LocalizedError { - public var errorDescription: String? { - switch self { - case let .cannotAdmit(modelName, _): - return "Not enough memory to load \(modelName) right now." - case let .cloudDisabled(modelName): - return "\(modelName) is a cloud model. Turn on cloud escalation to use it." - } - } - - /// The opt-in copyable detail (the eviction list the planner tried), for a "Show details / Copy" - /// disclosure and logs only — never the headline. `nil` when the headline already says everything. - public var copyableDetails: String? { - switch self { - case let .cannotAdmit(_, evictedDetails): - return evictedDetails - case .cloudDisabled: - return nil - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Fleet/FleetModelRegistry.swift b/Sources/ThreeFingerSwitcher/AI/Fleet/FleetModelRegistry.swift deleted file mode 100644 index d05edac..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Fleet/FleetModelRegistry.swift +++ /dev/null @@ -1,27 +0,0 @@ -import Foundation - -/// The fleet registry protocol (addendum §C1 — OWNED by `ai-model-fleet`). It evolves the model layer -/// from "one resident runtime" to "a registry with residency/eviction": every member is a -/// `ModelDescriptor`, the resident subset is queryable, and admitting a member may EVICT others under the -/// 48 GB unified-memory budget (the math lives in `ResidencyPlanner`). -/// -/// Two kinds of conformer: -/// - `FleetRoster` / `StubModelRegistry` — pure, in-memory roster views: `descriptors()` is the catalog, -/// `resident()` is the on-device-loaded subset, and `ensureResident` runs the planner + records the -/// bookkeeping WITHOUT touching real weights (so the residency math is `swift test`-verified). -/// - `ModelManager` — the live conformer: its `ensureResident` runs the SAME plan, then evicts each -/// planned id and loads the target through the EXISTING `ModelProvisioner` / `runtimeFactory` (D4). -/// -/// `ensureResident` is `async` here (the live load is async); the pure roster conformers satisfy it -/// synchronously. Cloud members (`provider: .cloud`) are in `descriptors()` (selectable/visible) but -/// NEVER in `resident()` and never loaded — `ensureResident` of a cloud id is a residency no-op. -public protocol ModelRegistry: Sendable { - /// Every known member, including cloud members (so selection + the Hub roster see them all). - func descriptors() -> [ModelDescriptor] - /// The on-device members currently resident in memory. Never includes a `.cloud` member. - func resident() -> [ModelDescriptor] - /// Admit `id`, evicting whatever the residency plan names to make room. Throws - /// `FleetError.cannotAdmit` when the target cannot fit even after evicting every evictable on-device - /// model. A cloud id is a residency no-op (cost 0, never resident). May EVICT under the 48 GB budget. - func ensureResident(_ id: String) async throws -} diff --git a/Sources/ThreeFingerSwitcher/AI/Fleet/FleetRoster.swift b/Sources/ThreeFingerSwitcher/AI/Fleet/FleetRoster.swift deleted file mode 100644 index b729d0d..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Fleet/FleetRoster.swift +++ /dev/null @@ -1,263 +0,0 @@ -import Foundation - -/// The standard fleet roster (design D2) — the §C1 `ModelRegistry` conformer that lists the V2.5 fleet: -/// - **chat** — Gemma on the GPU lane (the default, today's single model). -/// - **ternary** — the small CPU-lane model (`.cpuTernary`, ~0.5 GB resident), co-resides with chat. -/// - **image (Q4)** — a 4-bit generative-image backend (~7 GB), CO-RESIDES with chat + ternary + KV. -/// - **image (FP16)** — the full-precision image variant (~24 GB resident), EVICTS chat. -/// - **video** — a generative-video backend, EVICTS chat (the companion goes quiet while it paints). -/// - **Claude** + **GLM-5.2** — the two CLOUD members (`provider: .cloud`, `role: .cloudEscalation`, -/// `residencyBytes: 0`): visible/selectable in `descriptors()` for the Hub roster + escalation -/// routing, but NEVER in `resident()` and never loaded. GLM-5.2 (753B MoE / 1M ctx / MIT) does not fit -/// 48 GB — local residency is not an option. -/// -/// `descriptors()` returns ALL members (cloud included). `resident()` returns only the on-device members -/// currently loaded (cloud is never resident). `ensureResident` runs the `ResidencyPlanner` and records -/// the resident bookkeeping IN MEMORY — it does NOT touch real weights; the live load is `ModelManager`'s -/// job (D4). So FleetRoster is the `swift test`-verified roster + planning view, and ModelManager is the -/// live one. A fleet-of-one is `FleetRoster(members: [chat])` — trivially today's behavior. -/// -/// MLX-free Core. Only the user's stable-signed build proves the REAL residency/eviction with real -/// weights (does Video actually evict chat + reload? does Q4 image truly co-reside under 48 GB?). -public final class FleetRoster: ModelRegistry, @unchecked Sendable { - - private let members: [ModelDescriptor] - private let planner: ResidencyPlanner - /// The unified-memory budget the planner spends against (the 48 GB target by default). - private let budgetBytes: UInt64 - /// Injected live free-memory probe (mirrors `LaneResidencyBudget`); pure value in tests. - private let freeBytesProbe: @Sendable () -> UInt64 - - /// Resident bookkeeping (ids loaded). Mutated by `ensureResident` — in-memory only, NO real weights. - private let lock = NSLock() - private var residentIDs: [String] = [] - - public init(members: [ModelDescriptor], - planner: ResidencyPlanner = ResidencyPlanner(), - budgetBytes: UInt64 = FleetRoster.unifiedBudget48GB, - freeBytesProbe: @escaping @Sendable () -> UInt64 = { FleetRoster.unifiedBudget48GB }) { - self.members = members - self.planner = planner - self.budgetBytes = budgetBytes - self.freeBytesProbe = freeBytesProbe - } - - /// The 48 GB unified-memory budget on the M5 Pro target (the hard, shared budget this slice encodes). - public static let unifiedBudget48GB: UInt64 = 48 * 1024 * 1024 * 1024 - - // MARK: - ModelRegistry - - public func descriptors() -> [ModelDescriptor] { members } - - public func resident() -> [ModelDescriptor] { - lock.lock(); defer { lock.unlock() } - let byID = Dictionary(uniqueKeysWithValues: members.map { ($0.id, $0) }) - // Cloud members can never be resident, even if an id slipped in — filter defensively. - return residentIDs.compactMap { byID[$0] }.filter { $0.provider == .onDevice } - } - - public func ensureResident(_ id: String) async throws { - guard let target = members.first(where: { $0.id == id }) else { - throw RuntimeError.modelMissing - } - // Cloud target → residency no-op (it routes to escalation; this view never loads it). - if target.provider == .cloud { return } - - let plan = planner.plan(target: id, - descriptors: members, - budgetBytes: budgetBytes, - freeBytes: freeBytesProbe(), - currentlyResident: snapshotResident()) - if plan.infeasible { - throw FleetError.cannotAdmit(modelName: target.displayName, - evictedDetails: plan.evict.isEmpty ? nil - : "Tried to evict: \(plan.evict.joined(separator: ", "))") - } - // Apply the plan to the in-memory bookkeeping (NO real weights here). - lock.lock() - residentIDs.removeAll { plan.evict.contains($0) } - for admitted in plan.admit where !residentIDs.contains(admitted) { - residentIDs.append(admitted) - } - lock.unlock() - } - - private func snapshotResident() -> [String] { - lock.lock(); defer { lock.unlock() } - return residentIDs - } - - // MARK: - Selection helpers (parity with the value catalog) - - /// Look up a member by id. - public func descriptor(id: String) -> ModelDescriptor? { - members.first { $0.id == id } - } - - /// Capability-based selection over the ON-DEVICE members (cloud members are escalation targets, never - /// capability-selected here): the first member satisfying ALL required capabilities, or a clear - /// failure. Preserves the value catalog's behavior so the executor's `runtime(requiring:)` still - /// finds the chat model. - public func selectModel(requiring required: Set) throws -> ModelDescriptor { - let candidates = members.filter { $0.provider == .onDevice && required.isSubset(of: $0.capabilities) } - guard let first = candidates.first else { - let names = required.map(\.rawValue).sorted().joined(separator: ", ") - throw RuntimeError.unavailable(reason: "No registered model satisfies required capabilities: [\(names)]") - } - return first - } - - // MARK: - The standard fleet - - /// The built-in V2.5 fleet. Chat + ternary + Q4 image co-reside under 48 GB; FP16 image + video evict - /// chat; Claude + GLM-5.2 are cloud-only. Resident-byte numbers are honest approximations of the real - /// footprints (the user's stable-signed build verifies the live bytes). - public static let standard = FleetRoster(members: [ - // chat — GPU lane, ~17 GB resident (today's default model). - ModelDescriptor( - id: "gemma-4-31b", - displayName: "Gemma 4 31B (chat, text + vision, 4-bit)", - sizeBytes: 17 * 1024 * 1024 * 1024, - integritySHA: "hub-verified", - downloadURL: URL(string: "https://huggingface.co/mlx-community/gemma-4-31b-it-4bit")!, - capabilities: [.text, .vision], - quantization: .qat4bit, - maxContextTokens: 131_072, - role: .chat, - lane: .gpu, - provider: .onDevice, - residencyBytes: 17 * 1024 * 1024 * 1024 - ), - // ternary — CPU lane, ~0.5 GB resident, co-resides (bandwidth-frugal; ~32× smaller weights). - ModelDescriptor( - id: "ternary-cpu-chat", - displayName: "Ternary CPU model (routing / classify, ~0.5 GB)", - sizeBytes: 512 * 1024 * 1024, - integritySHA: "hub-verified", - downloadURL: URL(string: "https://huggingface.co/mlx-community/ternary-cpu-chat")!, - capabilities: [.text], - quantization: .int8, - maxContextTokens: 8_192, - role: .ternaryChat, - lane: .cpuTernary, - provider: .onDevice, - residencyBytes: 512 * 1024 * 1024 - ), - // image (Q4) — GPU lane, ~7 GB resident, CO-RESIDES with chat + ternary + KV under 48 GB. - ModelDescriptor( - id: "image-q4", - displayName: "FLUX.2 Klein 4B (4-bit, ~7 GB) — co-resides with chat", - sizeBytes: 7 * 1024 * 1024 * 1024, - integritySHA: "hub-verified", - downloadURL: URL(string: "https://huggingface.co/black-forest-labs/FLUX.2-klein-4B")!, - capabilities: [.text, .vision], - quantization: .qat4bit, - maxContextTokens: 8_192, - role: .image, - lane: .gpu, - provider: .onDevice, - residencyBytes: 7 * 1024 * 1024 * 1024 - ), - // image (FP16) — GPU lane, ~24 GB resident, EVICTS chat (above the FP16 threshold). - ModelDescriptor( - id: "image-fp16", - displayName: "FLUX.2 Klein 4B (bf16, ~24 GB) — pauses chat while it paints", - sizeBytes: 24 * 1024 * 1024 * 1024, - integritySHA: "hub-verified", - downloadURL: URL(string: "https://huggingface.co/black-forest-labs/FLUX.2-klein-4B")!, - capabilities: [.text, .vision], - quantization: .fp16, - maxContextTokens: 8_192, - role: .image, - lane: .gpu, - provider: .onDevice, - residencyBytes: 24 * 1024 * 1024 * 1024 - ), - // video — GPU lane, ~24 GB resident, EVICTS chat (always evicts regardless of size). - ModelDescriptor( - id: "video-ltxv", - displayName: "Video model (LTXV) — pauses chat while it animates", - sizeBytes: 24 * 1024 * 1024 * 1024, - integritySHA: "hub-verified", - downloadURL: URL(string: "https://huggingface.co/mlx-community/video-ltxv")!, - capabilities: [.text, .vision], - quantization: .fp16, - maxContextTokens: 8_192, - role: .video, - lane: .gpu, - provider: .onDevice, - residencyBytes: 24 * 1024 * 1024 * 1024 - ), - // Claude — CLOUD escalation member, never resident (residencyBytes 0, lane nil). - ModelDescriptor( - id: "claude-cloud", - displayName: "Claude (cloud) — escalation, not resident locally", - sizeBytes: 0, - integritySHA: "cloud", - downloadURL: URL(string: "https://api.anthropic.com")!, - capabilities: [.text, .vision], - quantization: .bf16, - maxContextTokens: 200_000, - role: .cloudEscalation, - lane: nil, - provider: .cloud, - residencyBytes: 0 - ), - // GLM-5.2 — CLOUD escalation member (753B MoE / 1M ctx / MIT). Does NOT fit 48 GB — cloud only. - ModelDescriptor( - id: "glm-5.2-cloud", - displayName: "GLM-5.2 (cloud) — 753B MoE, 1M context, MIT (escalation only)", - sizeBytes: 0, - integritySHA: "cloud", - downloadURL: URL(string: "https://open.bigmodel.cn")!, - capabilities: [.text, .vision], - quantization: .bf16, - maxContextTokens: 1_000_000, - role: .cloudEscalation, - lane: nil, - provider: .cloud, - residencyBytes: 0 - ) - ]) -} - -/// A test-only roster (design D2, task 2.3) that scripts an ARBITRARY set of members — including the -/// degenerate fleet-of-one (just the chat descriptor → today's behavior). A thin wrapper over -/// `FleetRoster` so the conformance + planning are exercised identically to production. -public final class StubModelRegistry: ModelRegistry, @unchecked Sendable { - private let roster: FleetRoster - - public init(members: [ModelDescriptor], - planner: ResidencyPlanner = ResidencyPlanner(), - budgetBytes: UInt64 = FleetRoster.unifiedBudget48GB, - freeBytesProbe: @escaping @Sendable () -> UInt64 = { FleetRoster.unifiedBudget48GB }) { - self.roster = FleetRoster(members: members, planner: planner, - budgetBytes: budgetBytes, freeBytesProbe: freeBytesProbe) - } - - /// A fleet-of-one: a single chat descriptor (today's single-model shape, with fleet defaults). - public static func fleetOfOne(id: String = "gemma-4-31b") -> StubModelRegistry { - StubModelRegistry(members: [ - ModelDescriptor( - id: id, - displayName: "Gemma 4 (chat)", - sizeBytes: 17 * 1024 * 1024 * 1024, - integritySHA: "hub-verified", - downloadURL: URL(string: "https://huggingface.co/mlx-community/gemma-4-31b-it-4bit")!, - capabilities: [.text, .vision], - quantization: .qat4bit, - role: .chat, - lane: .gpu, - provider: .onDevice, - residencyBytes: 17 * 1024 * 1024 * 1024) - ]) - } - - public func descriptors() -> [ModelDescriptor] { roster.descriptors() } - public func resident() -> [ModelDescriptor] { roster.resident() } - public func ensureResident(_ id: String) async throws { try await roster.ensureResident(id) } - public func descriptor(id: String) -> ModelDescriptor? { roster.descriptor(id: id) } - public func selectModel(requiring required: Set) throws -> ModelDescriptor { - try roster.selectModel(requiring: required) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Fleet/ResidencyPlanner.swift b/Sources/ThreeFingerSwitcher/AI/Fleet/ResidencyPlanner.swift deleted file mode 100644 index 35d5f66..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Fleet/ResidencyPlanner.swift +++ /dev/null @@ -1,180 +0,0 @@ -import Foundation - -/// The plan for admitting one target model under the 48 GB unified-memory budget (design D3). PURE DATA, -/// not a side effect: `ModelManager` is the only place that APPLIES it (evict → load). Returning the plan -/// as data lets the Hub PREVIEW the cost ("selecting Video pauses chat") before the user commits. -public struct ResidencyPlan: Equatable, Sendable { - /// Ids to LOAD to satisfy the request (the target — and, on a fleet-of-one / co-resident plan, the - /// target alone; never re-admits already-resident members). - public var admit: [String] - /// Ids to EVICT to make room for the target (smallest-GPU-victim-first; empty when the target - /// co-resides). The CPU-lane ternary is never here for a GPU gen (different lane, negligible bytes). - public var evict: [String] - /// The resident set AFTER applying the plan (the post-admission, post-eviction occupants). The Hub - /// reads this to render who survives; a member in `evict` is absent here. - public var coResident: [String] - /// True when the target cannot fit even after evicting every evictable on-device model. The caller - /// (`ModelManager.ensureResident`) maps this to `FleetError.cannotAdmit` — an observable `.failed`, - /// never a false `.loaded`. - public var infeasible: Bool - - public init(admit: [String] = [], evict: [String] = [], coResident: [String] = [], - infeasible: Bool = false) { - self.admit = admit - self.evict = evict - self.coResident = coResident - self.infeasible = infeasible - } -} - -/// The pure residency / eviction MATH (design D3). Given the fleet descriptors, the unified-memory budget, -/// the live free bytes (INJECTED — no Metal here), the currently-resident set, and a target, it computes -/// what to admit + evict. It NEVER calls Metal or the provisioner; `ModelManager` applies the plan. -/// -/// The hard physical fact it encodes: 48 GB is a shared budget. Chat (~17 GB GPU) + ternary (~0.5 GB CPU) -/// + a Q4 image model (~7 GB) + the live KV reserve CO-RESIDE; a video gen or an FP16 image model (~24 GB) -/// CANNOT, and evicts chat. Cloud members never fit and never try. -public struct ResidencyPlanner: Sendable { - - /// The KV-cache headroom reserved alongside the resident weight set (the live decode caches that grow - /// with concurrency). Held out of the budget so a co-residency plan leaves room for generation, never - /// packs weights edge-to-edge. Injectable for tests; a sane default for the 48 GB target. - public var kvReserveBytes: UInt64 - - /// The resident-footprint threshold above which an `.image` member is treated as the heavy (FP16) - /// variant that EVICTS chat rather than co-residing. A Q4 image model sits below it (co-resides); an - /// FP16 image model sits above it (evicts). `.video` always evicts regardless of size. - public var fp16ImageThresholdBytes: UInt64 - - public init(kvReserveBytes: UInt64 = 4 * 1024 * 1024 * 1024, // ~4 GB KV headroom - fp16ImageThresholdBytes: UInt64 = 16 * 1024 * 1024 * 1024) { // > 16 GB resident ⇒ heavy - self.kvReserveBytes = kvReserveBytes - self.fp16ImageThresholdBytes = fp16ImageThresholdBytes - } - - /// Compute the plan to admit `target` under `budgetBytes`, given the live `freeBytes`, the catalog of - /// `descriptors`, and the ids `currentlyResident`. - /// - /// Rules (D3), encoded once and tested: - /// 1. **Cloud target** (`provider: .cloud`) → empty admit/evict (cost 0, never resident). - /// 2. **Already resident** → empty admit/evict (warm; nothing to do). - /// 3. **Co-residency:** the target + the already-resident on-device members + the KV reserve fit - /// within the budget AND the live free bytes → admit the target, evict nothing. - /// 4. **Eviction:** otherwise free room by evicting GPU-lane occupants smallest-victim-first (chat is - /// the expected victim) until the target + survivors + KV reserve fit. The CPU-lane ternary is - /// never evicted for a GPU gen (different lane, negligible bytes). - /// 5. **Infeasible:** if the target cannot fit even after evicting every evictable GPU-lane occupant - /// → `infeasible: true` (the caller throws `FleetError.cannotAdmit`). - public func plan(target targetID: String, - descriptors: [ModelDescriptor], - budgetBytes: UInt64, - freeBytes: UInt64, - currentlyResident: [String]) -> ResidencyPlan { - - let byID = Dictionary(uniqueKeysWithValues: descriptors.map { ($0.id, $0) }) - guard let target = byID[targetID] else { - // Unknown target → nothing to admit; not infeasible (the caller surfaces "unknown" upstream). - return ResidencyPlan(coResident: currentlyResident) - } - - // (1) Cloud target: never resident, cost 0 — a residency no-op (it routes to escalation, D5). - if target.provider == .cloud { - return ResidencyPlan(admit: [], evict: [], coResident: currentlyResident, infeasible: false) - } - - // The on-device resident set (cloud ids can never be resident, but filter defensively). - let residentDescriptors = currentlyResident.compactMap { byID[$0] }.filter { $0.provider == .onDevice } - - // (2) Already resident → warm, nothing to do. - if residentDescriptors.contains(where: { $0.id == target.id }) { - return ResidencyPlan(admit: [], evict: [], - coResident: residentDescriptors.map(\.id), infeasible: false) - } - - // The effective ceiling is the smaller of the declared budget and the live free bytes — never - // plan beyond what the machine actually has free RIGHT NOW. - let ceiling = min(budgetBytes, freeBytes) - - func footprint(_ ids: [String]) -> UInt64 { - ids.reduce(UInt64(0)) { $0 &+ (byID[$1]?.residencyBytes ?? 0) } - } - - // Is the target a HEAVY GPU generation (`.video`, or an `.image` above the FP16 threshold)? Such - // a target EVICTS the GPU-lane chat even when the bytes would technically fit — the GPU lane - // cannot run a heavy diffusion AND stream chat concurrently (~153 GB/s bus contention; the - // companion goes quiet while it paints, documented). So a heavy target SKIPS the co-residency - // fast-path and goes straight to eviction. Q4 image (below the threshold) co-resides. - let isHeavyGPUGen: Bool = { - guard target.lane == .gpu else { return false } - switch target.role { - case .video: return true - case .image: return target.residencyBytes > fp16ImageThresholdBytes - case .chat, .ternaryChat, .cloudEscalation: return false - } - }() - - // (3) Co-residency: a NON-heavy target that fits alongside ALL current residents + the KV reserve - // co-resides (no eviction). Heavy GPU gens never take this path. - let coResidentIDs = residentDescriptors.map(\.id) - let withTarget = footprint(coResidentIDs) &+ target.residencyBytes &+ kvReserveBytes - if !isHeavyGPUGen && withTarget <= ceiling { - return ResidencyPlan(admit: [target.id], evict: [], - coResident: coResidentIDs + [target.id], infeasible: false) - } - - // (4) Eviction. Only GPU-lane occupants are candidates; the CPU-lane ternary ALWAYS survives a GPU - // gen (different lane, negligible bytes). Two sub-cases: - // • HEAVY GPU gen → the GPU lane is exclusive to the heavy gen: evict EVERY other GPU - // occupant (chat is the expected victim), regardless of byte-fit. Then the target must - // still fit byte-wise against the survivors (ternary only) + KV. - // • Non-heavy over-budget target → free bytes by evicting GPU occupants smallest-victim-first - // until the target fits. - let gpuOccupants = residentDescriptors.filter { $0.lane == .gpu } - let survivorsBase = residentDescriptors.filter { $0.lane != .gpu }.map(\.id) - - var evicted: [String] = [] - var remainingVictims = gpuOccupants.sorted { $0.residencyBytes < $1.residencyBytes } // smallest-first - - func fits() -> Bool { - let survivors = survivorsBase + remainingVictims.map(\.id) - return footprint(survivors) &+ target.residencyBytes &+ kvReserveBytes <= ceiling - } - - if isHeavyGPUGen { - // Lane exclusivity: every other GPU occupant goes, regardless of byte-fit. - evicted = remainingVictims.map(\.id) - remainingVictims = [] - } else { - // Byte pressure: shed smallest GPU victims until the target fits. - while !fits() && !remainingVictims.isEmpty { - evicted.append(remainingVictims.removeFirst().id) - } - } - - if fits() { - let survivors = survivorsBase + remainingVictims.map(\.id) - return ResidencyPlan(admit: [target.id], evict: evicted, - coResident: survivors + [target.id], infeasible: false) - } - - // (5) Infeasible: even after evicting every evictable GPU occupant the target does not fit. - return ResidencyPlan(admit: [], evict: evicted, - coResident: survivorsBase, infeasible: true) - } - - // MARK: - Honest cost preview (Hub) - - /// Whether admitting `targetID` would EVICT the GPU-lane chat — the Hub reads this (computed from the - /// plan, NOT hard-coded) to render the inline "selecting ‹Role› pauses the chat model" disclosure - /// (design D6, task 7.2). True iff the plan evicts a `.chat` member. - public func admissionEvictsChat(targetID: String, - descriptors: [ModelDescriptor], - budgetBytes: UInt64, - freeBytes: UInt64, - currentlyResident: [String]) -> Bool { - let p = plan(target: targetID, descriptors: descriptors, budgetBytes: budgetBytes, - freeBytes: freeBytes, currentlyResident: currentlyResident) - let byID = Dictionary(uniqueKeysWithValues: descriptors.map { ($0.id, $0) }) - return p.evict.contains { byID[$0]?.role == .chat } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/ForwardingLLMRuntime.swift b/Sources/ThreeFingerSwitcher/AI/ForwardingLLMRuntime.swift deleted file mode 100644 index a5db0fa..0000000 --- a/Sources/ThreeFingerSwitcher/AI/ForwardingLLMRuntime.swift +++ /dev/null @@ -1,68 +0,0 @@ -import Foundation - -/// A lazily-resolving `LLMRuntime` adapter (`wire-memory-skills`). The `SkillToolProvider` requires a -/// concrete `runtime` at construction so it can generate the skill's text result, but the live model is -/// loaded lazily through `ModelManager.runtime(requiring:)` and may not be resident when the tool -/// registry is built. This forwarder defers resolution to call time: each `generate`/`structured`/`chat` -/// resolves the resident runtime (kept resident between calls, so this is cheap) and forwards. -/// -/// MLX-free Core — it depends only on the `LLMRuntime` seam and an injected `@Sendable` resolver, never on -/// Gemma/MLX. The resolver throws `RuntimeError` when no model is loaded; that surfaces as a clean -/// `.failed` step through the established taxonomy (never raw text, never a false "Done"). -struct ForwardingLLMRuntime: LLMRuntime { - /// Resolve the resident runtime for the given required modalities (typically `modelManager.runtime`). - let resolve: @Sendable (Set) async throws -> LLMRuntime - - /// Advertised capabilities. The forwarder claims text+vision so a vision-capable skill isn't filtered - /// out before resolution; the real runtime enforces its own capabilities at resolve time. - var capabilities: Set { [.text, .vision] } - - func generate(_ request: LLMRequest) -> AsyncThrowingStream { - let caps: Set = request.image == nil ? [.text] : [.text, .vision] - let resolve = self.resolve - return AsyncThrowingStream { continuation in - let task = Task { - do { - let runtime = try await resolve(caps) - for try await token in runtime.generate(request) { - continuation.yield(token) - } - continuation.finish() - } catch { - continuation.finish(throwing: error) - } - } - continuation.onTermination = { _ in task.cancel() } - } - } - - func structured( - _ request: LLMRequest, - schema: StructuredSchema, - as type: T.Type - ) async throws -> StructuredOutcome { - let caps: Set = request.image == nil ? [.text] : [.text, .vision] - let runtime = try await resolve(caps) - return try await runtime.structured(request, schema: schema, as: type) - } - - func chat(_ request: LLMChatRequest) -> AsyncThrowingStream { - let needsVision = !request.effectiveImages.isEmpty - let caps: Set = needsVision ? [.text, .vision] : [.text] - let resolve = self.resolve - return AsyncThrowingStream { continuation in - let task = Task { - do { - let runtime = try await resolve(caps) - for try await token in runtime.chat(request) { - continuation.yield(token) - } - continuation.finish() - } catch { - continuation.finish(throwing: error) - } - } - continuation.onTermination = { _ in task.cancel() } - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/FullPotential/AIGatingSnapshot.swift b/Sources/ThreeFingerSwitcher/AI/FullPotential/AIGatingSnapshot.swift deleted file mode 100644 index 423f0a2..0000000 --- a/Sources/ThreeFingerSwitcher/AI/FullPotential/AIGatingSnapshot.swift +++ /dev/null @@ -1,68 +0,0 @@ -import Foundation -import os - -// MARK: - Thread-safe live gating snapshot (MLX-free Core) -// -// The route loop (`AgentLoop.run`) is a NON-isolated `Sendable` struct: its `descriptors()` / -// `registry.run` / contributor calls execute on the cooperative thread pool, NOT the main actor. The -// Full Potential gating flags + the cloud-video budget cap, however, live on the `@MainActor` -// `AppSettings`. Reading them from the loop's `@Sendable` closures via `MainActor.assumeIsolated` -// TRAPS (EXC_BREAKPOINT) off the main actor — that crashes the first AI command that routes through -// the tool loop (the media contributor's availability gate is read while ranking/dispatching tools). -// -// `AIGatingSnapshot` is the safe-from-any-thread mirror that fixes it WITHOUT losing live semantics: -// the main actor writes the few gating values into it whenever the relevant `AppSettings` flags change -// (`refresh(from:)`, driven by Combine observation), and the loop's `@Sendable` closures read them -// thread-safely through an `OSAllocatedUnfairLock`. So media / cloud gates still appear/disappear LIVE -// as the user toggles Full Potential or its sub-flags — never a stale snapshot frozen at registry-build -// time (when everything is off), and never an off-main trap. - -/// A thread-safe live mirror of the AI route loop's `AppSettings`-derived gating inputs. The main actor -/// refreshes it on every relevant settings change; the route loop's `@Sendable` closures read it off-main -/// without trapping. `internal` (it consumes the internal `AppSettings`); the gating values it exposes are -/// the same the (public) `FullPotentialGate` resolves. -final class AIGatingSnapshot: Sendable { - /// The plain value the lock guards — every gating read the off-main route loop needs. - private struct Values: Sendable { - var fullPotentialEnabled = false - var mediaGenUnlocked = false - var fleetCloudUnlocked = false - var mediaVideoBudgetPerDay = 0 - } - - private let storage = OSAllocatedUnfairLock(initialState: Values()) - - init() {} - - // MARK: Refresh (main-actor writer) - - /// Mirror the live Full Potential gates + cloud-video budget into the snapshot. Called on the main - /// actor on every relevant settings change (and once at start) so the off-main reads see the current - /// values. Each gate routes through the SINGLE resolver (`FullPotentialGate.isUnlocked`), so a master / - /// ai-commands OFF closes media + cloud at once (the calm panic-off), never just their own sub-flag. - @MainActor - func refresh(from settings: AppSettings) { - let gate = settings.fullPotentialGate - let full = settings.fullPotentialEnabled - let media = gate.isUnlocked(.mediaGen) - let cloud = gate.isUnlocked(.fleetCloud) - let budget = settings.mediaVideoBudgetPerDay - storage.withLock { - $0.fullPotentialEnabled = full - $0.mediaGenUnlocked = media - $0.fleetCloudUnlocked = cloud - $0.mediaVideoBudgetPerDay = budget - } - } - - // MARK: Reads (safe from any thread — no `assumeIsolated`) - - /// The master Full Potential gate (raw — the media contributor ANDs it with `isMediaGenUnlocked`). - var isFullPotentialEnabled: Bool { storage.withLock { $0.fullPotentialEnabled } } - /// Whether the `mediaGen` capability is unlocked through the single resolver (master ∧ ai-commands ∧ sub). - var isMediaGenUnlocked: Bool { storage.withLock { $0.mediaGenUnlocked } } - /// Whether the `fleetCloud` capability is unlocked through the single resolver. - var isFleetCloudUnlocked: Bool { storage.withLock { $0.fleetCloudUnlocked } } - /// The live cloud-video per-day budget cap. - var mediaVideoBudgetPerDay: Int { storage.withLock { $0.mediaVideoBudgetPerDay } } -} diff --git a/Sources/ThreeFingerSwitcher/AI/FullPotential/FullPotentialGate.swift b/Sources/ThreeFingerSwitcher/AI/FullPotential/FullPotentialGate.swift deleted file mode 100644 index 7894347..0000000 --- a/Sources/ThreeFingerSwitcher/AI/FullPotential/FullPotentialGate.swift +++ /dev/null @@ -1,96 +0,0 @@ -import Foundation - -// MARK: - Full Potential gate (MLX-free Core) -// -// The single source of truth for "is heavy AI capability X actually unlocked." V2.5 evolves the -// on-device agent into a two-lane, multi-model, media-generating fleet (see -// `docs/ai-agent-v2-addendum-compute-media-fleet.md` §D1) — but the project ships **calm**: every -// heavy capability is OFF until the user performs one deliberate act ("Release Full Potential") and -// then opts into the specific capability, cost disclosed in the same breath. -// -// This file owns the master gate's *gating logic only* — a pure, total `Bool` resolver every heavy -// slice consults before activating. It does NOT redefine sibling-owned types (`ComputeLane`/ -// `LaneRouting` §A1, `MediaRuntime` §B1, `ModelRegistry`/`ModelDescriptor` §C1, `WritePolicyTier`): -// the enum below only NAMES the capabilities the flags gate. The persisted flags live in -// `AppSettings`; the adapter (`AppSettings.fullPotentialGate`) maps them into the gate's input. - -/// The five heavy AI capabilities the Full Potential flags gate, one case per sub-flag (addendum §D1). -/// `CaseIterable` so the Hub renders the five rows by iterating and a test can assert every case is -/// gated; `Codable`/`Sendable` so it can ride any persistence or cross-actor boundary. The `rawValue` -/// strings are stable identifiers (do not rename — they may key UI/telemetry). -public enum FullPotentialCapability: String, CaseIterable, Codable, Sendable { - /// The CPU ternary lane (`ai-compute-tiers` → `cpuLaneEnabled`). - case cpuLane - /// The K-stream GPU batched runtime + growable context (`ai-batched-runtime-and-context` → - /// `batchedRuntimeEnabled`). - case batchedRuntime - /// Image/video generation tools (`ai-media-runtime` + backends → `mediaGenEnabled`). - case mediaGen - /// Parked auto-vs-escalate + whitelist + audit (`ai-background-autonomy` → `backgroundAutonomyEnabled`). - case backgroundAutonomy - /// Cloud fleet members — Claude / GLM-5.2 (`ai-model-fleet` cloud members → `fleetCloudEscalationEnabled`). - case fleetCloud -} - -/// The gate's pure input: the AI-commands opt-in, the master, and the five per-capability sub-flags. -/// A plain value type (`Equatable`/`Sendable`) assembled from `AppSettings` by the adapter; the gate -/// reads it, never mutates it (turning the master off RELOCKS by computation, RETAINING the stored -/// sub-flag values — design Decision 2). -public struct FullPotentialFlags: Equatable, Sendable { - /// The existing AI feature opt-in (`AppSettings.aiCommandsEnabled`). The fleet is a strict subset - /// of the AI feature — if this is off there is no resident model at all, so every fleet capability - /// is meaningless and locked. - public var aiCommandsEnabled: Bool - /// The master "Release Full Potential" gate (`fullPotentialEnabled`). Off ⇒ every capability locked - /// (the calm panic-off). - public var fullPotentialEnabled: Bool - - public var cpuLane: Bool - public var batchedRuntime: Bool - public var mediaGen: Bool - public var backgroundAutonomy: Bool - public var fleetCloud: Bool - - public init(aiCommandsEnabled: Bool, - fullPotentialEnabled: Bool, - cpuLane: Bool, - batchedRuntime: Bool, - mediaGen: Bool, - backgroundAutonomy: Bool, - fleetCloud: Bool) { - self.aiCommandsEnabled = aiCommandsEnabled - self.fullPotentialEnabled = fullPotentialEnabled - self.cpuLane = cpuLane - self.batchedRuntime = batchedRuntime - self.mediaGen = mediaGen - self.backgroundAutonomy = backgroundAutonomy - self.fleetCloud = fleetCloud - } -} - -/// The pure, total Full Potential resolver. The single boolean check each heavy slice consults before -/// activating: a capability is unlocked **only when** `aiCommandsEnabled ∧ fullPotentialEnabled ∧ its -/// own sub-flag`. No throw, no async, no IO, no model linkage — callable on any thread, in any sink. -public struct FullPotentialGate: Sendable { - public let flags: FullPotentialFlags - - public init(flags: FullPotentialFlags) { - self.flags = flags - } - - /// Resolve whether `capability` is unlocked. Returns `false` (locked) whenever the AI-commands - /// opt-in is off OR the master is off — closing every gate at once (the calm panic-off) — and - /// otherwise gates each capability on ONLY its own sub-flag. - public func isUnlocked(_ capability: FullPotentialCapability) -> Bool { - // master closed → every capability closed (the calm panic-off); ai-commands closed → likewise - // (the fleet is a strict subset of the AI feature, which owns the resident model). - guard flags.aiCommandsEnabled, flags.fullPotentialEnabled else { return false } - switch capability { - case .cpuLane: return flags.cpuLane - case .batchedRuntime: return flags.batchedRuntime - case .mediaGen: return flags.mediaGen - case .backgroundAutonomy: return flags.backgroundAutonomy - case .fleetCloud: return flags.fleetCloud - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Handoff/ClaudeHandoffContributor.swift b/Sources/ThreeFingerSwitcher/AI/Handoff/ClaudeHandoffContributor.swift deleted file mode 100644 index ea52425..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Handoff/ClaudeHandoffContributor.swift +++ /dev/null @@ -1,315 +0,0 @@ -import Foundation - -/// The mutable, thread-safe budget holder the contributor records spend against (`ai-claude-handoff`). -/// `HandoffBudget` is a PURE value type; this reference wrapper owns the single mutable instance + its -/// persistence so the `Sendable` contributor can `record`/`reap`/`refund` across an `await` without -/// data races. Reads (`allows`) snapshot under the lock. MLX-free Core. -final class HandoffBudgetBox: @unchecked Sendable { - private let lock = NSLock() - private var budget: HandoffBudget - private let store: HandoffLedgerStore - - init(maxCallsPerDay: Int, maxConcurrent: Int = 1, store: HandoffLedgerStore = InMemoryHandoffLedgerStore()) { - self.store = store - // Seed the ledger from disk so the rolling-window cap survives a relaunch. - self.budget = HandoffBudget(maxCallsPerDay: maxCallsPerDay, maxConcurrent: maxConcurrent, - ledger: store.load()) - } - - func allows(now: Date) -> Bool { - lock.lock(); defer { lock.unlock() } - return budget.allows(now: now) - } - - func record(at: Date, skillID: String?) { - lock.lock(); defer { lock.unlock() } - budget.record(at: at, skillID: skillID) - store.save(budget.ledger) - } - - func reap() { - lock.lock(); defer { lock.unlock() } - budget.reap() - } - - func refund(at: Date) { - lock.lock(); defer { lock.unlock() } - budget.refund(at: at) - store.save(budget.ledger) - } - - /// A snapshot for assertions / tests. - func snapshot() -> HandoffBudget { - lock.lock(); defer { lock.unlock() } - return budget - } -} - -/// The resolved effective gate for one handoff call (`ai-claude-handoff`, design Decision 4). The -/// descriptor is ALWAYS `.dangerous`; the per-skill `auto` opt-in is an effective-tier downgrade resolved -/// HERE (never a descriptor change), and only WITHIN what the user whitelist permits. -enum HandoffGate: Equatable { - /// The skill carries handoff but it is off (or the resolved cap is 0) → declined, no spend. - case disabled - /// Skill `auto` AND under budget AND whitelist permits → run now, no per-call approval (still audited). - case autoRun - /// Default-confirm (or whitelist keeps it dangerous) → foreground DOWN=approve / RIGHT=skip. - case needsApproval - /// Over the daily cap → degrade to a foreground confirm (active) / needs-you (parked); never auto-run. - case overBudget -} - -/// The `launch_claude` capability as a `ToolContributor` (`ai-claude-handoff`, design Decision 2–4). It -/// projects a single `.dangerous` descriptor and, on `run`, resolves the effective cost gate (skill -/// confirm-mode ∩ user whitelist ∩ budget), audits exactly one outcome, and — when permitted — reuses the -/// open-claude-here `.command` handoff to fire Claude FIRE-AND-FORGET. The round-trip (consume Claude's -/// output, resume the conversation) is a documented future; the seam is left, the behavior is not built. -/// MLX-free Core; `swift test`-verified with a fake launcher + scripted gate + recording audit/escalation. -struct ClaudeHandoffContributor: ToolContributor { - /// The active skill's handoff config (or `.init()` for a default-confirm free handoff). - let config: ClaudeHandoffConfig - /// The mutable, persisted budget the call records against. - let budget: HandoffBudgetBox - /// The side-effecting spawn (production = `OpenClaudeHandoffLauncher`; tests = a recording fake). - let launcher: HandoffLauncher - /// The shared append-only audit log seam. - let audit: HandoffAuditing - /// The whitelist-aware effective-tier resolver (descriptor default ∩ user whitelist). - let resolver: WritePolicyResolving - /// Parked → needs-you escalation seam. - let escalation: HandoffEscalating - /// The session this contributor serves (for the audit record's attribution). - let sessionID: AgentSessionID - /// The active skill id (for the spend ledger's attribution); nil for a free handoff. - let skillID: String? - /// True when the session is parked (background) — routes a needs-approval call to escalation. - let isParked: Bool - /// The global per-window default cap when the skill's `maxPerDay` is nil/0. - let globalDefaultPerDay: Int - /// The injected clock (deterministic, `DockHoverModel`-style). - let now: @Sendable () -> Date - - static let toolName = "launch_claude" - - init(config: ClaudeHandoffConfig = ClaudeHandoffConfig(), - budget: HandoffBudgetBox, - launcher: HandoffLauncher, - audit: HandoffAuditing = NoopHandoffAudit(), - resolver: WritePolicyResolving = DescriptorWritePolicy(), - escalation: HandoffEscalating = NoopHandoffEscalation(), - sessionID: AgentSessionID = AgentSessionID(), - skillID: String? = nil, - isParked: Bool = false, - globalDefaultPerDay: Int = 10, - now: @escaping @Sendable () -> Date = { Date() }) { - self.config = config - self.budget = budget - self.launcher = launcher - self.audit = audit - self.resolver = resolver - self.escalation = escalation - self.sessionID = sessionID - self.skillID = skillID - self.isParked = isParked - self.globalDefaultPerDay = globalDefaultPerDay - self.now = now - } - - // MARK: - Descriptor - - /// The single `launch_claude` descriptor — ALWAYS `.dangerous` (the `auto` opt-in is a per-skill - /// effective-tier downgrade, NOT a descriptor change, so the router never sees a globally-auto - /// handoff). - static func descriptor() -> ToolDescriptor { - let schema = StructuredSchema(name: toolName, json: """ - {"type":"object","required":["prompt"],"properties":{\ - "folder":{"type":"string","description":"absolute path to open Claude in; omit to use the skill's default working directory"},\ - "prompt":{"type":"string","description":"the starting prompt Claude opens with"}}} - """) - return ToolDescriptor( - name: toolName, - summary: "Hand the task to Claude Code in a folder with a starting prompt — use when the task exceeds the local model.", - argsSchema: schema, - writePolicy: .dangerous, - keywords: ["claude", "handoff", "escalate", "code", "refactor", "big task"]) - } - - func descriptors() -> [ToolDescriptor] { [Self.descriptor()] } - - func canHandle(_ tool: String) -> Bool { tool == Self.toolName } - - // MARK: - Effective gate (the cost gate) - - /// Resolve the resolved per-window cap: the skill's `maxPerDay` when set (> 0), else the global - /// default. A resolved cap of 0 means handoff is disabled. - var resolvedCap: Int { - if let perDay = config.maxPerDay, perDay > 0 { return perDay } - return globalDefaultPerDay - } - - /// The effective gate for THIS call (design Decision 4). Whitelist intersects the descriptor's - /// `.dangerous` FIRST: a user who has NOT whitelisted handoff keeps it `.dangerous` → - /// `.needsApproval`/needs-you regardless of the skill's `auto`. - func effectiveGate() -> HandoffGate { - // Disabled: a 0 resolved cap means the user/global turned handoff off. - guard resolvedCap > 0 else { return .disabled } - - // The whitelist-aware tier: .dangerous can NEVER be lowered to .auto by the resolver, so a - // non-whitelisted handoff stays dangerous → always foreground. Only when the resolver lowers the - // descriptor below dangerous AND the skill opted auto can the call run unprompted. - let effectiveTier = resolver.effectiveTier(for: Self.descriptor()) - let whitelistPermitsAuto = (effectiveTier != .dangerous) - - if !budget.allows(now: now()) { return .overBudget } - - switch config.confirmMode { - case .auto: - return whitelistPermitsAuto ? .autoRun : .needsApproval - case .confirm: - return .needsApproval - } - } - - // MARK: - Run (the state machine) - - func run(_ call: RoutedCall, gate: ApprovalGate) async -> ToolStepResult { - let toolName = call.descriptor.name - - // Folder resolution: route folder → skill default → fail clean (no spend). - guard let folder = resolveFolder(call) else { - await emit(outcome: .failed(headline: HandoffError.missingFolder.errorDescription ?? ""), - folder: nil, prompt: prompt(call)) - return ToolStepResult(tool: toolName, - status: .failed(headline: AIError.message(for: HandoffError.missingFolder).headline), - summary: "Couldn't open Claude: no folder.") - } - - let promptText = prompt(call) - - switch effectiveGate() { - case .disabled: - let reason = AIError.message(for: HandoffError.disabled).headline - await emit(outcome: .declined(reason: reason), folder: folder, prompt: promptText) - return ToolStepResult(tool: toolName, status: .declined(reason: reason), summary: reason) - - case .autoRun: - // Skill auto, under budget, whitelist permits — runs even when parked (audited). - return await spendAndLaunch(toolName: toolName, folder: folder, prompt: promptText, background: isParked) - - case .needsApproval: - return await confirmThenLaunch(toolName: toolName, folder: folder, prompt: promptText, - gate: gate, overBudget: false) - - case .overBudget: - // Never auto-run over budget, never silently dropped. Active → degrade to a foreground confirm; - // parked → escalate to needs-you (no spend until the user returns). - return await confirmThenLaunch(toolName: toolName, folder: folder, prompt: promptText, - gate: gate, overBudget: true) - } - } - - /// The confirm / over-budget path: parked → escalate (needs-you); active → await the gate. - private func confirmThenLaunch(toolName: String, folder: URL, prompt: String, - gate: ApprovalGate, overBudget: Bool) async -> ToolStepResult { - let cardReason = overBudget - ? "Daily Claude handoff limit reached. Approve to open Claude in \(folder.lastPathComponent)?" - : "Hand this to Claude Code in \(folder.lastPathComponent)?" - - if isParked { - // Dangerous-while-parked escalates: needs-you badge, no spend until the user returns. - await escalation.escalate(sessionID, reason: cardReason) - await emit(outcome: .awaitingApproval, folder: folder, prompt: prompt) - return ToolStepResult(tool: toolName, status: .awaitingApproval, summary: cardReason) - } - - let review = TaskReview.action(title: "Open Claude Code", - fields: [ReviewField("Folder", folder.path), - ReviewField("Prompt", AuditRedaction.summary(forRawArguments: prompt))], - payload: .openTool(tool: Self.toolName, - action: ParsedOpenTool(applicable: true, reason: nil, payload: prompt))) - switch await gate.awaitDecision(for: review) { - case .approve: - return await spendAndLaunch(toolName: toolName, folder: folder, prompt: prompt, background: false) - case .skip: - let reason = "skipped" - await emit(outcome: .declined(reason: reason), folder: folder, prompt: prompt) - return ToolStepResult(tool: toolName, status: .declined(reason: reason), - summary: "Skipped the Claude handoff.") - case .cancel: - // The whole canvas was discarded — end the loop quietly (NOT a failure), no spend, no launch. - await emit(outcome: .declined(reason: TaskKindToolContributor.cancelledReason), - folder: folder, prompt: prompt) - return ToolStepResult(tool: toolName, - status: .declined(reason: TaskKindToolContributor.cancelledReason), - summary: "Cancelled.") - } - } - - /// Record the spend, audit, fire fire-and-forget. A launch that throws → `.failed` + refund (a handoff - /// that didn't land didn't spend). V1: reap immediately after a successful open. - private func spendAndLaunch(toolName: String, folder: URL, prompt: String, background: Bool) async -> ToolStepResult { - let at = now() - budget.record(at: at, skillID: skillID) - do { - try await launcher.launch(folder: folder, prompt: prompt) - budget.reap() - await emit(outcome: .done, folder: folder, prompt: prompt, background: background) - return ToolStepResult(tool: toolName, status: .done, - summary: "Opened Claude Code in \(folder.lastPathComponent).") - } catch { - budget.refund(at: at) // a launch that didn't land didn't spend — the cap stays honest. - let headline = AIError.message(for: error).headline - await emit(outcome: .failed(headline: headline), folder: folder, prompt: prompt, background: background) - return ToolStepResult(tool: toolName, status: .failed(headline: headline), - summary: "Couldn't open Claude Code.") - } - } - - // MARK: - Folder / prompt extraction - - private func resolveFolder(_ call: RoutedCall) -> URL? { - if let routed = stringArg(call, "folder")?.trimmingCharacters(in: .whitespacesAndNewlines), !routed.isEmpty { - return URL(fileURLWithPath: (routed as NSString).expandingTildeInPath) - } - if let dir = config.folder?.trimmingCharacters(in: .whitespacesAndNewlines), !dir.isEmpty { - return URL(fileURLWithPath: (dir as NSString).expandingTildeInPath) - } - return nil - } - - private func prompt(_ call: RoutedCall) -> String { - if let p = stringArg(call, "prompt"), !p.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty { - return p - } - // Fall back to the skill's configured starting prompt, then the user's text. - if let sp = config.startingPrompt, !sp.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty { - return sp - } - return "" // empty → a bare claude session (allowed) - } - - private func stringArg(_ call: RoutedCall, _ key: String) -> String? { - guard let data = call.route.argumentsJSON.data(using: .utf8), - let obj = try? JSONSerialization.jsonObject(with: data) as? [String: Any] else { return nil } - return obj[key] as? String - } - - // MARK: - Audit (one record per outcome; redacted summary) - - /// Emit EXACTLY one audit record for this outcome. `argumentsSummary` is the folder + a TRUNCATED - /// prompt — NEVER the full prompt verbatim (a prompt can carry secrets; raw text rides only in logs). - private func emit(outcome: ToolStepStatus, folder: URL?, prompt: String, background: Bool? = nil) async { - let folderPart = folder.map { AuditRedaction.lastComponents($0.path) } ?? "(no folder)" - let promptPart = AuditRedaction.summary(forRawArguments: prompt) - let summary = AuditRedaction.middleTruncate("\(folderPart) · \(promptPart)", - to: AuditRedaction.maxSummaryLength) - let record = AuditRecord(sessionID: sessionID, - tool: Self.toolName, - policy: .dangerous, - argumentsSummary: summary, - outcome: outcome, - wasBackground: background ?? isParked, - timestamp: now()) - await audit.record(record) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Handoff/HandoffBudget.swift b/Sources/ThreeFingerSwitcher/AI/Handoff/HandoffBudget.swift deleted file mode 100644 index 94778c8..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Handoff/HandoffBudget.swift +++ /dev/null @@ -1,150 +0,0 @@ -import Foundation - -/// One spend record in the append-only handoff ledger (`ai-claude-handoff`, design Decision 5). A -/// successful (or in-flight) handoff appends one; a launch that throws after recording removes it -/// (refund), so the cap stays honest. `Codable` so the ledger persists across a relaunch within the -/// rolling window — a process restart cannot reset the daily budget. MLX-free Core. -struct HandoffSpend: Codable, Equatable, Sendable { - let at: Date - let skillID: String? - - init(at: Date, skillID: String? = nil) { - self.at = at - self.skillID = skillID - } -} - -/// The pure rate/budget gate over an append-only spend ledger (`ai-claude-handoff`, design Decision 5). -/// `allows(now:)` is a PURE predicate — `now` is an INPUT (deterministic, `DockHoverModel`-style) — so an -/// autonomous agent loop physically cannot rack up real spend and the window is unit-testable without a -/// clock. The window is **rolling 24h** (a sliding count over `now`), NOT a calendar-day reset, so a loop -/// cannot dump N calls at 23:59 and N more at 00:01. MLX-free Core. -struct HandoffBudget: Equatable, Sendable { - /// The resolved per-window cap (a skill cap or the global default; 0 means "disabled" — see the - /// contributor's resolution). A `<= 0` cap never allows. - let maxCallsPerDay: Int - /// At-most-N in-flight handoffs (v1 default 1). v1 reaps immediately on a successful open, so this - /// really means "one handoff per loop step" (design §6 / Q3). - let maxConcurrent: Int - /// The append-only spend records; in-flight is tracked separately. - private(set) var ledger: [HandoffSpend] - /// The count of opened-but-not-yet-reaped handoffs. - private(set) var inFlight: Int - - /// The rolling window length: 24 hours. - static let window: TimeInterval = 24 * 60 * 60 - - init(maxCallsPerDay: Int, maxConcurrent: Int = 1, - ledger: [HandoffSpend] = [], inFlight: Int = 0) { - self.maxCallsPerDay = maxCallsPerDay - self.maxConcurrent = max(1, maxConcurrent) - self.ledger = ledger - self.inFlight = inFlight - } - - /// The number of spends inside the rolling 24h window ending at `now`. - func callsInLast24h(_ now: Date) -> Int { - let cutoff = now.addingTimeInterval(-Self.window) - return ledger.reduce(0) { $0 + ($1.at > cutoff ? 1 : 0) } - } - - /// PURE predicate: under the rolling-window cap AND below the concurrency limit. `now` is an INPUT. - func allows(now: Date) -> Bool { - guard maxCallsPerDay > 0 else { return false } - return callsInLast24h(now) < maxCallsPerDay && inFlight < maxConcurrent - } - - /// Spend a call: append the record and increment in-flight. - mutating func record(at: Date, skillID: String? = nil) { - ledger.append(HandoffSpend(at: at, skillID: skillID)) - inFlight += 1 - } - - /// A fire-and-forget launch is "done" → decrement in-flight (v1: immediately after a successful open). - /// The ledger entry stays (it counts against the daily cap); only the in-flight count drops. - mutating func reap() { - inFlight = max(0, inFlight - 1) - } - - /// A launch that threw didn't spend → remove the matching ledger entry (by `at`) and drop in-flight, - /// leaving the cap unchanged from before the call. - mutating func refund(at: Date) { - if let idx = ledger.lastIndex(where: { $0.at == at }) { - ledger.remove(at: idx) - } - inFlight = max(0, inFlight - 1) - } -} - -/// The injectable persistence seam for the handoff ledger (`ai-claude-handoff`, task 2.3). The ledger -/// persists under Application Support (like `ClipboardStore`/`ParkedSessionStore`) so the cap survives a -/// relaunch within the rolling window. Tests inject an in-memory or temp-dir store. MLX-free Core. -protocol HandoffLedgerStore: Sendable { - func load() -> [HandoffSpend] - func save(_ ledger: [HandoffSpend]) -} - -/// The default no-disk store (tests / a fresh process with nothing persisted). Records are held in -/// memory only. -final class InMemoryHandoffLedgerStore: HandoffLedgerStore, @unchecked Sendable { - private let lock = NSLock() - private var ledger: [HandoffSpend] - - init(_ ledger: [HandoffSpend] = []) { self.ledger = ledger } - - func load() -> [HandoffSpend] { - lock.lock(); defer { lock.unlock() } - return ledger - } - - func save(_ ledger: [HandoffSpend]) { - lock.lock(); defer { lock.unlock() } - self.ledger = ledger - } -} - -/// The durable JSON ledger store under Application Support (mirroring `DiskAuditLog`'s shape). A read -/// failure yields an empty ledger (a fresh budget); a write failure is swallowed (the in-memory budget -/// still holds the spend) — auditing/cap honesty is best-effort persistence, never a thrown break into -/// the loop. MLX-free Core; its real IO is exercised by the user's build. -final class DiskHandoffLedgerStore: HandoffLedgerStore, @unchecked Sendable { - private let fileURL: URL - private let encoder: JSONEncoder = { - let e = JSONEncoder(); e.outputFormatting = [.sortedKeys]; e.dateEncodingStrategy = .iso8601; return e - }() - private let decoder: JSONDecoder = { - let d = JSONDecoder(); d.dateDecodingStrategy = .iso8601; return d - }() - - init(fileURL: URL = DiskHandoffLedgerStore.defaultFileURL()) { - self.fileURL = fileURL - } - - /// `~/Library/Application Support/ThreeFingerSwitcher/handoff/ledger.json`. - static func defaultFileURL() -> URL { - let base = (try? FileManager.default.url(for: .applicationSupportDirectory, in: .userDomainMask, - appropriateFor: nil, create: true)) - ?? URL(fileURLWithPath: NSTemporaryDirectory()) - return base - .appendingPathComponent("ThreeFingerSwitcher/handoff", isDirectory: true) - .appendingPathComponent("ledger.json") - } - - func load() -> [HandoffSpend] { - guard let data = try? Data(contentsOf: fileURL), - let records = try? decoder.decode([HandoffSpend].self, from: data) else { return [] } - return records - } - - func save(_ ledger: [HandoffSpend]) { - do { - try FileManager.default.createDirectory(at: fileURL.deletingLastPathComponent(), - withIntermediateDirectories: true) - let data = try encoder.encode(ledger) - try data.write(to: fileURL, options: .atomic) - } catch { - // Best-effort: the in-memory budget still holds the spend; a persistence failure does not - // break the loop (never thrown into the caller). - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Handoff/HandoffError.swift b/Sources/ThreeFingerSwitcher/AI/Handoff/HandoffError.swift deleted file mode 100644 index 9dea555..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Handoff/HandoffError.swift +++ /dev/null @@ -1,45 +0,0 @@ -import Foundation - -/// The handoff-specific error taxonomy (`ai-claude-handoff`, design Decision 10) — a clean -/// `LocalizedError` for ONLY the cases `RuntimeError`/`TaskError`/`ClaudeLaunchError` cannot carry. -/// Every case has a per-case, user-facing headline (never a reflected enum dump or raw OS text); raw -/// vendor/OS text rides ONLY in `details` / logs. The production launcher maps `ClaudeLaunchError` into -/// `.launchFailed` at the launch boundary so Core stays consistent and the existing clean headline flows -/// through. `AIError.message(for:)` is extended (THE one translator) to render this taxonomy identically -/// on every surface. MLX-free Core. -enum HandoffError: Error, Equatable { - /// The skill carries a handoff config but it is turned off (or the global budget is disabled). - case disabled - /// Over the daily cap AND parked with no one to confirm — escalated to needs-you, never auto-run. - case overBudgetNoUser - /// No folder in the route AND no default working directory — nothing to open Claude in. - case missingFolder - /// The launch itself failed; wraps a mapped `ClaudeLaunchError`. `details` is opt-in copyable text. - case launchFailed(headline: String, details: String?) -} - -extension HandoffError: LocalizedError { - var errorDescription: String? { - switch self { - case .disabled: - return "Claude handoff is turned off for this skill." - case .overBudgetNoUser: - return "The daily Claude handoff limit was reached. Return to approve this one." - case .missingFolder: - return "No folder to open Claude in. Pick a folder and try again." - case let .launchFailed(headline, _): - return headline - } - } - - /// The opt-in copyable detail (raw error text captured at the boundary), for a "Show details / Copy" - /// disclosure and logs only. `nil` when the headline already says everything. - var copyableDetails: String? { - switch self { - case .disabled, .overBudgetNoUser, .missingFolder: - return nil - case let .launchFailed(_, details): - return details - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Handoff/HandoffLauncher.swift b/Sources/ThreeFingerSwitcher/AI/Handoff/HandoffLauncher.swift deleted file mode 100644 index a6ac513..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Handoff/HandoffLauncher.swift +++ /dev/null @@ -1,66 +0,0 @@ -import Foundation -#if canImport(AppKit) -import AppKit -#endif - -/// The side-effecting spawn seam (`ai-claude-handoff`, design Decision 3 / 7). Behind a protocol so the -/// contributor is headless-testable: `swift test` drives a FAKE launcher that records `(folder, prompt)` -/// without spawning anything; the production `OpenClaudeHandoffLauncher` does the real `.command` + -/// `NSWorkspace.open` handoff. Fire-and-forget v1: `launch` opens Claude and returns; it throws a mapped -/// `HandoffError` if it can't open. MLX-free Core. -protocol HandoffLauncher: Sendable { - func launch(folder: URL, prompt: String) async throws -} - -/// The production adapter (`ai-claude-handoff`, design Decision 7) — the ONLY side-effecting code in the -/// slice. It composes the EXISTING open-claude-here launch path (`ClaudeLauncher.shellQuote`/ -/// `resolveClaudePath`/`writeCommandFile` + `NSWorkspace.shared.open`) byte-for-byte — NO new launch -/// mechanism, NO new permission (no Apple Events). A non-empty prompt becomes the inner command -/// `claude ''`; an empty prompt opens a bare `claude` session. Off-main resolution + write, -/// main-actor open, success-needs-no-notification (the terminal window is its own feedback). Failures map -/// at this boundary into `HandoffError.launchFailed` (the existing clean `ClaudeLaunchError` headline -/// flows through). `Launcher/ClaudeLaunch.swift` is UNCHANGED by this slice; this only CALLS its builders. -struct OpenClaudeHandoffLauncher: HandoffLauncher { - - init() {} - - /// Build the inner command for a starting prompt: empty → nil (bare `claude` via the script default); - /// else `claude ''` (the prompt passed as Claude's argument, exactly how a custom - /// `command` rides `ClaudeLauncher.commandScript`'s inner-command slot). Pure + unit-tested. - static func innerCommand(forPrompt prompt: String) -> String? { - let trimmed = prompt.trimmingCharacters(in: .whitespacesAndNewlines) - guard !trimmed.isEmpty else { return nil } - return "claude \(ClaudeLauncher.shellQuote(prompt))" - } - - func launch(folder: URL, prompt: String) async throws { - let inner = Self.innerCommand(forPrompt: prompt) - // Off-main: a bare session resolves the absolute claude path (so a non-PATH install still runs); - // a prompt session lets the script's `claude`-from-PATH carry it (the inner command is `claude …`). - let claudePath = inner == nil ? ClaudeLauncher.resolveClaudePath() : nil - let url: URL - do { - url = try ClaudeLauncher.writeCommandFile(folder: folder, command: inner, claudePath: claudePath) - } catch let e as ClaudeLaunchError { - throw Self.map(e) - } catch { - throw HandoffError.launchFailed(headline: ClaudeLaunchError.scriptWriteFailed(details: nil).errorDescription - ?? "Couldn't prepare the Claude launch. Please try again.", - details: String(describing: error)) - } - - #if canImport(AppKit) - let opened = await MainActor.run { NSWorkspace.shared.open(url) } - if !opened { - throw Self.map(.terminalOpenFailed(details: "NSWorkspace.open returned false for \(url.lastPathComponent)")) - } - #endif - } - - /// Map a `ClaudeLaunchError` into `HandoffError.launchFailed` at the launch boundary — the clean - /// headline flows through; raw OS/vendor text rides only in `details`. - static func map(_ e: ClaudeLaunchError) -> HandoffError { - .launchFailed(headline: e.errorDescription ?? "Couldn't open Claude Code.", - details: e.copyableDetails) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Handoff/HandoffSeams.swift b/Sources/ThreeFingerSwitcher/AI/Handoff/HandoffSeams.swift deleted file mode 100644 index 63a95e0..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Handoff/HandoffSeams.swift +++ /dev/null @@ -1,50 +0,0 @@ -import Foundation - -/// The narrow audit seam (`ai-claude-handoff`, design Decision 6). The contributor records exactly one -/// `AuditRecord` per `run` outcome through this protocol; the production conformer bridges to -/// `ai-background-autonomy`'s shared append-only `AuditLog` (one log, blueprint §3.7 — NOT a forked -/// handoff-only log). A no-op default + a recording test double keep the slice headless-testable. -/// MLX-free Core. -protocol HandoffAuditing: Sendable { - func record(_ record: AuditRecord) async -} - -/// The no-op default (a slice with no audit log wired yet). Records nowhere — used only as a safe -/// fallback; production always wires `AuditLogHandoffAuditing`. -struct NoopHandoffAudit: HandoffAuditing { - init() {} - func record(_ record: AuditRecord) async {} -} - -/// The production audit bridge: records into the shared `AuditLog` (`ai-background-autonomy`). `record` -/// is non-blocking and infallible from the caller's view (the log swallows persistence failures and -/// surfaces them on its viewer, never throws into the loop). -struct AuditLogHandoffAuditing: HandoffAuditing { - let log: AuditLog - init(_ log: AuditLog) { self.log = log } - func record(_ record: AuditRecord) async { log.record(record) } -} - -/// The narrow escalation seam (`ai-claude-handoff`, design Decision 8). A dangerous handoff that needs -/// approval inside a PARKED session does NOT auto-run and does NOT silently wait — it raises -/// `ParkState.needsYou` via this seam so the user is pulled back to approve the spend. The production -/// conformer bridges to `ai-parked-sessions`' `ParkScheduler.escalate`. A no-op default + a recording -/// double keep the slice headless-testable. MLX-free Core. -protocol HandoffEscalating: Sendable { - func escalate(_ sessionID: AgentSessionID, reason: String) async -} - -/// The no-op default (an active-session-only context with no parked scheduler wired). -struct NoopHandoffEscalation: HandoffEscalating { - init() {} - func escalate(_ sessionID: AgentSessionID, reason: String) async {} -} - -/// The production escalation bridge: raises the parked session's needs-you badge via `ParkScheduler`. -struct ParkSchedulerHandoffEscalation: HandoffEscalating { - let scheduler: ParkScheduler - init(_ scheduler: ParkScheduler) { self.scheduler = scheduler } - func escalate(_ sessionID: AgentSessionID, reason: String) async { - scheduler.escalate(sessionID, reason: reason) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/LLMRuntime.swift b/Sources/ThreeFingerSwitcher/AI/LLMRuntime.swift deleted file mode 100644 index 8da69a9..0000000 --- a/Sources/ThreeFingerSwitcher/AI/LLMRuntime.swift +++ /dev/null @@ -1,380 +0,0 @@ -import Foundation - -/// The on-device model layer's single seam. Feature code (the band, the executor, the tasks) -/// depends ONLY on this file — never on a concrete model or framework — so that an additional -/// model (another Gemma 4 size, a future Gemma, Apple Foundation Models, or a cloud model) can be -/// added later as one new conformer without touching feature code (see design D1). -/// -/// The real Gemma-4-via-MLX conformer is DEFERRED to a separate `xcodebuild`-only target; this -/// slice ships only `StubLLMRuntime`, which exercises every path below deterministically. - -// MARK: - Modality - -/// What kinds of input a runtime (or a model descriptor) can handle. A vision command requires a -/// `.vision`-capable runtime; `.audio` is reserved for a future audio-capable Gemma 4 so an audio -/// command can be routed without changing feature code. -public enum Modality: String, Codable, Sendable, CaseIterable { - case text - case vision - case audio -} - -// MARK: - Request - -/// Generation tuning knobs. Pragmatic defaults; the executor overrides per command. -public struct GenerationParameters: Equatable, Sendable { - /// Hard cap on emitted tokens (a safety bound on runaway generation). - public var maxTokens: Int - /// Sampling temperature. 0 ≈ greedy/deterministic; higher = more varied. - public var temperature: Double - - public static let `default` = GenerationParameters(maxTokens: 1024, temperature: 0.7) - - public init(maxTokens: Int = 1024, temperature: Double = 0.7) { - self.maxTokens = maxTokens - self.temperature = temperature - } -} - -/// One unit of model work: a text prompt, encoded image bytes for vision (a turn may carry MULTIPLE — -/// design D2), and tuning. `images` carries encoded image data (e.g. PNGs of captured regions); a -/// `.text`-only runtime ignores them. Kept a value type so requests are cheap to build and pass around. -public struct LLMRequest: Sendable { - /// The fully-resolved prompt text (templating happens upstream in the executor). - public var prompt: String - /// Encoded images for a `.vision` request (empty for text-only). A SINGLE turn may carry MULTIPLE - /// images (design D2); the runtime feeds each to the model input. - public var images: [Data] - /// Encoded audio payloads for an `.audio` request (`add-voice-computer-use-agent`): the statically - /// typed v4+ seam for direct audio-in Gemma (the vendored audio tower mirrors the vision path). - /// Until a conformer serves audio, every runtime — including the stub — REJECTS a non-empty value - /// with `unsupportedModality(.audio)`; the bytes are never silently dropped. Default `[]`. - public var audio: [Data] - public var parameters: GenerationParameters - /// When true, the runtime should let the model think (reasoning) but stream/return only the final - /// response — never the thinking. - public var reasoning: Bool - - /// The FIRST image, or nil — the single-image convenience (design D2: `images` is the source of truth). - public var image: Data? { images.first } - - /// Designated init (multi-image). `images`/`audio` default to `[]`. - public init(prompt: String, images: [Data] = [], audio: [Data] = [], - parameters: GenerationParameters = .default, - reasoning: Bool = false) { - self.prompt = prompt - self.images = images - self.audio = audio - self.parameters = parameters - self.reasoning = reasoning - } - - /// Single-image convenience (design D2): folds one optional image into `images` so existing callers - /// (`LLMRequest(prompt:image:)`) keep compiling unchanged. A nil image yields `[]`. - public init(prompt: String, image: Data?, parameters: GenerationParameters = .default, - reasoning: Bool = false) { - self.init(prompt: prompt, images: image.map { [$0] } ?? [], - parameters: parameters, reasoning: reasoning) - } - - /// Whether this request needs a `.vision`-capable runtime (it carries at least one image). - public var requiresVision: Bool { !images.isEmpty } - - /// Whether this request needs an `.audio`-capable runtime (it carries at least one audio payload). - public var requiresAudio: Bool { !audio.isEmpty } -} - -/// A multi-turn conversation request: a role-tagged message list (already compacted upstream by the -/// executor) plus the same tuning the single-prompt `LLMRequest` carries. Additive over `LLMRequest` -/// (design D2 / blueprint §3.2) — served by `LLMRuntime.chat(_:)`, which is default-implemented in terms -/// of `generate(_:)` so existing conformers work unchanged. `tools` is declared in its final shape for -/// route-mode (owned by `ai-tool-routing`); this slice leaves it nil and ignores it. -public struct LLMChatRequest: Sendable { - /// The role-tagged multi-turn context. The conversation's reasoning is NOT re-fed: the assembler - /// (`ChatTemplate.flatten`, or a conformer's chat template) reads each message's committed `text` only. - public var messages: [AgentMessage] - /// The turn's images (design D2 — ALL of the latest turn's images, not just one); a `.vision`-capable - /// runtime is required when non-empty. Empty = a text turn. - public var images: [Data] - /// The turn's audio payloads (`add-voice-computer-use-agent` — the v4+ seam, mirroring `images`). - /// An `.audio`-capable runtime is required when non-empty; every current runtime rejects it with - /// `unsupportedModality(.audio)` until the audio-tower conformer lands. Default `[]`. - public var audio: [Data] - public var parameters: GenerationParameters - /// When true, the runtime lets the model think but streams/returns only the final response. - public var reasoning: Bool - /// Advertised tools for route-mode (owned by `ai-tool-routing`); nil = plain chat (this slice). - public var tools: [ToolDescriptor]? - - /// The FIRST request-level image, or nil — the single-image convenience (design D2). - public var image: Data? { images.first } - - /// Designated init (multi-image). `images`/`audio` default to `[]`. - public init(messages: [AgentMessage], - images: [Data] = [], - audio: [Data] = [], - parameters: GenerationParameters = .default, - reasoning: Bool = false, - tools: [ToolDescriptor]? = nil) { - self.messages = messages - self.images = images - self.audio = audio - self.parameters = parameters - self.reasoning = reasoning - self.tools = tools - } - - /// Single-image convenience (design D2): folds one optional image into `images` so existing callers - /// (`LLMChatRequest(messages:image:)`) keep compiling unchanged. A nil image yields `[]`. - public init(messages: [AgentMessage], - image: Data?, - parameters: GenerationParameters = .default, - reasoning: Bool = false, - tools: [ToolDescriptor]? = nil) { - self.init(messages: messages, images: image.map { [$0] } ?? [], - parameters: parameters, reasoning: reasoning, tools: tools) - } - - /// ALL of the effective turn's images (design D2): the request-level `images` if set, else the latest - /// message that carries images. Forwarded in full to the runtime so a multi-image turn reaches the model. - public var effectiveImages: [Data] { - if !images.isEmpty { return images } - return messages.last(where: { !$0.images.isEmpty })?.images ?? [] - } - - /// The latest effective image (first of `effectiveImages`), for single-image consumers/tests. - public var effectiveImage: Data? { effectiveImages.first } - - /// Whether this chat turn needs an `.audio`-capable runtime. - public var requiresAudio: Bool { !audio.isEmpty } -} - -// MARK: - Streaming token - -/// Which channel a streamed token belongs to (design: "show the model's thinking"). The runtime -/// classifies each chunk as either the model's reasoning (`.thinking`) or the final answer -/// (`.response`); the preview canvas streams `.thinking` into a collapsible section and commits ONLY -/// `.response`. Legacy emitters that don't classify default to `.response`, so today's behavior — a -/// single response stream — is byte-identical until a runtime opts into emitting `.thinking`. -public enum TokenChannel: Equatable, Sendable { - /// The final answer — accumulated, streamed into `state`, and committed. - case response - /// The model's reasoning — streamed into the canvas's collapsible Thinking section, NEVER committed. - case thinking -} - -/// One incremental chunk of streamed output. A runtime emits these in order as the model produces -/// them; the preview canvas concatenates `text` to render generation live (design D4). `channel` -/// classifies the chunk as the model's reasoning (`.thinking`) or its final answer (`.response`) so -/// the canvas can split the two — thinking is shown but never committed. -public struct Token: Equatable, Sendable { - /// The piece of text produced for this step (a sub-word, word, or fragment). - public var text: String - /// True for the final token of a stream (lets a consumer finalize without waiting on stream end). - public var isFinal: Bool - /// Which channel this chunk belongs to. Defaults to `.response` so existing emitters/tests that - /// build `Token(text)` keep compiling and mean "final answer". - public var channel: TokenChannel - - public init(_ text: String, isFinal: Bool = false, channel: TokenChannel = .response) { - self.text = text - self.isFinal = isFinal - self.channel = channel - } -} - -// MARK: - Structured output - -/// A JSON-Schema wrapper handed to `structured(...)`. We carry the schema as a string (its JSON -/// representation) rather than a parsed tree so it can be embedded in prompts, logged, and validated -/// uniformly across conformers. `name` labels the target shape for the model's benefit. -public struct StructuredSchema: Codable, Equatable, Sendable { - /// A short identifier for the target shape (e.g. "calendar_event"). - public var name: String - /// The JSON Schema document, as a JSON string. - public var json: String - - public init(name: String, json: String) { - self.name = name - self.json = json - } -} - -/// The result of a `structured(...)` call: EITHER a validated, decoded value, OR an explicit -/// decline. The decline path is first-class on purpose (design D2): the model is allowed to refuse -/// — "this isn't a meeting" — instead of being forced to fabricate a well-formed-but-false value. -public enum StructuredOutcome: Sendable where Value: Sendable { - /// The model produced output that validated against the schema and decoded into `Value`. - case value(Value) - /// The model declined the task as not applicable; carries a human-readable reason. - case declined(reason: String) - - /// The decoded value if produced, else nil (decline). - public var value: Value? { - if case let .value(v) = self { return v } - return nil - } - - /// The decline reason if declined, else nil. - public var declineReason: String? { - if case let .declined(reason) = self { return reason } - return nil - } - - public var isDeclined: Bool { - if case .declined = self { return true } - return false - } -} - -// MARK: - Errors - -/// Failures the runtime layer can report. Distinct cases so the UI can message precisely (a missing -/// model asks for a download; an integrity failure asks for a re-download; `unavailable` is the -/// "this machine/config can't serve the feature" terminal state — never a silent degrade). -/// -/// This is the SHARED error taxonomy for the AI feature (design D1/D6): each runtime backend maps its -/// own native errors (e.g. a vendor download-library error, an `NSURLError`) into these cases at its -/// boundary, so feature/UI code only ever sees this type — never a raw vendor/OS error. The taxonomy -/// is `LocalizedError` so every case is self-describing with a clean, user-facing string; the central -/// `AIError.message(for:)` translator routes through that `errorDescription` (never a reflected enum -/// dump). Associated values stay `Equatable` (and carry no non-`Equatable` `Error`) so the enum is -/// `Equatable` — copyable raw detail rides on `AIPresentedError.details`, derived at translation time. -public enum RuntimeError: Error, Equatable { - /// The feature can't be served on this machine/configuration (no silent fallback). - case unavailable(reason: String) - /// Weights are not present (not yet downloaded). - case modelMissing - /// A downloaded model failed its integrity (SHA) check; it must not be loaded. - case integrityFailed - /// The work was cancelled mid-flight (Task cancellation / discard). - case cancelled - /// `structured(...)` exhausted its bounded repair/retry loop without a conforming value. - case couldNotProduceValid(attempts: Int) - /// Output could not be decoded into the requested `Decodable` type. - case decodeFailed(detail: String) - /// The runtime lacks a capability the request needs (e.g. vision asked of a text-only model). - case unsupportedModality(Modality) - /// No internet connection reached the model service (e.g. provision/download with wifi off). - case offline - /// The model service was reachable but could not serve the request (5xx, transient outage). - case serverUnavailable - /// Access to the model was refused (auth/forbidden/not-found at the download endpoint). - case authOrAccessDenied - /// The weights downloaded but could not be loaded into the runtime. `detail` is opt-in copyable - /// diagnostic text (kept off the user-facing headline; surfaced only as `AIPresentedError.details`). - case modelLoadFailed(detail: String?) -} - -/// Self-describing, user-facing messages for every case (so the "clean path" — reading -/// `errorDescription` — never falls back to a reflected enum dump). These are the canonical strings -/// the central `AIError.message(for:)` translator returns as the headline; raw error text never -/// appears here, only in opt-in `details`/logs (spec: "No raw error text in user-facing strings"). -extension RuntimeError: LocalizedError { - public var errorDescription: String? { - switch self { - case let .unavailable(reason): return reason - case .modelMissing: return "The model is not downloaded yet." - case .integrityFailed: return "The model failed its integrity check; re-download required." - case .cancelled: return "Cancelled." - case let .couldNotProduceValid(attempts): return "Could not produce a valid result (\(attempts) attempts)." - case .decodeFailed: return "Could not read the model's result." - case let .unsupportedModality(modality): return "The model can't handle \(modality.rawValue) input." - case .offline: return "No internet connection. Connect to the internet and try again." - case .serverUnavailable: return "The model service is temporarily unavailable. Please try again shortly." - case .authOrAccessDenied: return "Access to the model was denied. It may require sign-in or has moved." - case .modelLoadFailed: return "The model could not be loaded." - } - } -} - -// MARK: - The protocol - -/// The swappable model runtime. All language-model functionality is reached through this. -/// -/// Conformers must: -/// - declare their `capabilities` so capability-based selection can route correctly, -/// - stream text via `generate(_:)`, honoring Task cancellation promptly (discard stops work), -/// - and serve `structured(_:schema:as:)` by validating against the schema, repairing/retrying -/// within a bounded loop on mismatch, and allowing a typed `.declined` outcome rather than -/// fabricating a value. -public protocol LLMRuntime: Sendable { - /// The modalities this runtime can serve. - var capabilities: Set { get } - - /// Stream generated text token-by-token. Cancelling the consuming Task SHALL stop generation - /// promptly (the stream finishes; no further tokens are emitted). - func generate(_ request: LLMRequest) -> AsyncThrowingStream - - /// Produce a schema-targeted, validated, decoded structured value — or a typed decline. - /// - /// The contract (design D2): request output matching `schema`, VALIDATE it, REPAIR/RETRY within a - /// bounded loop on mismatch, decode into `T`, and allow a `.declined` outcome when the input does - /// not fit the task. Throws `RuntimeError.couldNotProduceValid` only when the bounded loop is - /// exhausted without a conforming value (and the model did not decline). - func structured( - _ request: LLMRequest, - schema: StructuredSchema, - as type: T.Type - ) async throws -> StructuredOutcome - - /// Stream a multi-turn conversation token-by-token. ADDITIVE (design D2): declared as a requirement - /// WITH a default implementation in the extension below, so (a) existing conformers - /// (`StubLLMRuntime`, `DevAIRuntime`, the Gemma conformer) keep compiling UNCHANGED — they inherit - /// the default — while (b) a conformer that overrides it (the batched MLX runtime, for true KV-reuse) - /// is dispatched correctly when called through the `LLMRuntime` existential. Declaring it only in the - /// extension would static-dispatch to the default and silently ignore the override. - func chat(_ request: LLMChatRequest) -> AsyncThrowingStream -} - -extension LLMRuntime { - /// DEFAULT multi-turn implementation (design D2 / blueprint §3.2): flatten the message list into one - /// prompt via the model-agnostic `ChatTemplate`, then serve it through `generate(_:)` — passing - /// `image`/`reasoning`/`parameters` straight through so the `.thinking`/`.response` channel split and - /// the vision path are inherited for free. Existing conformers get multi-turn with no change; the - /// batched MLX conformer OVERRIDES this for true key/value-cache reuse across turns. - public func chat(_ request: LLMChatRequest) -> AsyncThrowingStream { - let prompt = ChatTemplate.flatten(request.messages) - return generate(LLMRequest(prompt: prompt, - images: request.effectiveImages, // ALL of the turn's images (design D2) - audio: request.audio, // carried so the refusal contract holds - parameters: request.parameters, - reasoning: request.reasoning)) - } - - /// Convenience: collect a full generation into a single string (used by tests and non-streaming - /// callers). Propagates cancellation and errors from the underlying stream. - public func generateText(_ request: LLMRequest) async throws -> String { - var out = "" - for try await token in generate(request) { - out += token.text - } - return out - } -} - -// MARK: - Batched (continuous-batching) runtime - -/// A runtime that multiplexes K conversation streams over ONE weight read per decode step (blueprint -/// §3.6, `ai-batched-runtime-and-context`). Decode is memory-bandwidth-bound — K independent -/// `generate()` calls would re-read the ~17 GB weights K times (K× bandwidth, no win); continuous -/// batching reads the weights once and advances K streams together. `maxConcurrentStreams` is RAM-derived -/// (see `ConcurrencyBudget`), not a constant, because the per-stream KV cache grows with context. -/// -/// MLX-free Core declares the seam; the real conformer (`BatchedGemmaMLXRuntime`) lives in GemmaRuntime -/// and is `xcodebuild` compile-verified only. It refines `LLMRuntime`, so the foreground `chat()` path is -/// unchanged (a single-session `chat` is just a K=1 batch). -public protocol BatchedLLMRuntime: LLMRuntime { - /// Advance K streams one batched step at a time, de-multiplexing each emitted token back to its - /// session. New requests may join mid-flight (continuous, not static, batching); a finished stream - /// frees its slot for the next runnable session with the weights still resident. - func batchStep(_ requests: [AgentSessionID: LLMChatRequest]) - -> AsyncThrowingStream<(AgentSessionID, Token), Error> - - /// K — how many streams fit RIGHT NOW, derived from free RAM at the current context length and - /// KV-quant bits. The real conformer recomputes this ON EACH READ from the live free-memory probe, - /// so a context-setting change or a memory-pressure event is reflected at the next batch admission - /// (there is no push-based recompute; `ModelManager`'s wired pressure observer handles eviction, - /// and admission simply reads the smaller K afterwards). - var maxConcurrentStreams: Int { get } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/Image/ImageCostDisclosure.swift b/Sources/ThreeFingerSwitcher/AI/Media/Image/ImageCostDisclosure.swift deleted file mode 100644 index 4f21fff..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/Image/ImageCostDisclosure.swift +++ /dev/null @@ -1,82 +0,0 @@ -import Foundation - -/// The pure cost-disclosure + busy-painting STATE values for the local image backend (design D5, tasks -/// 5.1–5.3 pure parts). The disclosure ethos: state RAM / heat / latency cost IN THE SAME BREATH the -/// capability is offered. Both this disclosure AND the runtime's pre-flight "busy painting" decision read -/// the SAME `ImageResidencyClass` value (task 2.3) — so what the user is TOLD before firing and what the -/// runtime DOES at fire can never disagree. -/// -/// All values are pure (computed from the chosen descriptor + the injected `ImageResidencyClass`), so the -/// underlying numbers are `swift test`-verified; the app UI (a Hub row / the canvas pre-fire card) renders -/// these strings and is `xcodebuild` compile-verified, run-verified by the user. -/// -/// MLX-free Core. -public struct ImageCostDisclosure: Equatable, Sendable { - /// The RAM line — the resident footprint AND its co-reside-vs-evict-chat consequence (from the class). - public let ram: String - /// The heat/compute note — sustained M5 GPU diffusion (the neural-accelerator sweet spot is still a burn). - public let heat: String - /// The latency note — seconds-to-tens-of-seconds at default steps. - public let latency: String - /// True when the chosen variant EVICTS chat (FP16) — the UI emphasises the "chat pauses" consequence. - public let evictsChat: Bool - - public init(ram: String, heat: String, latency: String, evictsChat: Bool) { - self.ram = ram - self.heat = heat - self.latency = latency - self.evictsChat = evictsChat - } - - /// Build the disclosure for a chosen image `descriptor` given its residency `classification` (the - /// SINGLE input — design D4/D5). The RAM line reflects co-resident (~7 GB, chat stays alive) vs - /// evicts-chat (~24 GB, chat pauses). - public static func make(descriptor: ModelDescriptor, - classification: ImageResidencyClass) -> ImageCostDisclosure { - let gb = Double(descriptor.residencyBytes) / Double(1024 * 1024 * 1024) - let gbRounded = (gb * 10).rounded() / 10 - let evicts = (classification == .evictsChat) - let ram: String - if evicts { - ram = "~\(format(gbRounded)) GB resident — this PAUSES the chat model while it paints (chat resumes when it's done)." - } else { - ram = "~\(format(gbRounded)) GB resident — co-resides with chat (the assistant keeps talking while it paints)." - } - return ImageCostDisclosure( - ram: ram, - heat: "Sustained GPU diffusion — the M5 neural accelerators run hot for the duration.", - latency: "Seconds to tens of seconds per image at default steps.", - evictsChat: evicts - ) - } - - private static func format(_ v: Double) -> String { - v == v.rounded() ? String(Int(v)) : String(format: "%.1f", v) - } -} - -/// The pre-flight "busy painting" decision (design D5, task 5.2) — a PURE function over the residency -/// class. It is the runtime/sink's pre-flight state decision input, the SAME class value the disclosure -/// reads (task 2.3). `.busyPainting` ↔ `.evictsChat`; `.coResident` otherwise. It maps onto the seam's -/// existing `MediaResidencyNote` (`ai-media-runtime`) so the canvas/rail surface is unchanged — this -/// slice does NOT introduce a second busy-state type, it derives the seam's note from its own classifier. -public enum ImageBusyPaintingState: Equatable, Sendable { - /// Chat stays available (Q4 co-resident) — no busy banner. - case coResident - /// Chat is unavailable while FP16 paints — surface the calm, bounded "busy painting" state (never an - /// `NSAlert`, never raw error text). Chat resumes when the gen finishes / parks. - case busyPainting - - /// Derive the pre-flight state from the residency classification (the single shared input). - public init(_ classification: ImageResidencyClass) { - self = (classification == .evictsChat) ? .busyPainting : .coResident - } - - /// Bridge to the seam's existing `MediaResidencyNote` (so the canvas/rail observer is unchanged). - public var residencyNote: MediaResidencyNote { - switch self { - case .coResident: return .coResident - case .busyPainting: return .busyPainting - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/Image/ImageModelCatalog.swift b/Sources/ThreeFingerSwitcher/AI/Media/Image/ImageModelCatalog.swift deleted file mode 100644 index 8b72937..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/Image/ImageModelCatalog.swift +++ /dev/null @@ -1,147 +0,0 @@ -import Foundation - -/// The image-role `ModelDescriptor` variant table this slice supplies (`ai-local-image-generation`, -/// design D3 / tasks 1.1–1.3). It does NOT redefine `ModelDescriptor` — it CONSUMES the §C1 type -/// (`AI/ModelRegistry.swift`, the live `ModelDescriptor`) and the fleet's `ModelRegistry` (`AI/Fleet/`), -/// supplying two honest variants the fleet registers + budgets: -/// - **Q4 (default, ~7 GB resident)** — CO-RESIDES with chat + ternary + KV under the 48 GB budget. -/// - **FP16 (opt-in, ~24 GB resident)** — EVICTS chat (above the planner's FP16 threshold). -/// Each carries `role: .image`, `lane: .gpu`, `provider: .onDevice`, and the supported capability TAGS -/// (`"image"`, `"img2img"`). -/// -/// REAL MODEL (Wave 2 / `MFluxImageRuntime`): both variants are **FLUX.2 Klein 4B** — the OPEN, -/// **Apache-2.0** FLUX.2 variant (`black-forest-labs/FLUX.2-klein-4B`, ungated, commercial-use OK), -/// served by `flux-2-swift-mlx`'s `Flux2Pipeline(model: .klein4B)`. Q4 is the package's on-the-fly -/// int4 quantization (~7 GB resident), FP16 is the bf16 weights (~24 GB resident). We DELIBERATELY do -/// NOT use FLUX.2-dev (BFL non-commercial) or Klein 9B (non-commercial) — only the Apache-2.0 Klein 4B -/// ships here. The text encoder is Qwen3-4B (Apache 2.0) and the VAE is the small-decoder (Apache 2.0). -/// -/// CAPABILITY TAGS (REAL): Klein 4B supports text-to-image (`.image`) and **conditioning-mode -/// image-to-image** (`.img2img`, 1–4 reference images via `generateImageToImage`). It is NOT wired for -/// `.inpaint`: FLUX.2 I2I is attention-conditioning (it does not consume a per-pixel mask), and the -/// package's mask-based path (`Flux2MaskedInpaintingChain`) needs a first-class mask the `MediaRequest` -/// seam does not carry (`seed: Data?` only — no mask field, by design D6). So `.inpaint` is DROPPED so a -/// seed-with-alpha request fails cleanly at the validator rather than silently ignoring the mask. -/// -/// NB on the capability shape: §C1 sketches `capabilities: Set` (image/video TAGS). The live -/// `ModelDescriptor.capabilities` is `Set` (the project's already-shipped capability set). So -/// the descriptor's `Modality` set advertises what the chat layer needs (`.text`/`.vision` for the seed), -/// and the STRING tags (`"image"`/`"img2img"`/`"inpaint"`) — which the seam's seed-capability gate reads -/// — live on `ImageModelDescriptorTags` keyed by descriptor id (kept here so we add NO field to the -/// pinned `ModelDescriptor` and NO field to the pinned `MediaRequest`/`MediaParameters`). -/// -/// The persisted `imageModelID` (addendum §1 key, OWNED by `ai-full-potential-toggle` — consumed, never -/// redefined here) selects which variant is used; the default is the Q4 id. An unknown id is REJECTED, -/// never silently coerced to a default (design D3 / spec scenario). -/// -/// MLX-free Core — `swift test`-verified (the byte/role/lane/provider/capabilities assertions + selection). -public enum ImageModelCatalog { - - // MARK: - Stable ids - - /// The default Q4 image model id (matches the fleet roster's `image-q4` so the same descriptor is the - /// one the registry already budgets — this slice does not fork the roster's ids). - public static let q4ID = "image-q4" - /// The opt-in FP16 image model id (matches the fleet roster's `image-fp16`). - public static let fp16ID = "image-fp16" - - /// The id selected when `imageModelID` is unset (the co-resident default — design D3). - public static let defaultID = q4ID - - // MARK: - Honest resident footprints - - public static let q4ResidencyBytes: UInt64 = 7 * 1024 * 1024 * 1024 // ~7 GB — co-resides - public static let fp16ResidencyBytes: UInt64 = 24 * 1024 * 1024 * 1024 // ~24 GB — evicts chat - - // MARK: - Capability tags (§C1 string tags, kept off the pinned ModelDescriptor) - - /// The image capability tags a variant supports — what `ai-media-runtime`'s seed gate reads to decide - /// img2img is allowed. Both Klein 4B variants support t2i + conditioning-mode i2i (`.image`, - /// `.img2img`); a hypothetical t2i-only variant would advertise only `.image` (and a seed against it - /// would be an error, not a degrade). `.inpaint` is intentionally absent — see the type doc (the FLUX.2 - /// I2I path is attention-conditioning, not mask-based, and the seam carries no mask field). - public enum ImageCapabilityTag: String, Codable, Sendable, CaseIterable { - case image - case img2img - } - - /// The supported tags per descriptor id. Both shipped Klein 4B variants are text-to-image AND - /// conditioning-mode image-to-image capable (`.img2img`), but NOT inpaint. - public static let capabilityTags: [String: Set] = [ - q4ID: [.image, .img2img], - fp16ID: [.image, .img2img] - ] - - /// The supported tags for `id` (empty for an id this catalog does not own — a non-image descriptor). - public static func tags(for id: String) -> Set { - capabilityTags[id] ?? [] - } - - /// Whether `id` is a seed-capable image descriptor (advertises `img2img`). The seed/inpaint static - /// gate (task 3.1, design D6) reads this — a seed against a non-`img2img` id is a `MediaError`, never - /// a silent text-to-image fallback. - public static func isSeedCapable(_ id: String) -> Bool { - tags(for: id).contains(.img2img) - } - - // MARK: - The descriptor variants (§C1 type, verbatim fields) - - /// The Q4 default descriptor — **FLUX.2 Klein 4B, on-the-fly int4** (~7 GB resident, co-resident). - /// `role: .image`, `lane: .gpu`, `provider: .onDevice`. The `downloadURL` is the REAL Apache-2.0 Klein - /// 4B repo (`black-forest-labs/FLUX.2-klein-4B`, ungated) — `MFluxImageRuntime` serves it through - /// `flux-2-swift-mlx` (`Flux2Pipeline(model: .klein4B)` + the package's own multi-file HF download). - public static let q4Descriptor = ModelDescriptor( - id: q4ID, - displayName: "FLUX.2 Klein 4B (4-bit, ~7 GB) — co-resides with chat", - sizeBytes: 7 * 1024 * 1024 * 1024, - integritySHA: "hub-verified", - downloadURL: URL(string: "https://huggingface.co/black-forest-labs/FLUX.2-klein-4B")!, - capabilities: [.text, .vision], // text prompt + a vision seed frame - quantization: .qat4bit, - maxContextTokens: 8_192, - role: .image, - lane: .gpu, - provider: .onDevice, - residencyBytes: q4ResidencyBytes - ) - - /// The FP16 opt-in descriptor — **FLUX.2 Klein 4B, bf16** (~24 GB resident, evicts chat). Same - /// role/lane/provider + repo as Q4; heavier resident bytes because the transformer + Qwen3 encoder load - /// un-quantized. Apache-2.0, ungated — NOT FLUX.2-dev (non-commercial) and NOT Klein 9B (non-commercial). - public static let fp16Descriptor = ModelDescriptor( - id: fp16ID, - displayName: "FLUX.2 Klein 4B (bf16, ~24 GB) — pauses chat while it paints", - sizeBytes: 24 * 1024 * 1024 * 1024, - integritySHA: "hub-verified", - downloadURL: URL(string: "https://huggingface.co/black-forest-labs/FLUX.2-klein-4B")!, - capabilities: [.text, .vision], - quantization: .fp16, - maxContextTokens: 8_192, - role: .image, - lane: .gpu, - provider: .onDevice, - residencyBytes: fp16ResidencyBytes - ) - - /// Both variants, Q4 first (the default). These are the descriptors the fleet's `ModelRegistry` - /// registers (the `FleetRoster.standard` already lists them by the SAME ids — task 1.3). - public static let descriptors: [ModelDescriptor] = [q4Descriptor, fp16Descriptor] - - // MARK: - Selection (task 1.2) - - /// Resolve the selected image descriptor for a persisted `imageModelID`. - /// - `nil`/empty → the Q4 default (co-resident). - /// - the FP16 id → the FP16 descriptor. - /// - any id this catalog does not own → REJECTED (`nil`), never silently coerced to a default - /// (design D3 / spec "imageModelID selects the variant" scenario). The caller surfaces the - /// rejection (a clean failure), it does not fall back. - public static func selected(imageModelID: String?) -> ModelDescriptor? { - guard let id = imageModelID, !id.isEmpty else { return q4Descriptor } - return descriptors.first { $0.id == id } - } - - /// True iff `id` names a known image variant (a valid selection). An unknown id → false (rejected). - public static func isKnown(_ id: String) -> Bool { - descriptors.contains { $0.id == id } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/Image/ImageRequestValidation.swift b/Sources/ThreeFingerSwitcher/AI/Media/Image/ImageRequestValidation.swift deleted file mode 100644 index db82664..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/Image/ImageRequestValidation.swift +++ /dev/null @@ -1,102 +0,0 @@ -import Foundation - -/// Pure validation of a `MediaRequest` for the LOCAL IMAGE backend (design D6, task 3.1). The image -/// runtime (native-linked) and the stub both run this at the boundary BEFORE any compute, so a malformed -/// request, an out-of-range parameter, or a seed against a non-seed-capable descriptor is a `MediaError` -/// VALUE — never a silent text-only fallback, never compute on a bad request. -/// -/// It validates against the supported ranges of the CHOSEN image descriptor (selected via `imageModelID`, -/// resolved by `ImageModelCatalog`), and the descriptor's seed CAPABILITY TAGS (`ImageModelCatalog.tags`): -/// a non-nil `seed` (img2img) statically REQUIRES a descriptor advertising `img2img`. This mirrors the -/// chat runtime's vision-required static rule (no degrade): a seed against a t2i-only selection maps to a -/// `MediaError` at the boundary and is surfaced as a bounded, non-blocking failure (the seam carries only -/// `MediaRequest.seed: Data?` — this slice does NOT add a mask field; design D6). -/// -/// MLX-free Core — `swift test`-verified (valid t2i passes; valid img2img w/ seed-capable descriptor -/// passes; seed vs non-seed descriptor → mismatch error; out-of-range params rejected). -public enum ImageRequestValidator { - - /// The supported parameter ranges for an image generation. Aligned with the seam's `generate_image` - /// args schema (`MediaToolContributor.imageArgsSchemaJSON`: width/height 64…2048, steps 1…100) so a - /// route the schema admitted validates here too; guidance is an open positive range (nil → backend - /// default). These are the image-backend bounds — the descriptor's "supported ranges" the spec names. - public struct Bounds: Sendable, Equatable { - public var minDimension: Int - public var maxDimension: Int - public var minSteps: Int - public var maxSteps: Int - public var maxGuidance: Double - - public init(minDimension: Int = 64, maxDimension: Int = 2048, - minSteps: Int = 1, maxSteps: Int = 100, - maxGuidance: Double = 50) { - self.minDimension = minDimension - self.maxDimension = maxDimension - self.minSteps = minSteps - self.maxSteps = maxSteps - self.maxGuidance = maxGuidance - } - - public static let `default` = Bounds() - } - - /// Validate `request` against `descriptor` (the chosen image variant) and `bounds`. - /// - /// Returns `nil` on success; a `MediaError` VALUE on the first failure (the caller maps it through - /// `AIError.message(for:)` → a clean bounded headline, never a degrade). Order: kind → seed capability - /// → param bounds. A seed-bearing request against a non-`img2img` descriptor is the mismatch case. - public static func validate(_ request: MediaRequest, - descriptor: ModelDescriptor, - bounds: Bounds = .default) -> MediaError? { - - // (1) KIND — this backend serves images only. A non-image request here is a routing bug (the - // contributor only routes `.image` to the image runtime) — surface it, don't paint. - guard request.kind == .image else { - return .noCapableBackend(kind: request.kind) - } - - // (2) SEED CAPABILITY — a non-nil seed (img2img / inpaint) STATICALLY requires a seed-capable - // descriptor (advertises `img2img`). Mismatch → a clean `MediaError`, NEVER a silent t2i fallback - // (design D6). A nil seed is plain text-to-image and needs no seed capability. - if request.seed != nil, !ImageModelCatalog.isSeedCapable(descriptor.id) { - // The seed cannot be honored by this selection; surface it as a generation failure carrying a - // clean headline (the boundary's job is to refuse, not degrade). `.seedRequired` would imply a - // missing seed; here the seed is PRESENT but the descriptor can't use it — a capability - // mismatch, surfaced as a clean generationFailed headline. - return .generationFailed(headline: "This image model can't generate from a source image. Choose a model that supports image-to-image.") - } - - // (3) PARAM BOUNDS — size + steps within the descriptor's supported ranges. Out-of-range → reject - // (a route the schema let through with absurd values, or a programmatic request, never paints - // garbage). Guidance, when present, must be a sane positive scale. - let size = request.parameters.size - if size.width < bounds.minDimension || size.width > bounds.maxDimension - || size.height < bounds.minDimension || size.height > bounds.maxDimension { - return .generationFailed(headline: "That image size is out of range (\(bounds.minDimension)–\(bounds.maxDimension) px per side).") - } - let steps = request.parameters.steps - if steps < bounds.minSteps || steps > bounds.maxSteps { - return .generationFailed(headline: "That step count is out of range (\(bounds.minSteps)–\(bounds.maxSteps)).") - } - if let g = request.parameters.guidance, (g < 0 || g > bounds.maxGuidance) { - return .generationFailed(headline: "That guidance value is out of range (0–\(Int(bounds.maxGuidance))).") - } - - return nil - } - - /// Convenience: validate the request against the descriptor selected by `imageModelID`. An UNKNOWN - /// `imageModelID` is a rejected selection (design D3) — surfaced as `.noCapableBackend(.image)` rather - /// than silently coerced to the default. - public static func validate(_ request: MediaRequest, - imageModelID: String?, - bounds: Bounds = .default) -> MediaError? { - if let id = imageModelID, !id.isEmpty, !ImageModelCatalog.isKnown(id) { - return .noCapableBackend(kind: .image) // unknown selection → no usable image backend - } - guard let descriptor = ImageModelCatalog.selected(imageModelID: imageModelID) else { - return .noCapableBackend(kind: .image) - } - return validate(request, descriptor: descriptor, bounds: bounds) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/Image/ImageResidencyClass.swift b/Sources/ThreeFingerSwitcher/AI/Media/Image/ImageResidencyClass.swift deleted file mode 100644 index 75edc3a..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/Image/ImageResidencyClass.swift +++ /dev/null @@ -1,95 +0,0 @@ -import Foundation - -/// The pure residency CLASSIFICATION this slice OWNS (design D4, tasks 2.1–2.3). Given a chosen image -/// `ModelDescriptor` (its `residencyBytes`), the fleet's CURRENT resident set, and the unified-memory -/// ceiling (CONSUMED from `ai-model-fleet`, never re-derived here), it classifies the outcome of making -/// that image model resident as either: -/// - `.coResident` — it fits alongside the resident chat + ternary + KV under the ceiling (Q4 ~7 GB); -/// - `.evictsChat` — admitting it would force the GPU-lane chat model out (FP16 ~24 GB). -/// -/// IMPORTANT division of labor (design D4): the fleet OWNS the eviction *decision* (`ensureResident`); -/// this slice only CLASSIFIES + CONSUMES. The classification — not a fresh eviction decision — is the -/// SINGLE input to BOTH the pre-fire cost disclosure (§5.1, `ImageCostDisclosure`) AND the runtime's -/// honest "busy painting" pre-flight state (§5.2). So both surfaces read the same truth and can never -/// disagree. -/// -/// PURE: the resident set + ceiling + KV reserve are INJECTED — there is NO real free-RAM probe here -/// (mirrors `ConcurrencyBudget`/`ResidencyPlanner`'s injected-probe pattern), so the math is -/// `swift test`-verified with fixed inputs, no Metal, no real weights (task 2.2). -public enum ImageResidencyClass: Equatable, Sendable { - /// The image model co-resides with chat — chat stays available while it paints. - case coResident - /// The image model evicts the chat model — the companion goes quiet ("busy painting"). - case evictsChat -} - -/// The pure classifier (design D4). It mirrors the fleet `ResidencyPlanner`'s co-residency arithmetic, but -/// only to CLASSIFY (co-reside vs evict-chat) — it does NOT compute an eviction plan or touch weights. -public struct ImageResidencyClassifier: Sendable { - - /// The KV-cache headroom reserved alongside the resident weight set (the live decode caches). Held - /// out of the ceiling so a co-residency classification never packs weights edge-to-edge — the SAME - /// reserve concept the fleet's `ResidencyPlanner` uses (consumed convention, default kept aligned). - public var kvReserveBytes: UInt64 - - /// The resident-footprint threshold above which an `.image` model is the HEAVY (FP16) variant that - /// evicts chat by GPU-lane EXCLUSIVITY — not just by byte-fit. This MIRRORS the fleet - /// `ResidencyPlanner.fp16ImageThresholdBytes` (consumed convention, default kept aligned): the GPU lane - /// cannot run a heavy diffusion AND stream chat concurrently (~153 GB/s bus contention; the companion - /// goes quiet while it paints), so an FP16 image evicts chat even when ~45 GB technically fits under 48. - /// A Q4 image (below the threshold) is bandwidth-light enough to co-reside. - public var fp16ImageThresholdBytes: UInt64 - - public init(kvReserveBytes: UInt64 = 4 * 1024 * 1024 * 1024, // ~4 GB KV headroom (aligned w/ planner) - fp16ImageThresholdBytes: UInt64 = 16 * 1024 * 1024 * 1024) { // > 16 GB resident ⇒ heavy (aligned) - self.kvReserveBytes = kvReserveBytes - self.fp16ImageThresholdBytes = fp16ImageThresholdBytes - } - - /// Classify making `image` resident against the injected `resident` set under the injected `ceilingBytes`. - /// - /// Rules (D4), encoded once + tested: - /// - The chat model is the GPU-lane `.chat` member; if it is NOT currently resident there is nothing - /// to evict → the outcome cannot be `.evictsChat` for it (classify against what's actually there). - /// - **Co-resident:** the image model's bytes + ALL currently-resident on-device members + the KV - /// reserve fit within the ceiling → `.coResident`. - /// - **Evicts chat:** otherwise admitting the image model would require shedding the GPU-lane chat - /// model to fit → `.evictsChat`. (The CPU-lane ternary is bandwidth-frugal and is never the victim; - /// only the GPU-lane chat is, matching the planner's lane exclusivity.) - /// - /// - Parameters: - /// - image: the chosen image descriptor (its `residencyBytes` is the load to admit). - /// - resident: the fleet's CURRENT resident descriptors (injected — `registry.resident()`). - /// - ceilingBytes: the unified-memory ceiling (injected from the fleet — the 48 GB budget, or the - /// live free bytes, whichever the caller passes; this slice does NOT re-derive 48 GB). - public func classify(image: ModelDescriptor, - resident: [ModelDescriptor], - ceilingBytes: UInt64) -> ImageResidencyClass { - // Only on-device members occupy bytes (cloud members never do — filter defensively). - let onDeviceResident = resident.filter { $0.provider == .onDevice } - // If the image model is ALREADY resident, it is trivially co-resident (warm — nothing to admit). - if onDeviceResident.contains(where: { $0.id == image.id }) { return .coResident } - - let chatResident = onDeviceResident.contains { $0.role == .chat && $0.lane == .gpu } - - // HEAVY GPU gen (an FP16 image above the threshold) → GPU-lane EXCLUSIVITY: it evicts chat even - // when the bytes would technically fit, mirroring the fleet planner's `isHeavyGPUGen` rule. A Q4 - // image (below the threshold) falls through to the byte-fit test and co-resides. - let isHeavyImage = image.lane == .gpu - && image.role == .image - && image.residencyBytes > fp16ImageThresholdBytes - if isHeavyImage { - return chatResident ? .evictsChat : .coResident - } - - let residentBytes = onDeviceResident.reduce(UInt64(0)) { $0 &+ $1.residencyBytes } - let total = residentBytes &+ image.residencyBytes &+ kvReserveBytes - - // Co-resident iff everything currently loaded PLUS the image model PLUS the KV reserve fit. - if total <= ceilingBytes { return .coResident } - - // Over the ceiling → admitting the image model requires evicting a GPU-lane occupant. The expected - // (and only) GPU-lane weight victim is the chat model; if it is resident, this is an evict-chat. - return chatResident ? .evictsChat : .coResident - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/Image/StubImageRuntime.swift b/Sources/ThreeFingerSwitcher/AI/Media/Image/StubImageRuntime.swift deleted file mode 100644 index d079090..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/Image/StubImageRuntime.swift +++ /dev/null @@ -1,147 +0,0 @@ -import Foundation - -/// A deterministic, weight-free image `MediaRuntime` for `swift test` (design D8, tasks 3.2–3.3). Unlike -/// the generic `StubMediaRuntime` (which plays an arbitrary scripted shape), this stub models the IMAGE -/// backend's real generate shape: it runs `ImageRequestValidator` at the boundary (so the seed-capability -/// / param-bounds gate is exercised end-to-end), emits ordered `.step(index:total:preview:)` progress, -/// WRITES a real placeholder PNG to disk, and terminates in `.finished(MediaAsset)` whose `kind == .image` -/// and dimensions match the request. It honors cancellation (the stream ends WITHOUT a `.finished`, never -/// throwing — cancellation is not a failure, design D10). -/// -/// It lets the full route → progress → asset → (Files-entry shape) path be tested without MLX/GPU, -/// including the seed/img2img branch (a present seed routes through the seed-capable descriptor). -/// -/// MLX-free Core (test + integration support). -public final class StubImageRuntime: MediaRuntime, @unchecked Sendable { - - public let capabilities: Set = [.image] - - /// The descriptor this stub generates as (its capability tags gate the seed branch). Defaults to the - /// Q4 seed-capable variant so an img2img request passes; pass `ImageModelCatalog.q4Descriptor` etc. - private let descriptor: ModelDescriptor - /// Where placeholder PNGs are written (a temp dir by default — the asset URL is readable). - private let outputDirectory: URL - /// How many `.step`s to emit before finishing (defaults to the request's step count, capped for speed). - private let maxSteps: Int - /// An optional per-step delay (nanoseconds) so a test can reliably cancel a gen MID-FLIGHT. Default 0 - /// (instant — the common path stays fast). A non-zero value models the real diffusion's step latency. - private let perStepDelayNanos: UInt64 - /// Records the requests driven (so tests assert seed/kind/params threaded through). - public private(set) var receivedRequests: [MediaRequest] = [] - private let lock = NSLock() - - public init(descriptor: ModelDescriptor = ImageModelCatalog.q4Descriptor, - outputDirectory: URL = FileManager.default.temporaryDirectory - .appendingPathComponent("StubImageRuntime-\(UUID().uuidString)", isDirectory: true), - maxSteps: Int = 4, - perStepDelayNanos: UInt64 = 0) { - self.descriptor = descriptor - self.outputDirectory = outputDirectory - self.maxSteps = maxSteps - self.perStepDelayNanos = perStepDelayNanos - } - - public func generate(_ request: MediaRequest) -> AsyncThrowingStream { - lock.lock(); receivedRequests.append(request); lock.unlock() - let descriptor = self.descriptor - let outputDirectory = self.outputDirectory - let stepBudget = max(1, min(maxSteps, request.parameters.steps)) - let perStepDelayNanos = self.perStepDelayNanos - - return AsyncThrowingStream { continuation in - let task = Task { - // (1) BOUNDARY VALIDATION — the same gate the real runtime runs. A bad request (seed vs - // non-seed descriptor, out-of-range params) THROWS a clean `MediaError` and never paints. - if let err = ImageRequestValidator.validate(request, descriptor: descriptor) { - continuation.finish(throwing: err) - return - } - - // (2) ORDERED STEPS — ascending index, with a tiny placeholder preview frame. - let preview = Data(MediaSeedValidation.pngMagic) - for i in 0.. 0 { - do { try await Task.sleep(nanoseconds: perStepDelayNanos) } - catch { continuation.finish(); return } // sleep cancelled → stop, no .finished - } else { - await Task.yield() - } - } - if Task.isCancelled { continuation.finish(); return } - - // (3) WRITE the placeholder PNG + emit the terminal asset (dimensions match the request). - do { - let url = try Self.writePlaceholderPNG(in: outputDirectory) - let asset = MediaAsset(url: url, kind: .image, - width: request.parameters.size.width, - height: request.parameters.size.height) - continuation.yield(.finished(asset)) - continuation.finish() - } catch { - // A write failure is a real `.failed` (mapped to a clean MediaError) — never a false done. - continuation.finish(throwing: MediaError.outputWriteFailed(detail: String(describing: error))) - } - } - continuation.onTermination = { _ in task.cancel() } - } - } - - /// Write a minimal but VALID, readable PNG placeholder to `dir` (the asset URL the sink turns into a - /// Files-band `.fileEntry`). A real 1×1 PNG so the file is decodable, not just a magic-number stub. - static func writePlaceholderPNG(in dir: URL) throws -> URL { - try FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true) - let url = dir.appendingPathComponent("image-\(UUID().uuidString).png") - try minimalPNG.write(to: url) - return url - } - - /// A hand-rolled, byte-valid 1×1 opaque-black PNG (no AppKit/MLX dependency — Core stays portable). - /// Signature + IHDR(1×1, 8-bit RGB) + IDAT (zlib stored block of one filtered RGB scanline) + IEND. - static let minimalPNG: Data = { - func be32(_ v: UInt32) -> [UInt8] { - [UInt8((v >> 24) & 0xFF), UInt8((v >> 16) & 0xFF), UInt8((v >> 8) & 0xFF), UInt8(v & 0xFF)] - } - func crc32(_ bytes: [UInt8]) -> UInt32 { - var crc: UInt32 = 0xFFFF_FFFF - for b in bytes { - crc ^= UInt32(b) - for _ in 0..<8 { - crc = (crc & 1) != 0 ? (crc >> 1) ^ 0xEDB8_8320 : (crc >> 1) - } - } - return crc ^ 0xFFFF_FFFF - } - func chunk(_ type: String, _ payload: [UInt8]) -> [UInt8] { - let typeBytes = Array(type.utf8) - var out = be32(UInt32(payload.count)) - out += typeBytes - out += payload - out += be32(crc32(typeBytes + payload)) - return out - } - // Signature. - var png: [UInt8] = [0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A] - // IHDR: width=1, height=1, bitDepth=8, colorType=2 (RGB), compression=0, filter=0, interlace=0. - let ihdr = be32(1) + be32(1) + [8, 2, 0, 0, 0] - png += chunk("IHDR", ihdr) - // IDAT: one scanline = filter byte (0) + RGB (0,0,0). zlib: header 0x78 0x01, one STORED block. - let raw: [UInt8] = [0x00, 0x00, 0x00, 0x00] // filter + R,G,B - // adler32 of `raw`. - var a: UInt32 = 1, b: UInt32 = 0 - for byte in raw { a = (a + UInt32(byte)) % 65521; b = (b + a) % 65521 } - let adler = (b << 16) | a - let len = UInt16(raw.count) - var zlib: [UInt8] = [0x78, 0x01, 0x01] // CMF/FLG + final stored block - zlib += [UInt8(len & 0xFF), UInt8((len >> 8) & 0xFF)] // LEN (little-endian) - zlib += [UInt8(~len & 0xFF), UInt8((~len >> 8) & 0xFF)]// NLEN - zlib += raw - zlib += be32(adler) - png += chunk("IDAT", zlib) - png += chunk("IEND", []) - return Data(png) - }() -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/MediaArgs.swift b/Sources/ThreeFingerSwitcher/AI/Media/MediaArgs.swift deleted file mode 100644 index a335696..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/MediaArgs.swift +++ /dev/null @@ -1,76 +0,0 @@ -import Foundation - -/// Parses a routed media call's `argumentsJSON` into a `MediaRequest` (design D2). The router's JSON is a -/// HINT — it is parsed tolerantly (missing fields fall back to defaults), then the SINK re-resolves the -/// seed through the existing capture seams (the JSON only names WHICH capture, never carries bytes). A -/// well-formed args object validates; a malformed one falls back to a prompt-only request rather than -/// failing the whole route (the model still gets to paint something it can refine). Pure. MLX-free Core. -public struct MediaArgs: Sendable, Equatable { - public var prompt: String - public var width: Int? - public var height: Int? - public var steps: Int? - public var durationMs: Int? - /// The named capture the seed image should come from (`screenRegion` / `clipboardImage`), or nil for - /// text-to-media. The sink resolves the actual bytes through the existing capture seams. - public var seedImage: SeedHandle? - - public enum SeedHandle: String, Sendable, Equatable { - case screenRegion - case clipboardImage - } - - public init(prompt: String, width: Int? = nil, height: Int? = nil, steps: Int? = nil, - durationMs: Int? = nil, seedImage: SeedHandle? = nil) { - self.prompt = prompt - self.width = width - self.height = height - self.steps = steps - self.durationMs = durationMs - self.seedImage = seedImage - } - - /// Parse a route's `argumentsJSON` + the user's text. The prompt prefers the args' `prompt`, falling - /// back to the user text when the args omit it (so a bare "draw a cat" still paints). Never throws — - /// malformed JSON yields a prompt-only `MediaArgs`. - public static func parse(argumentsJSON: String, userText: String) -> MediaArgs { - let fallbackPrompt = userText.trimmingCharacters(in: .whitespacesAndNewlines) - guard let data = argumentsJSON.data(using: .utf8), - let obj = (try? JSONSerialization.jsonObject(with: data)) as? [String: Any] else { - return MediaArgs(prompt: fallbackPrompt) - } - let prompt = (obj["prompt"] as? String)?.trimmingCharacters(in: .whitespacesAndNewlines) - let resolvedPrompt = (prompt?.isEmpty == false ? prompt! : fallbackPrompt) - return MediaArgs( - prompt: resolvedPrompt, - width: intValue(obj["width"]), - height: intValue(obj["height"]), - steps: intValue(obj["steps"]), - durationMs: intValue(obj["durationMs"]), - seedImage: (obj["seedImage"] as? String).flatMap(SeedHandle.init(rawValue:)) - ) - } - - /// Build a `MediaRequest` for `kind`, folding in a resolved seed (PNG bytes) when one was supplied. - /// `durationMs` is honored only for `.video`. - public func request(kind: MediaKind, seed: Data?) -> MediaRequest { - let size: MediaSize - if let w = width, let h = height { size = MediaSize(width: w, height: h) } - else { size = .square1024 } - var params = MediaParameters(size: size, steps: steps ?? 28) - if kind == .video { params.durationMs = durationMs } - return MediaRequest(prompt: prompt, seed: seed, kind: kind, parameters: params) - } - - /// True when the route NAMED a seed image (so a tool is being used as img2img/img2video) — the sink - /// then requires that capture to resolve, else `MediaError.seedRequired`. - public var requiresSeed: Bool { seedImage != nil } - - // Accept an Int or a JSON number that decoded as Double/NSNumber. - private static func intValue(_ any: Any?) -> Int? { - if let i = any as? Int { return i } - if let d = any as? Double { return Int(d) } - if let n = any as? NSNumber { return n.intValue } - return nil - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/MediaCanvasResolve.swift b/Sources/ThreeFingerSwitcher/AI/Media/MediaCanvasResolve.swift deleted file mode 100644 index 9e297f0..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/MediaCanvasResolve.swift +++ /dev/null @@ -1,52 +0,0 @@ -import Foundation - -/// The pure canvas RESOLVE model (design D7, §6.3) — the canonical two-finger compass over a finished -/// media preview. **DOWN (only when the canvas is at its top) extracts** the result (save / paste / -/// set-as); **RIGHT discards** the preview (the asset is already durably in the gallery, so a discard -/// never loses the file). A sub-threshold two-finger scroll does NOT resolve (reading the preview never -/// extracts or discards it) — the resolve excursion sits ABOVE incidental scroll, mirroring the AI -/// canvas's `canvasResolveThreshold`. Pure + `swift test`-verified; the native overlay drives it. MLX-free -/// Core. -public enum MediaCanvasResolution: Equatable, Sendable { - /// DOWN-at-top past threshold → extract (save / paste / set-as). - case extract - /// RIGHT past threshold → discard the preview (file stays in the gallery). - case discard - /// Below threshold, or not at the top for a DOWN, or up (ignored) → no resolution. - case none -} - -/// What an extract DOES (the destinations the canonical "swipe-DOWN extracts" affords). The native -/// overlay turns the chosen destination into the concrete side effect (the file is already written). -public enum MediaExtractIntent: String, Equatable, Sendable { - case save // reveal / keep (it's already in the gallery — surface it) - case paste // paste into the front app - case setAs // set-as (wallpaper / etc.) -} - -public enum MediaCanvasResolver { - /// The resolve excursion threshold (normalized centroid travel) — ABOVE incidental two-finger scroll, - /// matching the AI canvas's `canvasResolveThreshold` so reading the preview never resolves it. - public static let resolveThreshold: Double = 0.22 - - /// Resolve a two-finger excursion over a finished preview. `dx`/`dy` are signed normalized centroid - /// travel (down = +y by the project's overlay convention); `atTop` is whether the canvas is scrolled - /// to its top (a DOWN extract requires it, per the compass). Returns `.none` for a non-terminal job, - /// a sub-threshold excursion, an up swipe, or a DOWN when not at the top. - public static func resolve(dx: Double, dy: Double, atTop: Bool, state: MediaJobState) -> MediaCanvasResolution { - // Only a TERMINAL preview resolves — a still-generating canvas is read-only. - guard state.isTerminal else { return .none } - let ax = abs(dx), ay = abs(dy) - // Sub-threshold scroll never resolves (reading ≠ extracting/discarding). - guard max(ax, ay) >= resolveThreshold else { return .none } - // Horizontal-dominant RIGHT → discard. (LEFT is treated as discard too — any horizontal dismiss.) - if ax >= ay { - return dx > 0 ? .discard : .discard - } - // Vertical-dominant: DOWN extracts ONLY at the top; UP is ignored. - if dy > 0 { - return atTop ? .extract : .none - } - return .none // up swipe → ignored - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/MediaError.swift b/Sources/ThreeFingerSwitcher/AI/Media/MediaError.swift deleted file mode 100644 index 1aea3b7..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/MediaError.swift +++ /dev/null @@ -1,77 +0,0 @@ -import Foundation - -/// The media-specific error taxonomy (design D10, addendum §1 "new error enums") — a clean -/// `LocalizedError` for ONLY the cases the shared `RuntimeError`/`TaskError`/`FleetError` cannot carry. -/// Vendor/OS errors (mflux/LTXV/ComfyUI, `Process`, `NSURLError`, `FileManager`) are mapped INTO this -/// taxonomy at the layer boundary (the sink / backend conformer, §8.2) BEFORE they reach a UI surface, so -/// feature/UI code never sees a raw vendor type. Every surface routes through `AIError.message(for:)` → -/// `AIPresentedError` (THE single translator), bounded + non-blocking — never an `NSAlert`, never raw -/// error text in a headline. -/// -/// Per-case `errorDescription`s are clean, user-facing sentences. Raw vendor/OS detail rides ONLY in the -/// opt-in `copyableDetails` (and logs), never the headline. **Cancellation is NOT a `MediaError`** — a -/// discarded/parked-then-discarded gen ends `.cancelled` (mapped to `RuntimeError.cancelled` / -/// `CancellationError`), never a `.failed` badge. MLX-free Core. -public enum MediaError: Error, Equatable { - /// No runtime advertises the requested kind (e.g. a video gen with no video provider configured). - case noCapableBackend(kind: MediaKind) - /// A tool authored as img2img / img2video ran with no resolvable seed image — never a fabricated - /// blank first frame (design D5). - case seedRequired - /// The supplied seed image could not be decoded to PNG (an undecodable capture). - case seedInvalid - /// The backend reported a generation failure. Carries a CLEAN headline already (the boundary mapped - /// the vendor error → a clean string); raw text rode into `copyableDetails`/logs at the boundary. - case generationFailed(headline: String) - /// The finished asset could not be written to the gallery (`FileManager` failure). The raw OS reason - /// rides in `detail` (copyable), never the headline. - case outputWriteFailed(detail: String? = nil) - /// The cloud-video per-day budget is exhausted — refused BEFORE any network call / spend (design D3). - case cloudBudgetExhausted - /// A cloud video provider is unreachable / not configured (distinct from budget-exhausted). - case cloudUnavailable - /// The selected video provider is DISABLED by a gate (e.g. local LTXV selected with the master toggle - /// off, or no provider configured) — a clean decline, NO compute, NO spend - /// (`ai-video-animation-generation`, task 8.1). Distinct from `.cloudUnavailable` (reachable-but-down) - /// and `.noCapableBackend` (no runtime advertises the kind). - case videoProviderDisabled -} - -extension MediaError: LocalizedError { - public var errorDescription: String? { - switch self { - case let .noCapableBackend(kind): - switch kind { - case .image: return "No image generator is available." - case .video: return "No video generator is available." - } - case .seedRequired: - return "This needs a source image. Capture a screen region or copy an image first." - case .seedInvalid: - return "That image couldn't be read as a source frame." - case let .generationFailed(headline): - // The headline is already clean (mapped at the boundary); surface it as-is. - return headline - case .outputWriteFailed: - return "The generated file couldn't be saved." - case .cloudBudgetExhausted: - return "Today's video budget is used up. It resets tomorrow." - case .cloudUnavailable: - return "The video service isn't reachable right now." - case .videoProviderDisabled: - return "Video generation isn't turned on." - } - } - - /// The opt-in copyable detail (the raw OS reason), for a "Show details / Copy" disclosure and logs - /// only — never the headline. `nil` when the headline already says everything. - public var copyableDetails: String? { - switch self { - case let .outputWriteFailed(detail): - return detail - case .noCapableBackend, .seedRequired, .seedInvalid, .generationFailed, - .cloudBudgetExhausted, .cloudUnavailable, .videoProviderDisabled: - return nil - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/MediaGallery.swift b/Sources/ThreeFingerSwitcher/AI/Media/MediaGallery.swift deleted file mode 100644 index 78aa6cc..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/MediaGallery.swift +++ /dev/null @@ -1,92 +0,0 @@ -import Foundation - -/// Output #1 — the generated-media GALLERY (design D6). A finished `MediaAsset.url` is written under a -/// dedicated, local-only, recoverable gallery root and surfaces as an ordinary Files-band `.fileEntry` -/// (reuse the band; do NOT build a new browser). The gallery honors the band's non-destructive scope: no -/// permanent delete, no overwrite (each asset is a uniquely-named file). -/// -/// MLX-free Core. The writer touches `FileManager` (that's the boundary where an OS failure maps to -/// `MediaError.outputWriteFailed`); the `.fileEntry` mapping is pure. -public protocol MediaGalleryWriting: Sendable { - /// Persist `bytes` for a finished generation under the gallery root and return the asset that - /// references the written file. Throws `MediaError.outputWriteFailed` (OS reason in details) on - /// failure — never a partial/false success. - func write(_ bytes: Data, kind: MediaKind, width: Int, height: Int, durationMs: Int?) throws -> MediaAsset - - /// The gallery root (so the Files band can list it). Local-only. - var root: URL { get } -} - -/// The on-disk gallery writer. The root defaults under Application Support (local-only, survives relaunch, -/// recoverable). Each asset gets a fresh UUID filename so a write NEVER overwrites an existing asset -/// (non-destructive scope, design D6). The file extension follows the kind (PNG image / MP4 video). -public struct MediaGallery: MediaGalleryWriting { - public let root: URL - private let fileManager: FileManager - - /// The default gallery root under Application Support: `…/ThreeFingerSwitcher/GeneratedMedia`. - public static func defaultRoot(fileManager: FileManager = .default) -> URL { - let base = (try? fileManager.url(for: .applicationSupportDirectory, in: .userDomainMask, - appropriateFor: nil, create: false)) - ?? fileManager.temporaryDirectory - return base.appendingPathComponent("ThreeFingerSwitcher", isDirectory: true) - .appendingPathComponent("GeneratedMedia", isDirectory: true) - } - - public init(root: URL? = nil, fileManager: FileManager = .default) { - self.root = root ?? MediaGallery.defaultRoot(fileManager: fileManager) - self.fileManager = fileManager - } - - public func write(_ bytes: Data, kind: MediaKind, width: Int, height: Int, - durationMs: Int?) throws -> MediaAsset { - let id = UUID() - let ext = MediaGallery.fileExtension(for: kind) - let url = root.appendingPathComponent("\(id.uuidString).\(ext)", isDirectory: false) - do { - try fileManager.createDirectory(at: root, withIntermediateDirectories: true) - // `.atomic` so a crash mid-write never leaves a half-file in the gallery listing. - try bytes.write(to: url, options: .atomic) - } catch { - // Map the FileManager/OS error at the boundary into the taxonomy (design D10). The raw OS - // reason rides in copyable details, never the headline. - throw MediaError.outputWriteFailed(detail: String(describing: error)) - } - return MediaAsset(id: id, url: url, kind: kind, width: width, height: height, durationMs: durationMs) - } - - /// The file extension for a kind. Images are PNG (the seam normalizes to PNG); video is MP4 (the - /// container the backends produce). Pure helper. - public static func fileExtension(for kind: MediaKind) -> String { - switch kind { - case .image: return "png" - case .video: return "mp4" - } - } -} - -// MARK: - .fileEntry mapping (output #1, §5.2) - -extension MediaAsset { - /// Map a gallery asset to a Files-band `FileEntry` — an ordinary entry the band lists, opens, - /// opens-with, and delivers (no new browser). Identity is the asset's PATH (via `FileEntry.init`, - /// which derives `id` from the standardized path), so re-listing the gallery never strobes the - /// highlight (design D6 / the band's path-stable identity rule). - func fileEntry(modificationDate: Date? = nil) -> FileEntry { - FileEntry(url: url, - name: url.lastPathComponent, - isDirectory: false, - modificationDate: modificationDate, - kind: kind.fileKind) - } -} - -extension MediaKind { - /// The Files-band `FileKind` row-glyph classification for this media kind. - var fileKind: FileKind { - switch self { - case .image: return .image - case .video: return .video - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/MediaGenSink.swift b/Sources/ThreeFingerSwitcher/AI/Media/MediaGenSink.swift deleted file mode 100644 index 9795711..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/MediaGenSink.swift +++ /dev/null @@ -1,306 +0,0 @@ -import Foundation - -/// The media job's observable progress sink — the canvas (output #2) + the parked feed (§7) subscribe to -/// it so they advance in lockstep with the generation, and so a heavy gen's "busy painting" state is -/// surfaced not hidden (design D9). The `MediaGenSink` calls these as it drives the runtime; a test spy -/// records them. Every method is non-throwing (observation never breaks the gen). MLX-free Core. -public protocol MediaJobObserving: Sendable { - /// Residency decided (before any compute) — `.busyPainting` means chat was evicted. - func didDecideResidency(_ note: MediaResidencyNote) - /// A diffusion step settled (live preview + the parked thinking badge). - func didStep(index: Int, total: Int, preview: Data?) - /// The gen finished with its terminal asset. - func didFinish(_ asset: MediaAsset) - /// The gen failed with a CLEAN headline (already routed through `AIError.message(for:)`). - func didFail(headline: String) - /// The gen was cancelled (a discard) — DISTINCT from a failure (no failed badge). - func didCancel() -} - -/// A no-op observer (a context with no canvas/rail wired) — a safe default. -public struct NoopMediaJobObserver: MediaJobObserving { - public init() {} - public func didDecideResidency(_ note: MediaResidencyNote) {} - public func didStep(index: Int, total: Int, preview: Data?) {} - public func didFinish(_ asset: MediaAsset) {} - public func didFail(headline: String) {} - public func didCancel() {} -} - -/// The route-loop executor for a routed media call (design D2/D3, §3). Invoked by the EXISTING route → -/// execute → continue loop when the router selects `generate_image`/`generate_video` — no new control -/// flow. It: -/// 1. resolves the EFFECTIVE write-policy tier via the injected `WritePolicyResolving` (a `.dangerous` -/// cloud-video tier is NEVER lowered); -/// 2. for `.confirm`/`.dangerous`, surfaces an AWAITING-APPROVAL step (DOWN=approve / RIGHT=skip) -/// BEFORE any compute or spend; -/// 3. enforces the cloud-video BUDGET CAP before the call (an exhausted budget → clean `.failed`/ -/// `.declined`, NO spend); -/// 4. resolves the SEED (img2img/img2video) from the existing captures (a missing-but-required seed → -/// `MediaError.seedRequired`; an undecodable seed → `.seedInvalid`); -/// 5. consults `ModelRegistry`/`ensureResident` (surfacing "busy painting" when the gen evicts chat); -/// 6. drives the runtime's `generate(_:)`, threading `MediaProgress` into the observer (canvas + parked -/// feed); -/// 7. writes the finished asset to the gallery (output #1); -/// 8. writes ONE `AuditRecord` for the terminal outcome; -/// 9. returns a `ToolStepResult` — `.done` (gallery path) / `.declined` / `.failed` (clean headline). -/// Cancellation is NOT a failure (design D10). -/// -/// Pure orchestration over injected seams — MLX-free Core, `swift test`-verified against `StubMediaRuntime`. -struct MediaGenSink: Sendable { - private let imageRuntime: MediaRuntime? - private let videoRuntime: MediaRuntime? - private let resolver: WritePolicyResolving - private let seed: MediaSeedResolving - private let gallery: MediaGalleryWriting - private let budget: MediaVideoBudgeting - private let residency: MediaResidencyCoordinator? - private let observer: MediaJobObserving - private let audit: AuditLog? - /// The session this call runs for (audit attribution + parked feed). A foreground call may pass a - /// fresh id; a parked subagent passes its session id. - private let sessionID: AgentSessionID - /// The runtime's model id for the residency check (image/video backend). nil → skip residency. - private let imageModelID: String? - private let videoModelID: String? - /// Whether this call is running in the background (parked) — recorded on the audit record. - private let isBackground: Bool - private let now: @Sendable () -> Date - - init(imageRuntime: MediaRuntime?, - videoRuntime: MediaRuntime?, - resolver: WritePolicyResolving = DescriptorWritePolicy(), - seed: MediaSeedResolving = NoSeed(), - gallery: MediaGalleryWriting, - budget: MediaVideoBudgeting, - residency: MediaResidencyCoordinator? = nil, - observer: MediaJobObserving = NoopMediaJobObserver(), - audit: AuditLog? = nil, - sessionID: AgentSessionID = AgentSessionID(), - imageModelID: String? = nil, - videoModelID: String? = nil, - isBackground: Bool = false, - now: @escaping @Sendable () -> Date = { Date() }) { - self.imageRuntime = imageRuntime - self.videoRuntime = videoRuntime - self.resolver = resolver - self.seed = seed - self.gallery = gallery - self.budget = budget - self.residency = residency - self.observer = observer - self.audit = audit - self.sessionID = sessionID - self.imageModelID = imageModelID - self.videoModelID = videoModelID - self.isBackground = isBackground - self.now = now - } - - /// The kind the routed tool generates (or nil for a non-media tool — defensive). - private func kind(for tool: String) -> MediaKind? { - switch tool { - case MediaTool.generateImage: return .image - case MediaTool.generateVideo: return .video - default: return nil - } - } - - private func runtime(for kind: MediaKind) -> MediaRuntime? { - switch kind { - case .image: return imageRuntime - case .video: return videoRuntime - } - } - - private func modelID(for kind: MediaKind) -> String? { - switch kind { - case .image: return imageModelID - case .video: return videoModelID - } - } - - func run(_ call: RoutedCall, gate: ApprovalGate) async -> ToolStepResult { - let tool = call.descriptor.name - guard let kind = kind(for: tool) else { - return failed(tool: tool, error: MediaError.noCapableBackend(kind: .image), - effectiveTier: call.descriptor.writePolicy, argsSummary: "") - } - - let args = MediaArgs.parse(argumentsJSON: call.route.argumentsJSON, userText: call.userText) - let argsSummary = AuditRedaction.summary(forRawArguments: call.route.argumentsJSON) - - // (1) Effective tier — `.dangerous` (cloud video) is NEVER lowered (the resolver enforces that). - let effectiveTier = resolver.effectiveTier(for: call.descriptor) - - // (3a) BUDGET — checked BEFORE approval AND before any spend for cloud video. An exhausted budget - // refuses with NO compute, NO network, NO spend (design D3). - let isCloudVideo = (kind == .video && effectiveTier == .dangerous) - if isCloudVideo, !budget.hasRemaining(now: now()) { - return failed(tool: tool, error: MediaError.cloudBudgetExhausted, - effectiveTier: effectiveTier, argsSummary: argsSummary) - } - - // (5) A capable runtime must exist (the contributor already gates this, but defend the dead end). - guard let runtime = runtime(for: kind), runtime.capabilities.contains(kind) else { - return failed(tool: tool, error: MediaError.noCapableBackend(kind: kind), - effectiveTier: effectiveTier, argsSummary: argsSummary) - } - - // (2) APPROVAL — `.confirm`/`.dangerous` pause as an awaiting-approval step BEFORE any compute or - // spend; `.auto` runs straight through (the resolver lowered it via an explicit whitelist). - switch effectiveTier { - case .auto: - break - case .confirm, .dangerous: - let review = Self.approvalReview(tool: tool, args: args, kind: kind, dangerous: effectiveTier == .dangerous) - switch await gate.awaitDecision(for: review) { - case .approve: - break - case .skip: - recordAudit(tool: tool, tier: effectiveTier, - outcome: .declined(reason: "skipped"), argsSummary: argsSummary) - return ToolStepResult(tool: tool, status: .declined(reason: "skipped"), - summary: "Skipped \(Self.kindNoun(kind)) generation.") - case .cancel: - // The whole canvas was discarded — a cancellation, NOT a failure (no audit-failed, no - // failed badge). Recorded as declined-cancelled (the loop's sentinel). - observer.didCancel() - recordAudit(tool: tool, tier: effectiveTier, - outcome: .declined(reason: Self.cancelledReason), argsSummary: argsSummary) - return ToolStepResult(tool: tool, status: .declined(reason: Self.cancelledReason), - summary: "Cancelled.") - } - } - - // (4) SEED — resolve from the existing captures. A tool authored as img2img/img2video that names a - // seed but resolves none → `.seedRequired`; a present-but-undecodable seed → `.seedInvalid`. Never - // a fabricated blank frame, never compute on a bad seed. - var seedBytes: Data? - if args.requiresSeed { - guard let bytes = seed.resolveSeed() else { - return failed(tool: tool, error: MediaError.seedRequired, - effectiveTier: effectiveTier, argsSummary: argsSummary) - } - guard MediaSeedValidation.isDecodablePNG(bytes) else { - return failed(tool: tool, error: MediaError.seedInvalid, - effectiveTier: effectiveTier, argsSummary: argsSummary) - } - seedBytes = bytes - } - - // (3b) Spend is now COMMITTED for cloud video — count it against the per-day cap (the cap counts - // admitted generations, like the handoff per-day count). - if isCloudVideo { budget.consume(now: now()) } - - // (5) RESIDENCY — make room; surface "busy painting" if the gen evicts chat. A target that can't - // be admitted at all maps to a clean `.failed` (never a hang). - if let residency, let id = modelID(for: kind) { - do { - let note = try await residency.ensureRoom(for: id) - observer.didDecideResidency(note) - } catch { - return failed(tool: tool, error: error, effectiveTier: effectiveTier, argsSummary: argsSummary) - } - } - - // (6–7) DRIVE the runtime, thread progress, write the finished asset. - let request = args.request(kind: kind, seed: seedBytes) - do { - var finished: MediaAsset? - for try await progress in runtime.generate(request) { - switch progress { - case let .step(index, total, preview): - observer.didStep(index: index, total: total, preview: preview) - case let .finished(asset): - finished = asset - } - } - guard let asset = finished else { - // The stream ended without finishing AND without throwing — treat as a cancellation - // (a discarded gen stops the stream cleanly). Distinct from a failure (design D10). - observer.didCancel() - recordAudit(tool: tool, tier: effectiveTier, - outcome: .declined(reason: Self.cancelledReason), argsSummary: argsSummary) - return ToolStepResult(tool: tool, status: .declined(reason: Self.cancelledReason), - summary: "Cancelled.") - } - observer.didFinish(asset) - recordAudit(tool: tool, tier: effectiveTier, outcome: .done, argsSummary: argsSummary) - return ToolStepResult(tool: tool, status: .done, - summary: "Generated \(Self.kindNoun(kind)) → \(asset.url.lastPathComponent).") - } catch { - // (8) A CANCELLATION is distinct from a failure (design D10): no failed badge, no failed audit. - if MediaOutcome.isCancellation(error) { - observer.didCancel() - recordAudit(tool: tool, tier: effectiveTier, - outcome: .declined(reason: Self.cancelledReason), argsSummary: argsSummary) - return ToolStepResult(tool: tool, status: .declined(reason: Self.cancelledReason), - summary: "Cancelled.") - } - return failed(tool: tool, error: error, effectiveTier: effectiveTier, argsSummary: argsSummary) - } - } - - // MARK: - Terminal helpers - - /// A sentinel decline reason the loop recognizes as "the user cancelled the whole canvas" (mirrors - /// `TaskKindToolContributor.cancelledReason`). - public static let cancelledReason = "__cancelled__" - - /// Build a clean `.failed` result: map the error → a clean headline (THE one translator), notify the - /// observer, write the failed audit (a `.failed` carries the clean headline only), and return. - private func failed(tool: String, error: Error, effectiveTier: WritePolicyTier, - argsSummary: String) -> ToolStepResult { - let headline = AIError.message(for: error).headline - observer.didFail(headline: headline) - recordAudit(tool: tool, tier: effectiveTier, outcome: .failed(headline: headline), - argsSummary: argsSummary) - return ToolStepResult(tool: tool, status: .failed(headline: headline), - summary: "Couldn't generate \(Self.kindNoun(kindNounFallback(tool))).") - } - - private func kindNounFallback(_ tool: String) -> MediaKind { - kind(for: tool) ?? .image - } - - /// Write exactly ONE audit record per terminal media step (auto/confirmed/declined/escalated/failed), - /// with the EFFECTIVE tier + a redacted args summary; a `.failed` carries only the clean headline - /// (already on the outcome). Non-blocking + infallible (the `AuditLog` contract). - private func recordAudit(tool: String, tier: WritePolicyTier, outcome: ToolStepStatus, - argsSummary: String) { - guard let audit else { return } - audit.record(AuditRecord(sessionID: sessionID, tool: tool, policy: tier, - argumentsSummary: argsSummary, outcome: outcome, - wasBackground: isBackground, timestamp: now())) - } - - // MARK: - Approval review - - /// The `TaskReview`-backed awaiting-approval surface for a `.confirm`/`.dangerous` media step — a - /// concrete preview (prompt + size + steps + the spend warning for cloud video) the canvas renders and - /// the compass resolves (DOWN=approve / RIGHT=skip). The `.unavailable` payload is unused; we model the - /// review as a no-op `.action` whose `fields` carry the preview (no side effect rides on it — the sink - /// runs the gen itself on approval). - static func approvalReview(tool: String, args: MediaArgs, kind: MediaKind, dangerous: Bool) -> TaskReview { - var fields: [ReviewField] = [ - ReviewField("Prompt", args.prompt.isEmpty ? "(none)" : args.prompt), - ReviewField("Size", "\(args.width ?? MediaSize.square1024.width)×\(args.height ?? MediaSize.square1024.height)") - ] - if let d = args.durationMs, kind == .video { fields.append(ReviewField("Duration", "\(d) ms")) } - if dangerous { fields.append(ReviewField("Note", "Cloud video — this spends from today's budget.")) } - let title = "Generate \(kindNoun(kind))" - // The payload is a benign placeholder — the sink, not the dispatcher, performs the generation on - // approval (media is a tool whose executor is the MediaRuntime, design D1/D2). - return .action(title: title, fields: fields, - payload: .openTool(tool: tool, - action: ParsedOpenTool(applicable: true, reason: nil, payload: args.prompt))) - } - - static func kindNoun(_ kind: MediaKind) -> String { - switch kind { - case .image: return "image" - case .video: return "video" - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/MediaJobState.swift b/Sources/ThreeFingerSwitcher/AI/Media/MediaJobState.swift deleted file mode 100644 index 7079830..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/MediaJobState.swift +++ /dev/null @@ -1,70 +0,0 @@ -import Foundation - -/// Output #2 (the pure part) — the canvas media STATE model (design D7, §6.1). The native player overlay -/// (FLAGGED) renders this; the model itself is pure + `swift test`-verified. It advances on -/// `MediaProgress` and terminates on finished / failed / cancelled. The canvas shows live step-progress + -/// the intermediate preview while generating; the finished image, or a player for video, on completion. -/// -/// The asset is ALREADY durable in the gallery (output #1) BEFORE the canvas resolves, so a discard never -/// loses the result — it only dismisses the preview. MLX-free Core. -public enum MediaJobState: Equatable, Sendable { - /// Not started / no job on the canvas. - case idle - /// Painting: the latest diffusion step + the optional intermediate preview frame (PNG). - case generating(index: Int, total: Int, preview: Data?) - /// Finished: the terminal asset (image or video) ready to extract / play. - case finished(MediaAsset) - /// A clean, bounded failure (the headline already routed through `AIError.message(for:)`). - case failed(headline: String) - /// The user discarded the in-flight gen. DISTINCT from `.failed` (design D10) — no failed indicator. - case cancelled - - /// True once the canvas has a terminal outcome (the preview can be resolved / dismissed). - public var isTerminal: Bool { - switch self { - case .finished, .failed, .cancelled: return true - case .idle, .generating: return false - } - } - - /// The finished asset if any (the extract target). - public var asset: MediaAsset? { - if case let .finished(asset) = self { return asset } - return nil - } - - /// Advance the state on one `MediaProgress`. A `.step` updates the live preview; `.finished` is - /// terminal. (Failure/cancellation enter via the thrown-error / discard paths, not a progress value.) - public mutating func advance(_ progress: MediaProgress) { - switch progress { - case let .step(index, total, preview): - self = .generating(index: index, total: total, preview: preview) - case let .finished(asset): - self = .finished(asset) - } - } - - /// Map a thrown generation error into a clean, bounded `.failed` — UNLESS it is a cancellation, which - /// is the distinct `.cancelled` outcome (never a failed badge, design D10). - public mutating func fail(with error: Error) { - if MediaOutcome.isCancellation(error) { - self = .cancelled - } else { - self = .failed(headline: AIError.message(for: error).headline) - } - } -} - -// MARK: - Cancellation classification - -public enum MediaOutcome { - /// True iff `error` represents a cancellation (a discarded gen), not a real failure. Used everywhere a - /// terminal outcome is decided so cancellation stays DISTINCT from failure (design D10): no `.failed` - /// badge, no failed audit, no false error card. - public static func isCancellation(_ error: Error) -> Bool { - if error is CancellationError { return true } - if let runtime = error as? RuntimeError, runtime == .cancelled { return true } - let ns = error as NSError - return ns.domain == NSCocoaErrorDomain && ns.code == NSUserCancelledError - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/MediaParkFeed.swift b/Sources/ThreeFingerSwitcher/AI/Media/MediaParkFeed.swift deleted file mode 100644 index ff7e798..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/MediaParkFeed.swift +++ /dev/null @@ -1,46 +0,0 @@ -import Foundation - -/// Parked-while-generating (design D8, §7) — FEEDS the existing `ParkScheduler` the media job's -/// observable state; it does NOT re-implement the scheduler/rail/glow. A generating parked session shows -/// the **thinking** badge (an intermediate `.step` advance); a `.finished` shows the **done** badge + the -/// unseen-result count (an idle-with-unseen advance); a `.dangerous` cloud-video step on a parked session -/// ESCALATES to needs-you (via `escalate`). A CANCELLATION leaves NO failed badge (it reports nothing — -/// the row just stops advancing). -/// -/// The scheduler decides scheduling; this seam only reports `didAdvance` / `escalate`. MLX-free Core. -public struct MediaParkFeed: Sendable { - private let scheduler: ParkScheduler - - public init(scheduler: ParkScheduler) { - self.scheduler = scheduler - } - - /// Report a painting step → the **thinking** badge (an intermediate advance). The `ToolStepResult` - /// is a `.done` beat (the scheduler's idle+badge bump shows progress). - public func reportPainting(_ id: AgentSessionID, tool: String) { - let step = ToolStepResult(tool: tool, status: .done, summary: "Painting…") - scheduler.didAdvance(id, result: step) - } - - /// Report a finished gen → the **done** badge + unseen count. The row idles with its unseen result; - /// it is NEVER auto-dismissed for finishing (`refactor-park-and-background-agents` retired the - /// terminal classification — removal is the user's, the opt-in expiry's, or eviction's alone). - public func reportFinished(_ id: AgentSessionID, tool: String, asset: MediaAsset) { - let step = ToolStepResult(tool: tool, status: .done, - summary: "Ready: \(asset.url.lastPathComponent)") - scheduler.didAdvance(id, result: step) - } - - /// Report a `.dangerous` cloud-video step on a parked session → ESCALATE to needs-you (ambient glow) - /// with a clean one-line reason, rather than spending in the background. - public func reportNeedsYou(_ id: AgentSessionID, reason: String) { - scheduler.escalate(id, reason: reason) - } - - /// Report a clean, bounded FAILURE → the row surfaces failed (the headline is already clean). NOT - /// called for a cancellation (cancellation reports nothing — no failed badge, design D10). - public func reportFailed(_ id: AgentSessionID, tool: String, headline: String) { - let step = ToolStepResult(tool: tool, status: .failed(headline: headline), summary: headline) - scheduler.didAdvance(id, result: step) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/MediaResidency.swift b/Sources/ThreeFingerSwitcher/AI/Media/MediaResidency.swift deleted file mode 100644 index b3db13f..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/MediaResidency.swift +++ /dev/null @@ -1,54 +0,0 @@ -import Foundation - -/// Honest residency (design D9, §3.4) — a heavy gen EVICTS chat ("busy painting"), surfaced not hidden. -/// The sink consults the `ModelRegistry` (`ensureResident` for the image/video runtime) BEFORE driving the -/// generation. When residency math (owned by `ai-model-fleet`, consumed here) decides the gen must evict -/// the chat model, the canvas/rail surface a calm "the assistant is busy painting" state rather than -/// pretending co-residency; chat resumes when the gen finishes / parks. -/// -/// The slice does NOT own the eviction math — it asks the registry to make room and OBSERVES whether chat -/// is still resident afterwards. MLX-free Core (orchestration over the injected registry). -public struct MediaResidencyCoordinator: Sendable { - private let registry: ModelRegistry - /// The `ModelRole` whose presence means "chat is resident". Eviction of it → busy-painting. - private let chatRole: ModelRole - - public init(registry: ModelRegistry, chatRole: ModelRole = .chat) { - self.registry = registry - self.chatRole = chatRole - } - - /// Make room for `modelID` (the image/video runtime). Returns the resulting residency note: - /// - `.coResident` — the gen fits alongside chat (e.g. Q4 image), chat stays available; - /// - `.busyPainting` — admitting the gen EVICTED chat; surface the calm busy state, chat resumes after. - /// Throws (mapped at the sink) when the target can't be admitted even after evicting everything - /// (`FleetError.cannotAdmit`) — a clean `.failed`, never a hang. - public func ensureRoom(for modelID: String) async throws -> MediaResidencyNote { - let chatResidentBefore = isChatResident() - try await registry.ensureResident(modelID) - let chatResidentAfter = isChatResident() - // If chat was resident and is now gone, the gen evicted it → busy painting. - if chatResidentBefore && !chatResidentAfter { return .busyPainting } - return .coResident - } - - private func isChatResident() -> Bool { - registry.resident().contains { $0.role == chatRole } - } -} - -/// The residency outcome the canvas/rail surface (design D9). -public enum MediaResidencyNote: Equatable, Sendable { - /// The gen co-resides with chat (chat stays available). - case coResident - /// The gen evicted chat — surface "the assistant is busy painting"; chat resumes on finish/park. - case busyPainting - - /// The calm, honest user-facing line for a busy-painting state (never a hang, never silence). - public var busyHeadline: String? { - switch self { - case .busyPainting: return "The assistant is busy painting. Chat resumes when it's done." - case .coResident: return nil - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/MediaRuntime.swift b/Sources/ThreeFingerSwitcher/AI/Media/MediaRuntime.swift deleted file mode 100644 index b872d4f..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/MediaRuntime.swift +++ /dev/null @@ -1,153 +0,0 @@ -import Foundation - -/// The generative-media seam + value types (addendum §B1, VERBATIM field shapes). A SECOND runtime seam, -/// parallel to `LLMRuntime` — NOT a token stream. A media generation is a LONG ASYNC JOB with step -/// progress that ends in a written file: `generate(_:)` returns an `AsyncThrowingStream` -/// that emits diffusion-step progress (with an optional intermediate preview frame) and terminates in a -/// `.finished(MediaAsset)` referencing the file on disk. -/// -/// This slice (`ai-media-runtime`) OWNS the seam + the value types + the tools + the sink + the output; -/// it does NOT own the concrete backends. The concrete diffusion runtimes conform to `MediaRuntime` in -/// their own slices (`ai-local-image-generation`, `ai-video-animation-generation`) exactly as Gemma -/// conforms to `LLMRuntime` — and could be swapped without touching feature code. MLX-free Core: the -/// protocol + value types verify under `swift build`, and `StubMediaRuntime` drives the whole slice under -/// `swift test` without any weights. - -// MARK: - Media kind - -/// The kind of media a runtime produces (addendum §B1). Drives capability gating (a runtime advertises -/// the set it can serve) and the per-tool descriptor (`generate_image` vs `generate_video`). -public enum MediaKind: String, Codable, Sendable, CaseIterable { - case image - case video -} - -// MARK: - Size - -/// A pixel size (width × height) for a generation. Its own value type (addendum §B1 carries -/// `MediaParameters.size` as a `MediaSize`) so the size is a first-class, `Codable`, comparable value. -public struct MediaSize: Codable, Equatable, Sendable { - public var width: Int - public var height: Int - - public init(width: Int, height: Int) { - self.width = width - self.height = height - } - - /// A reasonable square default for an image gen when the route omits a size. - public static let square1024 = MediaSize(width: 1024, height: 1024) -} - -// MARK: - Parameters - -/// Generation parameters (addendum §B1 verbatim shape). `durationMs` is VIDEO-ONLY — an image request -/// carries it `nil`; a video request carries a positive value (the sink/backend ignores it for an image -/// kind). `seedNumber` is the RNG seed for reproducibility, DISTINCT from the seed IMAGE on the request. -public struct MediaParameters: Codable, Equatable, Sendable { - /// width × height. - public var size: MediaSize - /// Diffusion steps. - public var steps: Int - /// RNG seed for reproducibility (nil → backend chooses). - public var seedNumber: UInt64? - /// Classifier-free guidance scale (nil → backend default). - public var guidance: Double? - /// Clip length in milliseconds — VIDEO ONLY (nil for images). - public var durationMs: Int? - - public init(size: MediaSize = .square1024, - steps: Int = 28, - seedNumber: UInt64? = nil, - guidance: Double? = nil, - durationMs: Int? = nil) { - self.size = size - self.steps = steps - self.seedNumber = seedNumber - self.guidance = guidance - self.durationMs = durationMs - } -} - -// MARK: - Request - -/// A media generation request (addendum §B1 verbatim shape). `seed` is the optional SEED IMAGE (PNG) -/// for img2img / img2video — the screen-region / clipboard capture promoted to the first frame (design -/// D5). It is `Data?`, not a number: a `nil` seed is a text-to-media request; a present seed is -/// img2img/img2video. (The numeric RNG seed lives on `parameters.seedNumber`.) -public struct MediaRequest: Sendable { - public var prompt: String - /// Optional SEED IMAGE (PNG) for img2img / img2video — the existing screen-region / clipboard capture. - public var seed: Data? - public var kind: MediaKind - public var parameters: MediaParameters - - public init(prompt: String, - seed: Data? = nil, - kind: MediaKind, - parameters: MediaParameters = MediaParameters()) { - self.prompt = prompt - self.seed = seed - self.kind = kind - self.parameters = parameters - } -} - -// MARK: - Progress - -/// Streamed progress of a generation (addendum §B1). `.step` carries the diffusion index/total plus an -/// OPTIONAL intermediate preview frame (PNG) the canvas paints live; `.finished` carries the terminal -/// asset. The stream THROWS on failure (mapped to `MediaError` at the boundary) and is cancelled on a -/// discard — cancellation is NOT a `.finished` and NOT a thrown failure (design D10). -public enum MediaProgress: Sendable { - /// Streamed diffusion progress + an optional intermediate preview frame (PNG bytes). - case step(index: Int, total: Int, preview: Data?) - /// The terminal asset (a written file). - case finished(MediaAsset) -} - -// MARK: - Asset - -/// A finished media asset (addendum §B1 verbatim shape). `url` is the written file — it becomes a -/// Files-band `.fileEntry` (the gallery, output #1) and the canvas preview/player source (output #2). -/// `durationMs` is video-only (nil for images). `Codable` so it can round-trip / be logged; `Identifiable` -/// so the canvas/rail can key on it. -public struct MediaAsset: Codable, Equatable, Identifiable, Sendable { - public let id: UUID - /// The written file (becomes a Files-band entry). - public var url: URL - public var kind: MediaKind - public var width: Int - public var height: Int - /// Clip length in ms — VIDEO ONLY (nil for images). - public var durationMs: Int? - - public init(id: UUID = UUID(), - url: URL, - kind: MediaKind, - width: Int, - height: Int, - durationMs: Int? = nil) { - self.id = id - self.url = url - self.kind = kind - self.width = width - self.height = height - self.durationMs = durationMs - } -} - -// MARK: - The seam - -/// The media runtime seam (addendum §B1 verbatim) — parallel to `LLMRuntime`, no extra methods. A -/// conformer advertises the kinds it can produce (`capabilities`) and runs a generation as an -/// `AsyncThrowingStream`. The concrete backends live in their own native-linked -/// slices; this Core protocol is exercised in tests via `StubMediaRuntime`. -public protocol MediaRuntime: Sendable { - /// The kinds this runtime can produce. The contributor advertises `generate_image`/`generate_video` - /// only if a runtime advertises the matching kind (design D11) — the router never routes to a dead end. - var capabilities: Set { get } - /// Run a generation. The stream emits `.step`/`.finished` and THROWS on failure; the consumer cancels - /// it on a discard (cancellation is observed as a stopped stream, not a thrown failure). - func generate(_ request: MediaRequest) -> AsyncThrowingStream -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/MediaSeed.swift b/Sources/ThreeFingerSwitcher/AI/Media/MediaSeed.swift deleted file mode 100644 index 44a9e05..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/MediaSeed.swift +++ /dev/null @@ -1,64 +0,0 @@ -import Foundation - -/// The seed (img2img / img2video) path (design D5). A `MediaRequest.seed: Data?` (PNG) is the first -/// frame, sourced from the EXISTING capture inputs — the interactive screen-region picker -/// (`.screenRegion`) and the on-demand live clipboard image (`.clipboardImage`, normalized to PNG) — the -/// same inputs the vision path already uses. NO new media-only picker; copying an image NEVER auto-fires a -/// gen (the clipboard read stays on-demand — the sink pulls it, nothing pushes). -/// -/// This seam is pure orchestration over a captured/clipboard image; the concrete capture lives behind the -/// existing `SelectionProviding.readClipboardImage()` / the region picker (`ScreenCaptureOutcome`), which -/// the app wires in. MLX-free Core. -public protocol MediaSeedResolving: Sendable { - /// The seed image (PNG) to use as the first frame, or `nil` when none is available. Pulled ON DEMAND - /// (never pushed) — a clipboard image present here does NOT auto-fire a gen; the sink decides whether - /// the routed tool actually requires it. - func resolveSeed() -> Data? -} - -/// A seed provider backed by a captured image (the screen-region `ScreenCaptureOutcome` or a clipboard -/// PNG already in hand). The app constructs it from whichever capture the user supplied. A pure value: -/// it just hands back the bytes it was given (the capture happened upstream, behind the existing seams). -public struct CapturedSeed: MediaSeedResolving { - private let bytes: Data? - - /// Seed from raw PNG bytes (a clipboard image or a region capture already encoded to PNG). - public init(png bytes: Data?) { - self.bytes = bytes - } - - /// Seed from a `ScreenCaptureOutcome`: a `.captured` carries bytes; `.permissionDenied`/`.unavailable` - /// resolve to no seed (the sink turns a missing-but-required seed into `MediaError.seedRequired`). - /// (Internal: `ScreenCaptureOutcome` is an internal capture type.) - init(screenCapture: ScreenCaptureOutcome) { - switch screenCapture { - case let .captured(data): self.bytes = data - case .permissionDenied, .unavailable: self.bytes = nil - } - } - - public func resolveSeed() -> Data? { bytes } -} - -/// The no-seed default (a text-to-media run with no capture supplied). A tool authored as img2img/img2video -/// with this resolver yields `MediaError.seedRequired` at the sink — never a fabricated blank frame. -public struct NoSeed: MediaSeedResolving { - public init() {} - public func resolveSeed() -> Data? { nil } -} - -// MARK: - PNG validation (boundary) - -public enum MediaSeedValidation { - /// The PNG 8-byte magic. A seed that is present but not a decodable PNG is `MediaError.seedInvalid` - /// (design D5) — never silently passed to the backend. We validate the signature here (Core, no - /// ImageIO) rather than a deep decode: the capture seams already normalize to PNG, so the magic is the - /// honest, dependency-free "is this a real PNG" gate at this layer. - static let pngMagic: [UInt8] = [0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A] - - /// True iff `data` begins with the PNG signature (a present, decodable-shaped seed). - public static func isDecodablePNG(_ data: Data) -> Bool { - guard data.count >= pngMagic.count else { return false } - return Array(data.prefix(pngMagic.count)) == pngMagic - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/MediaToolContributor.swift b/Sources/ThreeFingerSwitcher/AI/Media/MediaToolContributor.swift deleted file mode 100644 index 398adaa..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/MediaToolContributor.swift +++ /dev/null @@ -1,173 +0,0 @@ -import Foundation - -/// The two media tools in the `ToolRegistry` (design D2/D11). `generate_image` and `generate_video` are -/// ordinary `ToolDescriptor`s contributed through the EXISTING `ToolContributor` seam — no route-loop -/// change. When the router selects one, the loop dispatches to `MediaGenSink` (the contributor's `run`) -/// exactly as it dispatches a task kind. The model "chose the menu item"; the sink does the long job. -/// -/// Availability gating (design D11 — the router never routes to a dead end): the contributor advertises -/// - `generate_image` ONLY if `mediaGenEnabled && fullPotentialEnabled` AND an image `MediaRuntime` -/// advertises `.image`; -/// - `generate_video` ONLY if the master/media flags are on AND a video provider exists AND (for cloud) -/// `fleetCloudEscalationEnabled` is on AND the per-day budget has room left. -/// Under either flag OFF, the contributor contributes NOTHING. -/// -/// MLX-free Core. - -// MARK: - Availability - -/// The gating inputs the contributor reads to decide which media tools are live. The flags are OWNED by -/// `ai-full-potential-toggle` (§D1) / `ai-model-fleet`; this slice CONSUMES them via injected closures -/// (default OFF), exactly like `FleetCloudGate`. `now` feeds the budget check. -public struct MediaToolAvailability: Sendable { - /// Master gate (`fullPotentialEnabled`) — a sub-flag never overrides the master OFF. - public let isFullPotentialEnabled: @Sendable () -> Bool - /// The media sub-flag (`mediaGenEnabled`). - public let isMediaGenEnabled: @Sendable () -> Bool - /// Cloud escalation (`fleetCloudEscalationEnabled`) — required for a CLOUD video provider. - public let isCloudEscalationEnabled: @Sendable () -> Bool - /// Whether a video provider is configured at all (cloud or local LTXV). A `nil`/false → no video tool. - public let hasVideoProvider: @Sendable () -> Bool - /// Whether the configured video provider is cloud (→ `.dangerous` + budget + cloud gate) vs local. - public let videoProviderIsCloud: @Sendable () -> Bool - - public init(isFullPotentialEnabled: @escaping @Sendable () -> Bool = { false }, - isMediaGenEnabled: @escaping @Sendable () -> Bool = { false }, - isCloudEscalationEnabled: @escaping @Sendable () -> Bool = { false }, - hasVideoProvider: @escaping @Sendable () -> Bool = { false }, - videoProviderIsCloud: @escaping @Sendable () -> Bool = { true }) { - self.isFullPotentialEnabled = isFullPotentialEnabled - self.isMediaGenEnabled = isMediaGenEnabled - self.isCloudEscalationEnabled = isCloudEscalationEnabled - self.hasVideoProvider = hasVideoProvider - self.videoProviderIsCloud = videoProviderIsCloud - } - - /// The master ∧ media predicate — the floor for offering ANY media tool. - var mediaActive: Bool { isFullPotentialEnabled() && isMediaGenEnabled() } -} - -// MARK: - Tool names (stable routing keys) - -public enum MediaTool { - public static let generateImage = "generate_image" - public static let generateVideo = "generate_video" -} - -// MARK: - Contributor - -/// Contributes the two media tools and runs a routed media call via the injected `MediaGenSink`. -struct MediaToolContributor: ToolContributor { - private let availability: MediaToolAvailability - /// The image runtime (its `capabilities` must advertise `.image` for `generate_image` to appear). - private let imageRuntime: MediaRuntime? - /// The video runtime (its `capabilities` must advertise `.video` for `generate_video` to appear). - private let videoRuntime: MediaRuntime? - private let budget: MediaVideoBudgeting - private let sink: MediaGenSink - /// Injected clock for the budget check (deterministic in tests). - private let now: @Sendable () -> Date - - init(availability: MediaToolAvailability, - imageRuntime: MediaRuntime?, - videoRuntime: MediaRuntime?, - budget: MediaVideoBudgeting, - sink: MediaGenSink, - now: @escaping @Sendable () -> Date = { Date() }) { - self.availability = availability - self.imageRuntime = imageRuntime - self.videoRuntime = videoRuntime - self.budget = budget - self.sink = sink - self.now = now - } - - // MARK: ToolContributor - - func descriptors() -> [ToolDescriptor] { - guard availability.mediaActive else { return [] } // master ∧ media floor (D11) - var out: [ToolDescriptor] = [] - if imageAvailable { out.append(Self.imageDescriptor) } - if videoAvailable { out.append(videoDescriptor) } - return out - } - - func canHandle(_ tool: String) -> Bool { - tool == MediaTool.generateImage || tool == MediaTool.generateVideo - } - - func run(_ call: RoutedCall, gate: ApprovalGate) async -> ToolStepResult { - await sink.run(call, gate: gate) - } - - // MARK: Availability (D11) - - /// `generate_image` is live iff master ∧ media AND an image runtime advertises `.image`. - var imageAvailable: Bool { - guard availability.mediaActive else { return false } - return imageRuntime?.capabilities.contains(.image) ?? false - } - - /// `generate_video` is live iff master ∧ media AND a video runtime advertises `.video` AND a video - /// provider is configured AND (for cloud) cloud escalation is on AND the per-day budget has room. - var videoAvailable: Bool { - guard availability.mediaActive else { return false } - guard videoRuntime?.capabilities.contains(.video) ?? false else { return false } - guard availability.hasVideoProvider() else { return false } - if availability.videoProviderIsCloud() { - guard availability.isCloudEscalationEnabled() else { return false } - guard budget.hasRemaining(now: now()) else { return false } - } - return true - } - - // MARK: Descriptors - - /// `generate_image`: a `.confirm`-tier tool (design D3) — local but GPU-saturating + evicts chat, so - /// it confirms before painting. - static let imageDescriptor = ToolDescriptor( - name: MediaTool.generateImage, - summary: "Generate an image from a text prompt (optionally from a source image).", - argsSchema: StructuredSchema(name: MediaTool.generateImage, json: imageArgsSchemaJSON), - writePolicy: .confirm, - keywords: ["image", "picture", "draw", "render", "generate", "art", "illustration", "photo"] - ) - - /// `generate_video`: `.dangerous` when the provider is cloud (real spend, leaves the device — escalates - /// even when parked), `.confirm` for a local provider. Budget-capped at the sink regardless. - var videoDescriptor: ToolDescriptor { - let tier: WritePolicyTier = availability.videoProviderIsCloud() ? .dangerous : .confirm - return ToolDescriptor( - name: MediaTool.generateVideo, - summary: "Generate a short video clip from a prompt (optionally animating a source image).", - argsSchema: StructuredSchema(name: MediaTool.generateVideo, json: Self.videoArgsSchemaJSON), - writePolicy: tier, - keywords: ["video", "animate", "animation", "clip", "motion", "film", "generate"] - ) - } - - // MARK: argsSchema - - /// Prompt + size (width/height) + steps + an optional seed-image handle. The handle is a STRING tag - /// (`"screenRegion"`/`"clipboardImage"`) the sink resolves through the existing capture seams — the - /// JSON never carries raw image bytes. - static let imageArgsSchemaJSON = """ - {"type":"object","required":["prompt"],"properties":{\ - "prompt":{"type":"string"},\ - "width":{"type":"integer","minimum":64,"maximum":2048},\ - "height":{"type":"integer","minimum":64,"maximum":2048},\ - "steps":{"type":"integer","minimum":1,"maximum":100},\ - "seedImage":{"type":"string","enum":["screenRegion","clipboardImage"]}}} - """ - - /// As image, plus a video `durationMs`. - static let videoArgsSchemaJSON = """ - {"type":"object","required":["prompt"],"properties":{\ - "prompt":{"type":"string"},\ - "width":{"type":"integer","minimum":64,"maximum":2048},\ - "height":{"type":"integer","minimum":64,"maximum":2048},\ - "steps":{"type":"integer","minimum":1,"maximum":100},\ - "durationMs":{"type":"integer","minimum":200,"maximum":20000},\ - "seedImage":{"type":"string","enum":["screenRegion","clipboardImage"]}}} - """ -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/MediaVideoBudget.swift b/Sources/ThreeFingerSwitcher/AI/Media/MediaVideoBudget.swift deleted file mode 100644 index d4b5ef6..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/MediaVideoBudget.swift +++ /dev/null @@ -1,57 +0,0 @@ -import Foundation - -/// The cloud-video per-day BUDGET cap (design D3) — a media-side mirror of the Claude-handoff cost gate -/// (`mediaVideoBudgetPerDay`, mirroring `ClaudeHandoffConfig`). Cloud video spends real money and leaves -/// the device, so it is `.dangerous` AND rate-capped: the sink enforces the cap BEFORE any network call, -/// so an exhausted budget is a clean `MediaError.cloudBudgetExhausted` with NO spend (never a silent -/// spend, never a silent refusal). -/// -/// The flag `mediaVideoBudgetPerDay` is OWNED by `ai-full-potential-toggle` (§D1) — this slice CONSUMES it -/// via an injected closure (default 0 → no cloud video until configured). The per-day usage is tracked -/// against an injected `now:` so the rollover is deterministically testable. MLX-free Core. -public protocol MediaVideoBudgeting: Sendable { - /// True iff at least one more cloud-video generation may run today (usage < the per-day cap). - func hasRemaining(now: Date) -> Bool - /// Record one cloud-video generation as spent (called AFTER the budget check passes and the gen is - /// approved — the cap counts admitted generations, like the handoff per-day count). - func consume(now: Date) -} - -/// A simple per-calendar-day counter against `mediaVideoBudgetPerDay`. The cap closure is read live (so a -/// settings change takes effect immediately). Thread-safe; the day boundary is `Calendar.current`'s start -/// of day for `now`. A cap of 0 means cloud video is effectively off (no remaining, ever). -public final class PerDayVideoBudget: MediaVideoBudgeting, @unchecked Sendable { - /// Reads the persisted `mediaVideoBudgetPerDay` (owned by `ai-full-potential-toggle`). Injected so - /// this slice never defines the flag; default 0 (no cloud video until configured). - private let cap: @Sendable () -> Int - private let calendar: Calendar - private let lock = NSLock() - private var dayStart: Date? - private var usedToday = 0 - - public init(cap: @escaping @Sendable () -> Int = { 0 }, calendar: Calendar = .current) { - self.cap = cap - self.calendar = calendar - } - - public func hasRemaining(now: Date) -> Bool { - lock.lock(); defer { lock.unlock() } - rolloverIfNeeded(now) - return usedToday < max(0, cap()) - } - - public func consume(now: Date) { - lock.lock(); defer { lock.unlock() } - rolloverIfNeeded(now) - usedToday += 1 - } - - /// Reset the counter when `now` crosses into a new calendar day (the per-day rollover). - private func rolloverIfNeeded(_ now: Date) { - let start = calendar.startOfDay(for: now) - if dayStart != start { - dayStart = start - usedToday = 0 - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/StubMediaRuntime.swift b/Sources/ThreeFingerSwitcher/AI/Media/StubMediaRuntime.swift deleted file mode 100644 index 7e54e9b..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/StubMediaRuntime.swift +++ /dev/null @@ -1,77 +0,0 @@ -import Foundation - -/// A scripted `MediaRuntime` for `swift test` (design: the stub makes the whole slice verifiable without -/// weights). It plays a fixed `Script` as a deterministic `AsyncThrowingStream` — the same -/// shape the real diffusion backends will emit — so the contributor, sink, seed path, gallery writer, -/// parked feed, and `MediaError` mapping are all exercised end-to-end with no MLX and no GPU. -/// -/// Scripts cover every terminal shape the sink must handle: -/// - `.success(asset)` — straight to `.finished` (no intermediate previews). -/// - `.successWithPreviews(...)` — a run of `.step` (with preview frames) then `.finished`. -/// - `.failMidFlight(steps:headline:)` — some `.step`s then a thrown `MediaError.generationFailed`. -/// - `.cancelImmediately` — yields nothing and never finishes (the consumer cancels it; the -/// stream just stops — cancellation is observed as a stopped stream, -/// never a thrown failure, per design D10). -/// -/// MLX-free Core (test support — `public` so the test target can build scripts). -public final class StubMediaRuntime: MediaRuntime, @unchecked Sendable { - - /// What the next `generate(_:)` call plays. - public enum Script: Sendable { - /// Emit `.finished(asset)` immediately. - case success(MediaAsset) - /// Emit `count` `.step`s (each carrying `preview`) then `.finished(asset)`. - case successWithPreviews(count: Int, total: Int, preview: Data?, asset: MediaAsset) - /// Emit `steps` `.step`s then THROW `MediaError.generationFailed(headline:)`. - case failMidFlight(steps: Int, headline: String) - /// Yield nothing and hold open until cancelled — models a discarded gen. - case cancelImmediately - } - - public let capabilities: Set - private let script: Script - /// Records each request the sink drove (so tests assert the seed / kind / parameters wired through). - public private(set) var receivedRequests: [MediaRequest] = [] - private let lock = NSLock() - - public init(capabilities: Set = [.image], script: Script) { - self.capabilities = capabilities - self.script = script - } - - public func generate(_ request: MediaRequest) -> AsyncThrowingStream { - lock.lock(); receivedRequests.append(request); lock.unlock() - let script = self.script - return AsyncThrowingStream { continuation in - let task = Task { - switch script { - case let .success(asset): - continuation.yield(.finished(asset)) - continuation.finish() - case let .successWithPreviews(count, total, preview, asset): - for i in 0.. = [.video] - /// Which backend this stub models (only affects the disclosure flag for a seed-present run). - public let provider: VideoProvider - private let script: Script - /// Records each request the sink drove (so tests assert the seed / kind / duration wired through). - public private(set) var receivedRequests: [MediaRequest] = [] - private let lock = NSLock() - - public init(provider: VideoProvider, script: Script) { - self.provider = provider - self.script = script - } - - /// The upload disclosure a routed request WOULD build for this provider — the stub exposes it so a test - /// pins that a seed-present cloud run flags `bytesLeaveDevice && seedPresent` and a local run flags - /// neither (task 5.2 / 7.x). Pure; no side effect. - public func disclosure(for request: MediaRequest) -> VideoUploadDisclosure { - VideoUploadDisclosure.make(provider: provider, - prompt: request.prompt, - seedPresent: request.seed != nil) - } - - public func generate(_ request: MediaRequest) -> AsyncThrowingStream { - lock.lock(); receivedRequests.append(request); lock.unlock() - let script = self.script - return AsyncThrowingStream { continuation in - let task = Task { - switch script { - case let .success(asset): - continuation.yield(.finished(asset)) - continuation.finish() - case let .successWithPreviews(count, total, preview, asset): - for i in 0.. ScriptedVideoRuntime { - ScriptedVideoRuntime(provider: .cloud, script: script) - } -} - -/// The scripted LOCAL-LTXV frontier video backend (nothing uploaded). Same seam, same shapes — the swap -/// contract (task 9.1) means feature code can't tell them apart except via the disclosure flag. -public enum StubLocalVideoRuntime { - public static func make(script: ScriptedVideoRuntime.Script) -> ScriptedVideoRuntime { - ScriptedVideoRuntime(provider: .localLTXV, script: script) - } -} - -// MARK: - A finished-video asset helper - -public extension MediaAsset { - /// Build a finished VIDEO asset (kind `.video`, `durationMs` set) — the terminal shape both video - /// backends emit. Convenience for scripts + tests so the `durationMs != nil` invariant is upheld. - static func video(url: URL, width: Int, height: Int, durationMs: Int) -> MediaAsset { - MediaAsset(url: url, kind: .video, width: width, height: height, durationMs: durationMs) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/Video/VideoBudget.swift b/Sources/ThreeFingerSwitcher/AI/Media/Video/VideoBudget.swift deleted file mode 100644 index 92617f4..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/Video/VideoBudget.swift +++ /dev/null @@ -1,232 +0,0 @@ -import Foundation - -/// One spend record in the append-only CLOUD-video ledger (`ai-video-animation-generation`, design D3 — -/// the §3.8 `HandoffSpend` pattern reused, not reinvented). An admitted cloud video appends one; a launch -/// that fails after recording REMOVES it (refund), so the cap stays honest. `Codable` so the ledger -/// persists across a relaunch WITHIN the rolling window — a process restart cannot reset the daily budget. -/// MLX-free Core. -public struct VideoSpend: Codable, Equatable, Sendable { - public let at: Date - - public init(at: Date) { self.at = at } -} - -/// The pure ROLLING-24h rate/concurrency gate over an append-only spend ledger -/// (`ai-video-animation-generation`, design D3 — structurally the `HandoffBudget` pattern). `allows(now:)` -/// is a PURE predicate — `now` is an INPUT (deterministic) — so an autonomous loop physically cannot rack -/// up real spend and the window is unit-testable without a clock. The window is a SLIDING 24h count over -/// `now`, NOT a calendar-day reset, so a loop cannot dump N clips at 23:59 and N more at 00:01. -/// -/// This is DISTINCT from the seam-owner's `PerDayVideoBudget` (a calendar-day counter): this slice's gate -/// is the rolling window the spec requires ("Rolling window cannot be gamed across midnight"). MLX-free Core. -public struct VideoBudget: Equatable, Sendable { - /// The per-rolling-window call cap (`mediaVideoBudgetPerDay`). A `<= 0` cap never allows (cloud video - /// effectively off). - public let maxCallsPerWindow: Int - /// At-most-N concurrent in-flight cloud videos (default 1 — one slow clip per loop step). - public let maxConcurrent: Int - /// The append-only spend records; in-flight is tracked separately. - public private(set) var ledger: [VideoSpend] - /// The count of admitted-but-not-yet-reaped generations. - public private(set) var inFlight: Int - - /// The rolling window length: 24 hours. - public static let window: TimeInterval = 24 * 60 * 60 - - public init(maxCallsPerWindow: Int, maxConcurrent: Int = 1, - ledger: [VideoSpend] = [], inFlight: Int = 0) { - self.maxCallsPerWindow = maxCallsPerWindow - self.maxConcurrent = max(1, maxConcurrent) - self.ledger = ledger - self.inFlight = inFlight - } - - /// The number of spends inside the rolling 24h window ending at `now`. - public func callsInLast24h(_ now: Date) -> Int { - let cutoff = now.addingTimeInterval(-Self.window) - return ledger.reduce(0) { $0 + ($1.at > cutoff ? 1 : 0) } - } - - /// PURE predicate: under the rolling-window cap AND below the concurrency limit. `now` is an INPUT. - public func allows(now: Date) -> Bool { - guard maxCallsPerWindow > 0 else { return false } - return callsInLast24h(now) < maxCallsPerWindow && inFlight < maxConcurrent - } - - /// Spend a call: append the record + increment in-flight. (Called only AFTER `allows` passed AND the - /// gen was admitted — the cap counts admitted generations, like the handoff per-day count.) - public mutating func record(at: Date) { - ledger.append(VideoSpend(at: at)) - inFlight += 1 - } - - /// A fire-and-forget gen is "done" → decrement in-flight (the ledger entry STAYS; it counts against the - /// window cap). Only the concurrency count drops. - public mutating func reap() { - inFlight = max(0, inFlight - 1) - } - - /// A launch that FAILED after recording didn't truly spend → remove the matching ledger entry (by `at`) - /// and drop in-flight, leaving the cap unchanged from before the call (spec "A failed cloud launch - /// refunds its spend"). - public mutating func refund(at: Date) { - if let idx = ledger.lastIndex(where: { $0.at == at }) { - ledger.remove(at: idx) - } - inFlight = max(0, inFlight - 1) - } -} - -// MARK: - Persistence seam (relaunch-surviving ledger) - -/// The injectable persistence seam for the cloud-video ledger (`ai-video-animation-generation`, task 2.3 — -/// the §3.8 `HandoffLedgerStore` pattern). The ledger persists under Application Support so the rolling cap -/// survives a relaunch within the window. Tests inject an in-memory store. MLX-free Core. -public protocol VideoLedgerStore: Sendable { - func load() -> [VideoSpend] - func save(_ ledger: [VideoSpend]) -} - -/// The default no-disk store (tests / a fresh process). Records held in memory only. -public final class InMemoryVideoLedgerStore: VideoLedgerStore, @unchecked Sendable { - private let lock = NSLock() - private var ledger: [VideoSpend] - - public init(_ ledger: [VideoSpend] = []) { self.ledger = ledger } - - public func load() -> [VideoSpend] { - lock.lock(); defer { lock.unlock() } - return ledger - } - - public func save(_ ledger: [VideoSpend]) { - lock.lock(); defer { lock.unlock() } - self.ledger = ledger - } -} - -/// The durable JSON ledger store under Application Support (mirroring `DiskHandoffLedgerStore`). A read -/// failure yields an empty ledger (a fresh budget); a write failure is swallowed (the in-memory budget -/// still holds the spend) — cap honesty is best-effort persistence, never a thrown break into the loop. -/// MLX-free Core; its real IO is exercised by the user's build. -public final class DiskVideoLedgerStore: VideoLedgerStore, @unchecked Sendable { - private let fileURL: URL - private let encoder: JSONEncoder = { - let e = JSONEncoder(); e.outputFormatting = [.sortedKeys]; e.dateEncodingStrategy = .iso8601; return e - }() - private let decoder: JSONDecoder = { - let d = JSONDecoder(); d.dateDecodingStrategy = .iso8601; return d - }() - - public init(fileURL: URL = DiskVideoLedgerStore.defaultFileURL()) { - self.fileURL = fileURL - } - - /// `~/Library/Application Support/ThreeFingerSwitcher/media/video-ledger.json`. - public static func defaultFileURL() -> URL { - let base = (try? FileManager.default.url(for: .applicationSupportDirectory, in: .userDomainMask, - appropriateFor: nil, create: true)) - ?? URL(fileURLWithPath: NSTemporaryDirectory()) - return base - .appendingPathComponent("ThreeFingerSwitcher/media", isDirectory: true) - .appendingPathComponent("video-ledger.json") - } - - public func load() -> [VideoSpend] { - guard let data = try? Data(contentsOf: fileURL), - let records = try? decoder.decode([VideoSpend].self, from: data) else { return [] } - return records - } - - public func save(_ ledger: [VideoSpend]) { - do { - try FileManager.default.createDirectory(at: fileURL.deletingLastPathComponent(), - withIntermediateDirectories: true) - let data = try encoder.encode(ledger) - try data.write(to: fileURL, options: .atomic) - } catch { - // Best-effort: the in-memory budget still holds the spend; a persistence failure does not break - // the loop (never thrown into the caller). - } - } -} - -// MARK: - The mutable, thread-safe holder (the MediaVideoBudgeting conformer) - -/// The mutable, thread-safe box the sink records spend against (`ai-video-animation-generation`, the §3.8 -/// `HandoffBudgetBox` pattern). `VideoBudget` is a PURE value; this reference wrapper owns the single -/// mutable instance + its persistence so the `Sendable` sink can `consume`/`reap`/`refund` across an -/// `await` without data races. It CONFORMS to the seam-owner's `MediaVideoBudgeting` so it drops into the -/// EXISTING `MediaGenSink` / `MediaToolContributor` with NO seam change — the rolling-window budget is the -/// `videoProvider == .cloud` selection of the same protocol slot (the swap contract for the gate). -/// -/// `hasRemaining(now:)` / `consume(now:)` satisfy `MediaVideoBudgeting` (the sink calls these); `reap` / -/// `refund` are the richer concurrency/refund operations the cloud runtime + contributor drive. MLX-free -/// Core. -public final class RollingVideoBudget: MediaVideoBudgeting, @unchecked Sendable { - private let lock = NSLock() - private var budget: VideoBudget - private let store: VideoLedgerStore - /// The cap is read LIVE (a settings change takes effect immediately) — the persisted - /// `mediaVideoBudgetPerDay`, owned by `ai-full-potential-toggle`, CONSUMED via a closure (default 0). - private let cap: @Sendable () -> Int - private let maxConcurrent: Int - - public init(cap: @escaping @Sendable () -> Int = { 0 }, - maxConcurrent: Int = 1, - store: VideoLedgerStore = InMemoryVideoLedgerStore()) { - self.cap = cap - self.maxConcurrent = max(1, maxConcurrent) - self.store = store - // Seed the ledger from disk so the rolling-window cap survives a relaunch (task 2.3). - self.budget = VideoBudget(maxCallsPerWindow: cap(), maxConcurrent: self.maxConcurrent, - ledger: store.load()) - } - - /// `MediaVideoBudgeting`: at least one more cloud video may run (under the rolling cap + concurrency). - /// Rebuilds the pure budget with the LIVE cap so a settings bump takes effect at once. - public func hasRemaining(now: Date) -> Bool { - lock.lock(); defer { lock.unlock() } - syncCap() - return budget.allows(now: now) - } - - /// `MediaVideoBudgeting`: record one admitted cloud video as spent (after the budget check passed and - /// the gen was approved). Increments in-flight + persists the ledger. - public func consume(now: Date) { - lock.lock(); defer { lock.unlock() } - syncCap() - budget.record(at: now) - store.save(budget.ledger) - } - - /// A fire-and-forget gen finished → decrement in-flight (the ledger entry stays). - public func reap() { - lock.lock(); defer { lock.unlock() } - budget.reap() - } - - /// A launch that failed after `consume` → refund: remove the entry recorded at `at` + drop in-flight, - /// leaving the cap unchanged. Persists the corrected ledger. - public func refund(at: Date) { - lock.lock(); defer { lock.unlock() } - budget.refund(at: at) - store.save(budget.ledger) - } - - /// A snapshot for assertions / tests. - public func snapshot() -> VideoBudget { - lock.lock(); defer { lock.unlock() } - return budget - } - - /// Rebuild the pure budget with the LIVE cap (preserving the ledger + in-flight), so a settings change - /// to `mediaVideoBudgetPerDay` takes effect immediately without losing recorded spend. - private func syncCap() { - let live = cap() - if live != budget.maxCallsPerWindow { - budget = VideoBudget(maxCallsPerWindow: live, maxConcurrent: maxConcurrent, - ledger: budget.ledger, inFlight: budget.inFlight) - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/Video/VideoProvider.swift b/Sources/ThreeFingerSwitcher/AI/Media/Video/VideoProvider.swift deleted file mode 100644 index 0fa8709..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/Video/VideoProvider.swift +++ /dev/null @@ -1,55 +0,0 @@ -import Foundation - -/// The video-backend SELECTOR (`ai-video-animation-generation`, design D1/D2) backing the persisted -/// `videoProvider` key (addendum §1). One seam (`MediaRuntime`), two interchangeable backends — this value -/// names WHICH conformer is wired, exactly as `aiSelectedModelID` names the chat model. The default is -/// `.cloud` (a hosted-API escalation), the honest calm default: no 35 GB download, no chat eviction, no -/// fans (design D2). -/// -/// `.localLTXV` is the FRONTIER option — a ComfyUI/MPS 35 GB+ graph (NOT in-process MLX) — selectable ONLY -/// when the master `fullPotentialEnabled` AND `mediaGenEnabled` are both ON. This type carries the pure -/// validity rule (`isSelectable(...)`) so the gating is `swift test`-verified without the toggle page (the -/// flags are CONSUMED via injected booleans, owned by `ai-full-potential-toggle`). MLX-free Core. -public enum VideoProvider: String, Codable, Sendable, CaseIterable, Identifiable { - /// The default: a hosted video API (LTX Studio / equivalent). Spends money + uploads bytes → - /// `.dangerous` + budget-capped at the sink (design D3). Nothing is downloaded or made resident. - case cloud - /// The frontier: a local LTXV ComfyUI/MPS graph (35 GB+, minutes-per-clip, EVICTS chat). Behind the - /// master toggle; spends no money so it is OFF the cloud budget path (design D6). - case localLTXV - - public var id: String { rawValue } - - /// The honest default — the calm path (design D2). A user who never touched the setting gets cloud. - public static let defaultProvider: VideoProvider = .cloud - - /// True iff this is the cloud escalation (→ `.dangerous` tier + budget + upload disclosure). The local - /// provider is `false` (no spend, no upload). - public var isCloud: Bool { self == .cloud } - - /// Whether THIS provider may be SELECTED given the master gates. `.cloud` is always a valid selection - /// (it is the default, gated separately by `fleetCloudEscalationEnabled` at the contributor). `.localLTXV` - /// requires BOTH `fullPotentialEnabled` AND `mediaGenEnabled` — the frontier is never reachable with the - /// master off (design D2 / spec "Local LTXV requires the master toggle"). A sub-flag NEVER overrides the - /// master OFF. - public func isSelectable(fullPotentialEnabled: Bool, mediaGenEnabled: Bool) -> Bool { - switch self { - case .cloud: - return true - case .localLTXV: - return fullPotentialEnabled && mediaGenEnabled - } - } - - /// Resolve a STORED raw value into an effective provider under the current master gates: an invalid - /// `.localLTXV` (master off) falls back to the calm `.cloud` default rather than offering a frontier the - /// gates forbid (defends a settings desync — the toggle flipped off while `.localLTXV` was persisted). - public static func resolved(rawValue: String?, - fullPotentialEnabled: Bool, - mediaGenEnabled: Bool) -> VideoProvider { - let stored = rawValue.flatMap(VideoProvider.init(rawValue:)) ?? .defaultProvider - return stored.isSelectable(fullPotentialEnabled: fullPotentialEnabled, mediaGenEnabled: mediaGenEnabled) - ? stored - : .defaultProvider - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/Video/VideoTierResolution.swift b/Sources/ThreeFingerSwitcher/AI/Media/Video/VideoTierResolution.swift deleted file mode 100644 index 0ca2ba0..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/Video/VideoTierResolution.swift +++ /dev/null @@ -1,102 +0,0 @@ -import Foundation - -/// The effective-tier + over-budget DEGRADE resolution for `generate_video` -/// (`ai-video-animation-generation`, design D3 / tasks 3.1, 3.2 — the §3.8 `HandoffGate` pattern). Pure -/// policy: given the selected `VideoProvider`, the master gates, the user whitelist (via the shared -/// `WritePolicyResolving`), the budget, and whether the session is parked, it resolves WHICH gate the -/// routed call runs at. MLX-free Core, `swift test`-verified. -/// -/// The contract this enforces, layered onto the existing `MediaGenSink`: -/// - CLOUD → `.dangerous` tier, intersected with the user whitelist (a `.dangerous` descriptor is NEVER -/// lowered to `.auto` by the resolver — spec). Confirm-by-default per call; over-budget DEGRADES, never -/// silently drops, never auto-runs (spec "Over-budget cloud video degrades, never runs unprompted"). -/// - LOCAL → gated by the MASTER toggle (`fullPotentialEnabled && mediaGenEnabled`), NOT on the spend -/// axis: `.confirm` (frontier, expensive compute) but it consumes no budget ledger (design D6). - -/// The resolved gate for ONE `generate_video` call (parallel to `HandoffGate`). -public enum VideoGate: Equatable, Sendable { - /// The provider isn't available (local selected with the master OFF, or no provider configured) → - /// declined, no compute, no spend. - case providerDisabled - /// Local frontier, master-gated → confirm-by-default per call, NO budget, NO upload. - case localConfirm - /// Cloud, under budget → confirm-by-default per call (`.dangerous`); on approval it spends one budget - /// unit and uploads. - case cloudConfirm - /// Cloud, OVER budget → DEGRADE: active session → a foreground confirm STATING the cap was reached; - /// parked session → a needs-you badge. Never auto-run, never silently dropped. - case cloudOverBudget - - /// True when this gate spends from the cloud budget on approval (cloud only). - public var spendsCloudBudget: Bool { - switch self { - case .cloudConfirm, .cloudOverBudget: return true - case .providerDisabled, .localConfirm: return false - } - } - - /// The effective `WritePolicyTier` the audit record carries for this gate. Cloud → `.dangerous` - /// (spend + leaves device); local → `.confirm`. A disabled provider audits at `.dangerous` for cloud - /// (the descriptor default) but never runs. - public var auditTier: WritePolicyTier { - switch self { - case .cloudConfirm, .cloudOverBudget: return .dangerous - case .localConfirm: return .confirm - case .providerDisabled: return .confirm - } - } -} - -/// The pure resolver. Inputs are values / injected closures — no clock, no network, no MLX. -/// Internal because it composes the internal `WritePolicyResolving` seam (`DescriptorWritePolicy` / -/// `BackgroundPolicyResolver`); the test target reaches it via `@testable import`. -struct VideoTierResolver: Sendable { - private let provider: VideoProvider - private let resolver: WritePolicyResolving - private let isFullPotentialEnabled: @Sendable () -> Bool - private let isMediaGenEnabled: @Sendable () -> Bool - - init(provider: VideoProvider, - resolver: WritePolicyResolving = DescriptorWritePolicy(), - isFullPotentialEnabled: @escaping @Sendable () -> Bool = { false }, - isMediaGenEnabled: @escaping @Sendable () -> Bool = { false }) { - self.provider = provider - self.resolver = resolver - self.isFullPotentialEnabled = isFullPotentialEnabled - self.isMediaGenEnabled = isMediaGenEnabled - } - - /// The EFFECTIVE write-policy tier for the `generate_video` descriptor under this provider (task 3.1): - /// - cloud → `.dangerous` ∩ whitelist (the resolver NEVER lowers `.dangerous`); - /// - local → `.confirm` (master-gated, off the spend axis). - func effectiveTier(for descriptor: ToolDescriptor) -> WritePolicyTier { - if provider.isCloud { - // The descriptor is authored `.dangerous` for cloud; intersect with the whitelist (which can - // never lower `.dangerous` to `.auto`). The resulting tier stays `.dangerous`. - return resolver.effectiveTier(for: descriptor) - } - // Local frontier: confirm-by-default, never on the spend axis. - return .confirm - } - - /// Resolve the gate for THIS call (task 3.1 / 3.2). `budgetHasRoom` is the cloud budget predicate the - /// caller evaluates against the injected clock (kept out of this pure resolver so `now` stays an input - /// at the boundary that owns it). - func gate(budgetHasRoom: Bool) -> VideoGate { - if provider.isCloud { - return budgetHasRoom ? .cloudConfirm : .cloudOverBudget - } - // Local: only selectable with the master gates ON (a settings desync where local persisted under a - // since-disabled master degrades to provider-disabled rather than running a forbidden frontier). - guard isFullPotentialEnabled() && isMediaGenEnabled() else { return .providerDisabled } - return .localConfirm - } - - /// The over-budget DEGRADE message stated on the approval surface (task 3.2) — names that the cap was - /// reached (active → confirm card; parked → needs-you reason). Bounded, non-blocking; never raw text. - static func overBudgetReason(parked: Bool) -> String { - parked - ? "Today's video budget is used up — approve when you're back to spend over the cap." - : "Today's video budget is used up. Approve to generate one more (it spends over the cap)?" - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Media/Video/VideoUploadDisclosure.swift b/Sources/ThreeFingerSwitcher/AI/Media/Video/VideoUploadDisclosure.swift deleted file mode 100644 index 79dd3b9..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Media/Video/VideoUploadDisclosure.swift +++ /dev/null @@ -1,78 +0,0 @@ -import Foundation - -/// The HONEST upload + cost disclosure for a video generation (`ai-video-animation-generation`, design D4 / -/// tasks 1.2, 7.1, 7.2). A pure value so the confirm step AND the audit summary are testable WITHOUT -/// network: it states whether bytes leave the device, builds the REDACTED argument summary (provider + -/// TRUNCATED prompt + seed-present flag — never the full prompt verbatim, never the raw seed bytes), and — -/// for the cloud provider — carries the per-clip $ order; for local, the residency/heat/eviction cost. -/// -/// Redaction rule (blueprint): raw prompt text and raw seed bytes ride ONLY in logs / behind an opt-in -/// "Show details" disclosure — NEVER in the headline, NEVER in the audit summary. This builder enforces it -/// by middle-truncating the prompt through `AuditRedaction` and reducing the seed to a boolean. MLX-free -/// Core. -public struct VideoUploadDisclosure: Equatable, Sendable { - /// True iff the generation sends bytes off-device (the prompt + any seed frame). Cloud → true; local → - /// false (nothing is uploaded — spec "Local video does not claim an upload"). - public let bytesLeaveDevice: Bool - /// Which backend (cloud / local LTXV) — surfaced on the confirm step + recorded in the audit summary. - public let provider: VideoProvider - /// A TRUNCATED, single-line prompt preview (never the full prompt). Built through `AuditRedaction` so a - /// secret-shaped token is scrubbed + the line is bounded. - public let truncatedPrompt: String - /// Whether a seed IMAGE was sent (img2video). A boolean — the raw seed bytes NEVER appear here. - public let seedPresent: Bool - - public init(bytesLeaveDevice: Bool, - provider: VideoProvider, - truncatedPrompt: String, - seedPresent: Bool) { - self.bytesLeaveDevice = bytesLeaveDevice - self.provider = provider - self.truncatedPrompt = truncatedPrompt - self.seedPresent = seedPresent - } - - /// Build the disclosure for a routed video request. `provider.isCloud` decides `bytesLeaveDevice`; the - /// prompt is middle-truncated + secret-scrubbed; the seed is reduced to a present/absent flag. - public static func make(provider: VideoProvider, prompt: String, seedPresent: Bool) -> VideoUploadDisclosure { - VideoUploadDisclosure( - bytesLeaveDevice: provider.isCloud, - provider: provider, - truncatedPrompt: AuditRedaction.summary(forRawArguments: prompt), - seedPresent: seedPresent - ) - } - - // MARK: - Audit summary (the redacted record line) - - /// The single-line REDACTED argument summary for the audit record (task 4.1): provider · truncated - /// prompt · seed flag. Bounded to `AuditRedaction.maxSummaryLength`; the full prompt / raw seed bytes - /// are NEVER present. This is the SOLE string that reaches `AuditRecord.argumentsSummary` for a video - /// attempt. - public var auditSummary: String { - let seedTag = seedPresent ? "seed:yes" : "seed:no" - let promptPart = truncatedPrompt.isEmpty ? "(no prompt)" : truncatedPrompt - let line = "\(provider.rawValue) · \(promptPart) · \(seedTag)" - return AuditRedaction.middleTruncate(line, to: AuditRedaction.maxSummaryLength) - } - - // MARK: - Confirm-step copy (the user-facing disclosure) - - /// The CLOUD confirm-step line (task 7.1): states the bytes leave the device + the per-clip $ order - /// (from `mediaVideoBudgetPerDay` semantics — the caller passes the per-clip cost order). Only meaningful - /// when `bytesLeaveDevice`; for local it returns the residency/heat line instead via `localCostLine`. - public func cloudConfirmLine(perClipCostOrder: String) -> String { - guard bytesLeaveDevice else { return Self.localCostLine } - let seedClause = seedPresent ? " and your source image" : "" - return "Your prompt\(seedClause) will be sent to the remote video service " - + "(\(perClipCostOrder) per clip, from today's budget)." - } - - /// The LOCAL-LTXV selection disclosure (task 7.2): tens-of-GB residency, minutes-per-clip latency, chat - /// EVICTION (the assistant goes quiet while it paints — consumed from the fleet residency decision §C1), - /// and the thermal cost. Stated in the same breath the frontier is offered (spec "Selecting local LTXV - /// discloses its cost"). A static line — local video spends no money, so there is no $ to disclose. - public static let localCostLine = - "Local video is a frontier option: tens of gigabytes of weights, minutes per clip, the assistant " - + "goes quiet while it paints (chat is evicted), and your Mac runs hot. Nothing is uploaded." -} diff --git a/Sources/ThreeFingerSwitcher/AI/Memory/MemoryAudit.swift b/Sources/ThreeFingerSwitcher/AI/Memory/MemoryAudit.swift deleted file mode 100644 index 34936f5..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Memory/MemoryAudit.swift +++ /dev/null @@ -1,51 +0,0 @@ -import Foundation - -/// The audit seam this slice emits into (design §4 / task §6.3). `AuditRecord`/`AuditLog` are OWNED by -/// `ai-background-autonomy` (Wave 4) — when that slice lands it supplies the production conformer and the -/// durable log; this slice ships the protocol + a record so it compiles + tests its emission without a -/// DAG back-edge. Mirrors the `WritePolicyResolving`/`DescriptorWritePolicy` stand-in discipline. -/// -/// EVERY memory invoke (read included) emits one record. The `argumentsSummary` is REDACTED/SHORT — a -/// subfile name + a content length, never the raw secret content (design §4: "redacted/short"). - -/// One audited memory operation. -struct MemoryAuditRecord: Equatable, Sendable { - var sessionID: AgentSessionID? - var tool: String - var policy: WritePolicyTier - /// Redacted/short — never the raw content (e.g. `scope=subfile name=acme len=812`). - var argumentsSummary: String - var outcome: MemoryAuditOutcome - var wasBackground: Bool - var timestamp: Date - - init(sessionID: AgentSessionID?, tool: String, policy: WritePolicyTier, argumentsSummary: String, - outcome: MemoryAuditOutcome, wasBackground: Bool, timestamp: Date = Date()) { - self.sessionID = sessionID - self.tool = tool - self.policy = policy - self.argumentsSummary = argumentsSummary - self.outcome = outcome - self.wasBackground = wasBackground - self.timestamp = timestamp - } -} - -/// The outcome of an audited memory op (mirrors `ToolStepStatus` but audit-shaped). -enum MemoryAuditOutcome: Equatable, Sendable { - case done - case declined(reason: String) - case failed(headline: String) -} - -/// The sink memory writes its audit records into. `ai-background-autonomy` owns the durable conformer; -/// this slice injects it (tests inject a recording fake). -protocol MemoryAuditing: Sendable { - func record(_ record: MemoryAuditRecord) -} - -/// A no-op default so the store/provider compile + run without the audit slice present. -struct NullMemoryAuditing: MemoryAuditing { - init() {} - func record(_ record: MemoryAuditRecord) {} -} diff --git a/Sources/ThreeFingerSwitcher/AI/Memory/MemoryDocument.swift b/Sources/ThreeFingerSwitcher/AI/Memory/MemoryDocument.swift deleted file mode 100644 index 83f0302..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Memory/MemoryDocument.swift +++ /dev/null @@ -1,139 +0,0 @@ -import Foundation - -/// One ground-truth fact line in the CORE document. Short — identity facts ("I work at Acme"), not -/// detail (detail lives in subfiles). -struct MemoryFact: Equatable, Sendable { - var text: String - init(_ text: String) { self.text = text } -} - -/// One table-of-contents entry in the CORE document: a subfile's name + its one-line summary. Mirrors -/// the subfiles folder; kept honest by reconciliation (design §6). -struct MemoryTOCEntry: Equatable, Sendable { - var name: String - var summary: String -} - -/// The hard cap on the always-read CORE tier (design §2). The cap is on the SERIALIZED core because core -/// is injected into every session's context — the cost is the always-read token budget, not disk. Both a -/// byte cap and a fact-count cap (whichever binds first) make the decision robust to a few very long -/// facts vs. many short ones. -struct MemoryCap: Equatable, Sendable { - var maxBytes: Int - var maxFacts: Int - - /// Tuned defaults: ~8 KB / 60 facts keeps the always-read tier tiny by construction. - static let `default` = MemoryCap(maxBytes: 8 * 1024, maxFacts: 60) -} - -/// The pure CORE model (design §2, MLX-free Core). A `## Facts` list + a `## Contents` (TOC) list, with -/// pure `parse`/`serialized` and the cap/eviction decision as pure functions — no `FileManager`, fully -/// `swift test`-able headless. The `MemoryStore` bridges IO and turns evicted facts into real subfiles. -struct MemoryDocument: Equatable, Sendable { - var facts: [MemoryFact] - var contents: [MemoryTOCEntry] - - init(facts: [MemoryFact] = [], contents: [MemoryTOCEntry] = []) { - self.facts = facts - self.contents = contents - } - - static let factsHeading = "## Facts" - static let contentsHeading = "## Contents" - - // MARK: - Parse / serialize - - /// Parse `core.md` text. Tolerant: a missing section is empty; a fact is a `- ` bullet under - /// `## Facts`; a TOC entry is a `- name — summary` bullet under `## Contents`. An absent/empty - /// document is a valid empty doc (never an error). - static func parse(_ text: String) -> MemoryDocument { - var facts: [MemoryFact] = [] - var contents: [MemoryTOCEntry] = [] - enum Section { case none, facts, contents } - var section: Section = .none - - for rawLine in text.components(separatedBy: "\n") { - let line = rawLine.trimmingCharacters(in: .whitespaces) - if line == factsHeading { section = .facts; continue } - if line == contentsHeading { section = .contents; continue } - if line.hasPrefix("## ") { section = .none; continue } // an unrelated heading ends a section - guard line.hasPrefix("- ") else { continue } - let item = String(line.dropFirst(2)).trimmingCharacters(in: .whitespaces) - guard !item.isEmpty else { continue } - switch section { - case .facts: - facts.append(MemoryFact(item)) - case .contents: - if let entry = parseTOCLine(item) { contents.append(entry) } - case .none: - continue - } - } - return MemoryDocument(facts: facts, contents: contents) - } - - /// Serialize back to `core.md` text. `parse(serialized())` is a fixed point. Both sections are always - /// emitted (an empty section is just its heading) so the round-trip is byte-stable. - func serialized() -> String { - var lines: [String] = [Self.factsHeading] - for fact in facts { lines.append("- \(fact.text)") } - lines.append("") - lines.append(Self.contentsHeading) - for entry in contents { lines.append("- \(entry.name) — \(entry.summary)") } - return lines.joined(separator: "\n") + "\n" - } - - /// The CORE facts as plain text (for the `.memoryCore` IndexedDoc body the shared index serves). - func factsBody() -> String { - facts.map { "- \($0.text)" }.joined(separator: "\n") - } - - private static func parseTOCLine(_ item: String) -> MemoryTOCEntry? { - // `name — summary` (em-dash separator, mirroring the serialized form). Tolerate a plain ` - ` - // hyphen too. Name only (no separator) is allowed (empty summary). - for sep in [" — ", " - "] { - if let r = item.range(of: sep) { - let name = String(item[.. Bool { - var probe = self - probe.facts.append(fact) - return probe.exceeds(cap) - } - - func exceeds(_ cap: MemoryCap) -> Bool { - if facts.count > cap.maxFacts { return true } - if serialized().utf8.count > cap.maxBytes { return true } - return false - } - - /// Evict the lowest-value facts to fit the cap. v1 policy: **oldest-among-longest** — repeatedly move - /// out the longest fact, breaking ties toward the oldest (earliest index), favoring evicting bulky - /// detail-bearing lines while keeping terse identity facts. Returns the residual document + the - /// evicted facts (the store bundles them into a subfile + a TOC entry). Terminates: every step removes - /// one fact, so it cannot loop forever (a single fact still over the byte cap is the caller's - /// `.capExceeded` guard). - func evicting(toFit cap: MemoryCap) -> (kept: MemoryDocument, evicted: [MemoryFact]) { - var kept = self - var evicted: [MemoryFact] = [] - while kept.exceeds(cap), !kept.facts.isEmpty { - // Longest fact wins; tie → earliest index (oldest). - var victim = 0 - for i in kept.facts.indices where kept.facts[i].text.utf8.count > kept.facts[victim].text.utf8.count { - victim = i - } - evicted.append(kept.facts.remove(at: victim)) - } - return (kept, evicted) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Memory/MemoryError.swift b/Sources/ThreeFingerSwitcher/AI/Memory/MemoryError.swift deleted file mode 100644 index ba9ee7d..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Memory/MemoryError.swift +++ /dev/null @@ -1,34 +0,0 @@ -import Foundation - -/// Agent-memory load/write failures (design §7) — a new taxonomy ONLY for the cases `RuntimeError`/ -/// `TaskError`/`SkillError` cannot carry. Each case has a clean, user-facing `errorDescription`; raw -/// OS/parse text stays in opt-in details / logs (never the headline). `AIError.message(for:)` is -/// extended to translate this (the single translator). `FileManager`/parse throws map into this at the -/// `MemoryStore` IO boundary; Core stays MLX-free. -enum MemoryError: Error, Equatable, LocalizedError { - /// The CORE document could not be read off disk (permission, corruption). - case unreadableCore(detail: String) - /// A memory write's disk IO did not land (disk full, permission, read-only volume). - case writeFailed(detail: String) - /// A named subfile the agent referenced does not exist. - case subfileNotFound(name: String) - /// A single fact larger than the entire core cap was forced into core (eviction cannot make room). - case capExceeded - /// A subfile on disk has malformed front-matter (excluded from the index, the rest still load). - case malformedSubfile(name: String, detail: String) - - var errorDescription: String? { - switch self { - case .unreadableCore: - return "Your memory's core file could not be read." - case .writeFailed: - return "That memory couldn't be saved." - case let .subfileNotFound(name): - return "There's no memory note named “\(name)”." - case .capExceeded: - return "That's too long to keep as a core fact — save it as a note instead." - case let .malformedSubfile(name, _): - return "The memory note “\(name)” couldn't be read." - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Memory/MemoryReconcile.swift b/Sources/ThreeFingerSwitcher/AI/Memory/MemoryReconcile.swift deleted file mode 100644 index 4150874..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Memory/MemoryReconcile.swift +++ /dev/null @@ -1,28 +0,0 @@ -import Foundation - -/// The pure reconciliation pass (design §6, task §3.1). The subfiles folder is the **existence source of -/// truth**; the CORE's `## Contents` block is a derived view kept honest against it: a subfile with no TOC -/// entry gains one (from its front-matter `summary`); a TOC entry with no backing subfile is dropped. -/// `## Facts` is untouched. Pure — no `FileManager` (the store passes in the parsed subfile front-matters). -extension MemoryDocument { - /// Return a normalized document whose `## Contents` matches the actual subfiles. Order: existing TOC - /// entries first (preserving their order, summaries refreshed from the subfile), then any subfiles - /// missing a TOC entry, appended in the given order. - func reconciled(withSubfiles subfiles: [MemorySubfile]) -> MemoryDocument { - let byName = Dictionary(subfiles.map { ($0.name, $0) }, uniquingKeysWith: { a, _ in a }) - var seen = Set() - var newContents: [MemoryTOCEntry] = [] - - for entry in contents { - guard let sub = byName[entry.name] else { continue } // drop stale TOC line (no backing subfile) - if seen.insert(entry.name).inserted { - newContents.append(sub.tocEntry) // refresh summary from the subfile - } - } - for sub in subfiles where !seen.contains(sub.name) { - seen.insert(sub.name) - newContents.append(sub.tocEntry) // add missing TOC line from front-matter - } - return MemoryDocument(facts: facts, contents: newContents) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Memory/MemoryStore.swift b/Sources/ThreeFingerSwitcher/AI/Memory/MemoryStore.swift deleted file mode 100644 index 8de4e1b..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Memory/MemoryStore.swift +++ /dev/null @@ -1,326 +0,0 @@ -import Foundation -import os - -private let memoryLog = Logger(subsystem: "ThreeFingerSwitcher", category: "MemoryStore") - -/// The scope a write/forget targets. -enum MemoryScope: String, Equatable, Sendable { case fact, subfile } - -/// The result of a memory write/update/promote — what landed, and whether eviction ran. -struct MemoryWriteOutcome: Equatable, Sendable { - /// A short, redacted human description for the audit `argumentsSummary` / the tool `summary`. - var summary: String - /// Facts evicted to a subfile to keep core under the cap (informational; may be empty). - var evictedToSubfile: String? -} - -/// The result of a forget — how many entries it touched and whether the op classifies dangerous. -struct MemoryForgetOutcome: Equatable, Sendable { - var removedCount: Int - var dangerous: Bool - var summary: String -} - -/// The two-tier on-disk memory store (design §5, MLX-free Core). Parallels `DiskProjectStore`: an -/// Application-Support directory under `ThreeFingerSwitcher/memory`, a sanitized deterministic subfile -/// filename, every `FileManager`/`FileHandle` throw mapped into `MemoryError` AT THE IO BOUNDARY (raw OS -/// error to the log only). The pure `MemoryDocument`/`MemorySubfile`/reconcile logic never touches -/// `FileManager`; this store bridges IO and produces the `[IndexedDoc]` snapshot the SHARED -/// `InMemoryDocIndex` answers over (NO second retriever). Containment is structural: every write is -/// rooted at `directory` via the `DiskProjectStore` sanitizer, so a write can never escape the memory -/// folder — which is what makes auto-when-parked safe (design §4). -final class MemoryStore: @unchecked Sendable { - private let directory: URL - /// Where evicted core facts are bundled (design §2/§3). - static let evictionSubfileName = "evicted-from-core" - - /// Test/seam initializer: inject an isolated directory (e.g. a temp dir). - init(directory: URL = MemoryStore.defaultDirectory()) { - self.directory = directory - } - - /// Default: `~/Library/Application Support/ThreeFingerSwitcher/memory` (parallels `DiskProjectStore`). - static func defaultDirectory() -> URL { - let base = (try? FileManager.default.url(for: .applicationSupportDirectory, in: .userDomainMask, - appropriateFor: nil, create: true)) - ?? URL(fileURLWithPath: NSTemporaryDirectory()) - return base.appendingPathComponent("ThreeFingerSwitcher/memory", isDirectory: true) - } - - private var coreURL: URL { directory.appendingPathComponent("core.md") } - private var subfilesDir: URL { directory.appendingPathComponent("subfiles", isDirectory: true) } - - /// A safe, deterministic subfile filename — REUSES `DiskProjectStore.fileName(for:)` (alphanumerics - /// + `-_ `, never empty, traversal/slashes collapsed to `-`), so a hostile `name` (`../`, slashes) - /// stays rooted inside the memory folder (containment). - private func subfileURL(for name: String) -> URL { - subfilesDir.appendingPathComponent(DiskProjectStore.fileName(for: name)) - } - - // MARK: - Load + index (design §5, task §4.2) - - /// Read `core.md` off disk into a `MemoryDocument`, list + parse subfiles, run reconcile. Bounded: - /// a malformed subfile is excluded (a logged problem), the rest load; an absent folder is an empty - /// (valid) document. A genuinely unreadable core (present but undecodable) is `.unreadableCore`. - func loadCore() throws -> MemoryDocument { - let (doc, subfiles, _) = try loadAll() - return doc.reconciled(withSubfiles: subfiles) - } - - /// Load + reconcile + produce the memory `IndexedDoc`s — one `.memoryCore` (facts) + one - /// `.memorySubfile` per subfile — plus the bodies map, for merging into the SHARED snapshot. - func indexedDocs() throws -> (docs: [IndexedDoc], bodies: [String: String]) { - let (doc, subfiles, _) = try loadAll() - let reconciled = doc.reconciled(withSubfiles: subfiles) - - var docs: [IndexedDoc] = [] - var bodies: [String: String] = [:] - - let coreID = MemoryStore.coreDocID - docs.append(IndexedDoc(id: coreID, title: "Core memory", - summary: "Ground-truth facts about the user.", - keywords: ["memory", "facts", "core", "about", "me"], - kind: .memoryCore, bodyPath: coreURL, - updatedAt: (try? coreURL.modifiedDate()) ?? Date(timeIntervalSince1970: 0))) - bodies[coreID] = reconciled.factsBody() - - for sub in subfiles { - let id = MemorySubfile.docID(for: sub.name) - docs.append(sub.indexedDoc(bodyPath: subfileURL(for: sub.name))) - bodies[id] = sub.body - } - return (docs, bodies) - } - - static let coreDocID = "memory/core" - - /// A subfile body / the CORE facts text, for the shared index (cached at the file layer by the OS). - func body(of id: String) throws -> String { - if id == MemoryStore.coreDocID { return try loadCore().factsBody() } - let (_, subfiles, _) = try loadAll() - guard let sub = subfiles.first(where: { MemorySubfile.docID(for: $0.name) == id }) else { - throw MemoryError.subfileNotFound(name: id) - } - return sub.body - } - - // MARK: - Writes (design §3/§4, task §4.3) - - /// Write/update a fact or a subfile. A fact write applies the cap + eviction (evicted facts become a - /// subfile + TOC entry); a subfile write adds/refreshes its TOC entry. Atomic, then reconciled. - func write(scope: MemoryScope, name: String?, summary: String?, content: String, - cap: MemoryCap = .default) throws -> MemoryWriteOutcome { - switch scope { - case .fact: - return try addFact(content, cap: cap, forcePromote: false) - case .subfile: - let n = (name?.isEmpty == false) ? name! : Self.derivedSubfileName(from: content) - try writeSubfile(name: n, summary: summary ?? Self.derivedSummary(from: content), body: content) - return MemoryWriteOutcome(summary: "Saved note “\(n)” (\(content.utf8.count) bytes).", - evictedToSubfile: nil) - } - } - - /// Replace a named subfile's body/summary. Missing subfile → `.subfileNotFound`. - func update(name: String, content: String, summary: String?) throws -> MemoryWriteOutcome { - let (_, subfiles, _) = try loadAll() - guard let existing = subfiles.first(where: { $0.name == name }) else { - throw MemoryError.subfileNotFound(name: name) - } - try writeSubfile(name: name, summary: summary ?? existing.summary, body: content) - return MemoryWriteOutcome(summary: "Updated note “\(name)” (\(content.utf8.count) bytes).", - evictedToSubfile: nil) - } - - /// Propose a CORE fact (design §3). The cap is the structural backstop: an approved promotion that - /// would breach the cap evicts to a subfile. A single fact larger than the whole cap is `.capExceeded`. - func promote(content: String, cap: MemoryCap = .default) throws -> MemoryWriteOutcome { - try addFact(content, cap: cap, forcePromote: true) - } - - /// Remove a fact / a subfile (+ its TOC entry), or a `match` across either tier. A `match` hitting - /// MANY entries (or a core-wide clear) classifies dangerous (returned so the tool sets `.dangerous`). - /// A no-match forget is a clean no-op, never a throw. - func forget(scope: MemoryScope?, name: String?, match: String?) throws -> MemoryForgetOutcome { - var (doc, subfiles, _) = try loadAll() - doc = doc.reconciled(withSubfiles: subfiles) - var removed = 0 - - if let name, scope == .subfile { - if subfiles.contains(where: { $0.name == name }) { - try deleteSubfileFile(name: name) - subfiles.removeAll { $0.name == name } - removed += 1 - } - } else if let match, !match.isEmpty { - let needle = match.lowercased() - // Facts matching the needle. - let factsBefore = doc.facts.count - doc.facts.removeAll { $0.text.lowercased().contains(needle) } - removed += factsBefore - doc.facts.count - // Subfiles whose name/summary match. - for sub in subfiles where sub.name.lowercased().contains(needle) - || sub.summary.lowercased().contains(needle) { - try deleteSubfileFile(name: sub.name) - removed += 1 - } - subfiles.removeAll { $0.name.lowercased().contains(needle) || $0.summary.lowercased().contains(needle) } - } else if scope == .fact, let name { - // Forget a fact by exact text. - let before = doc.facts.count - doc.facts.removeAll { $0.text == name } - removed += before - doc.facts.count - } - - let reconciled = doc.reconciled(withSubfiles: subfiles) - try writeCore(reconciled) - let dangerous = Self.isDangerousForget(removedCount: removed, match: match) - return MemoryForgetOutcome(removedCount: removed, dangerous: dangerous, - summary: removed == 0 ? "Nothing matched — nothing removed." - : "Removed \(removed) memory item(s).") - } - - /// A bulk forget (a `match` removing many entries) or a core-wide clear is dangerous (design §4). - static func isDangerousForget(removedCount: Int, match: String?) -> Bool { - // A `match`-driven forget that hit more than one entry escalates; a targeted single forget does not. - (match != nil && removedCount > 1) - } - - // MARK: - Fact write + eviction core - - private func addFact(_ content: String, cap: MemoryCap, forcePromote: Bool) throws -> MemoryWriteOutcome { - let (rawDoc, subfiles, _) = try loadAll() - var doc = rawDoc.reconciled(withSubfiles: subfiles) - let fact = MemoryFact(content) - - // A single fact larger than the whole cap can never fit. If forced (promote), it's a clean - // `.capExceeded`; for a plain fact write it is routed to a subfile (it is detail, not a fact). - let soloByteCount = MemoryDocument(facts: [fact]).serialized().utf8.count - if soloByteCount > cap.maxBytes { - if forcePromote { throw MemoryError.capExceeded } - let n = Self.derivedSubfileName(from: content) - try writeSubfile(name: n, summary: Self.derivedSummary(from: content), body: content) - return MemoryWriteOutcome(summary: "Too long for core — saved as note “\(n)”.", - evictedToSubfile: n) - } - - doc.facts.append(fact) - var evictedName: String? - if doc.exceeds(cap) { - let (kept, evicted) = doc.evicting(toFit: cap) - doc = kept - if !evicted.isEmpty { - evictedName = try appendEviction(evicted) - // Re-list subfiles so the eviction subfile's TOC entry reconciles in. - let (_, freshSubfiles, _) = try loadAll() - doc = doc.reconciled(withSubfiles: freshSubfiles) - } - } - try writeCore(doc) - return MemoryWriteOutcome( - summary: evictedName == nil ? "Kept in core (\(content.utf8.count) bytes)." - : "Kept in core; evicted older facts to note “\(evictedName!)”.", - evictedToSubfile: evictedName) - } - - /// Bundle evicted facts into the eviction subfile (appending to its body if it already exists). - private func appendEviction(_ facts: [MemoryFact]) throws -> String { - let name = Self.evictionSubfileName - let (_, subfiles, _) = try loadAll() - let existingBody = subfiles.first(where: { $0.name == name })?.body - let appended = facts.map { "- \($0.text)" }.joined(separator: "\n") - let body = [existingBody, appended].compactMap { $0?.isEmpty == false ? $0 : nil }.joined(separator: "\n") - try writeSubfile(name: name, summary: "Detail facts evicted from core to stay within the cap.", - body: body) - return name - } - - // MARK: - IO boundary (every throw maps to MemoryError here; raw text to the log only) - - /// Read core + all subfiles + the bounded problems. An absent folder is an empty doc. - private func loadAll() throws -> (MemoryDocument, [MemorySubfile], [MemoryError]) { - let fm = FileManager.default - var doc = MemoryDocument() - if fm.fileExists(atPath: coreURL.path) { - do { - doc = MemoryDocument.parse(try String(contentsOf: coreURL, encoding: .utf8)) - } catch { - memoryLog.error("core read failed: \(String(describing: error), privacy: .public)") - throw MemoryError.unreadableCore(detail: String(describing: error)) - } - } - var subfiles: [MemorySubfile] = [] - var problems: [MemoryError] = [] - if let entries = try? fm.contentsOfDirectory(at: subfilesDir, includingPropertiesForKeys: nil) { - for file in entries.filter({ $0.pathExtension == "md" }) - .sorted(by: { $0.lastPathComponent < $1.lastPathComponent }) { - guard let text = try? String(contentsOf: file, encoding: .utf8) else { continue } - let stem = file.deletingPathExtension().lastPathComponent - switch MemorySubfile.parse(text, fallbackName: stem) { - case let .success(s): subfiles.append(s) - case let .failure(e): - problems.append(e) - memoryLog.error("subfile parse failed: \(String(describing: e), privacy: .public)") - } - } - } - return (doc, subfiles, problems) - } - - private func writeCore(_ doc: MemoryDocument) throws { - do { - try FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true) - try Data(doc.serialized().utf8).write(to: coreURL, options: .atomic) - } catch { - memoryLog.error("core write failed: \(String(describing: error), privacy: .public)") - throw MemoryError.writeFailed(detail: String(describing: error)) - } - } - - private func writeSubfile(name: String, summary: String, body: String) throws { - let sub = MemorySubfile(name: name, summary: summary, body: body, updatedAt: Date()) - do { - try FileManager.default.createDirectory(at: subfilesDir, withIntermediateDirectories: true) - try Data(sub.serialized().utf8).write(to: subfileURL(for: name), options: .atomic) - // Keep the core TOC honest after a subfile write (reconcile against the fresh folder). - let (doc, subfiles, _) = try loadAll() - try writeCore(doc.reconciled(withSubfiles: subfiles)) - } catch let e as MemoryError { - throw e - } catch { - memoryLog.error("subfile write failed: \(String(describing: error), privacy: .public)") - throw MemoryError.writeFailed(detail: String(describing: error)) - } - } - - private func deleteSubfileFile(name: String) throws { - do { - let url = subfileURL(for: name) - if FileManager.default.fileExists(atPath: url.path) { - try FileManager.default.removeItem(at: url) - } - } catch { - memoryLog.error("subfile delete failed: \(String(describing: error), privacy: .public)") - throw MemoryError.writeFailed(detail: String(describing: error)) - } - } - - // MARK: - Derivations - - static func derivedSubfileName(from content: String) -> String { - let firstLine = content.split(whereSeparator: \.isNewline).first.map(String.init) ?? "note" - let slug = SkillStore.slug(firstLine) - return slug.isEmpty ? "note" : String(slug.prefix(48)) - } - - static func derivedSummary(from content: String) -> String { - let firstLine = content.split(whereSeparator: \.isNewline).first.map(String.init) ?? content - return firstLine.count > 80 ? String(firstLine.prefix(80)) + "…" : firstLine - } -} - -private extension URL { - func modifiedDate() throws -> Date { - try resourceValues(forKeys: [.contentModificationDateKey]).contentModificationDate ?? Date() - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Memory/MemorySubfile.swift b/Sources/ThreeFingerSwitcher/AI/Memory/MemorySubfile.swift deleted file mode 100644 index 3e03d22..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Memory/MemorySubfile.swift +++ /dev/null @@ -1,111 +0,0 @@ -import Foundation - -/// A named memory subfile (design §1/§2.1): front-matter (`name`/`summary`/`keywords`/`updatedAt`) + a -/// free-Markdown detail body, mirroring the `.skill.md` shape so a subfile and a skill index identically. -/// Pure value type; parse/serialize never touch `FileManager`. A malformed file maps at the parse -/// boundary into `MemoryError.malformedSubfile` (a bounded problem — excluded from the index, the rest -/// still load). MLX-free Core. -struct MemorySubfile: Equatable, Sendable { - var name: String - var summary: String - var keywords: [String] - var body: String - var updatedAt: Date - - init(name: String, summary: String, keywords: [String] = [], body: String, updatedAt: Date = Date()) { - self.name = name - self.summary = summary - self.keywords = keywords - self.body = body - self.updatedAt = updatedAt - } - - /// The TOC entry this subfile contributes to `core.md`'s `## Contents` (design §6). - var tocEntry: MemoryTOCEntry { MemoryTOCEntry(name: name, summary: summary) } - - /// The `IndexedDoc` this subfile contributes to the SHARED `DocIndex` (kind `.memorySubfile`; the - /// body is the detail text, served by the store). Id namespaced under `memory/` so a subfile name can - /// never collide with a skill id (different folder + kind — the documented contract). - func indexedDoc(bodyPath: URL) -> IndexedDoc { - IndexedDoc(id: MemorySubfile.docID(for: name), title: name, summary: summary, - keywords: keywords, kind: .memorySubfile, bodyPath: bodyPath, updatedAt: updatedAt) - } - - /// The namespaced, path-relative id for a subfile (structurally distinct from any skill id). - static func docID(for name: String) -> String { "memory/subfile/\(name)" } - - // MARK: - Parse / serialize (mirrors SkillFile) - - /// Parse a subfile document. Returns a typed `MemoryError.malformedSubfile` on any malformation - /// (missing delimiters, missing `name`/`summary`). `fallbackName` (the on-disk filename stem) is used - /// in the error so a problem is attributable even when the front-matter `name` is the thing missing. - static func parse(_ text: String, fallbackName: String) -> Result { - guard let (frontMatter, body) = splitFrontMatter(text) else { - return .failure(.malformedSubfile(name: fallbackName, detail: "Missing the `---` front-matter delimiters.")) - } - let fields = parseFields(frontMatter) - guard let name = fields["name"], !name.isEmpty else { - return .failure(.malformedSubfile(name: fallbackName, detail: "Missing required field: name.")) - } - guard let summary = fields["summary"], !summary.isEmpty else { - return .failure(.malformedSubfile(name: name, detail: "Missing required field: summary.")) - } - let keywords = parseList(fields["keywords"]) - let updatedAt = fields["updatedAt"].flatMap(parseDate) ?? Date(timeIntervalSince1970: 0) - return .success(MemorySubfile(name: name, summary: summary, keywords: keywords, - body: body, updatedAt: updatedAt)) - } - - /// Serialize back to subfile text. `parse(serialize(s))` is a fixed point. - func serialized() -> String { - var lines = ["---"] - lines.append("name: \(name)") - lines.append("summary: \(summary)") - if !keywords.isEmpty { lines.append("keywords: [\(keywords.joined(separator: ", "))]") } - lines.append("updatedAt: \(Self.isoString(updatedAt))") - lines.append("---") - return lines.joined(separator: "\n") + "\n" + body - } - - // MARK: - Helpers (mirror SkillFile's hand-rolled subset) - - private static func splitFrontMatter(_ text: String) -> (frontMatter: [String], body: String)? { - let lines = text.components(separatedBy: "\n") - guard let firstDelim = lines.firstIndex(where: { $0.trimmingCharacters(in: .whitespaces) == "---" }) else { return nil } - let rest = lines[(firstDelim + 1)...] - guard let closeOffset = rest.firstIndex(where: { $0.trimmingCharacters(in: .whitespaces) == "---" }) else { return nil } - let fm = Array(lines[(firstDelim + 1).. [String: String] { - var fields: [String: String] = [:] - for line in lines { - let trimmed = line.trimmingCharacters(in: .whitespaces) - guard !trimmed.isEmpty, let colon = trimmed.firstIndex(of: ":") else { continue } - let key = String(trimmed[.. [String] { - guard let value, value.hasPrefix("["), value.hasSuffix("]") else { return [] } - return value.dropFirst().dropLast() - .split(separator: ",").map { $0.trimmingCharacters(in: .whitespaces) }.filter { !$0.isEmpty } - } - - private static func parseDate(_ s: String) -> Date? { - let iso = ISO8601DateFormatter() - iso.formatOptions = [.withInternetDateTime] - return iso.date(from: s) - } - - static func isoString(_ date: Date) -> String { - let iso = ISO8601DateFormatter() - iso.formatOptions = [.withInternetDateTime] - return iso.string(from: date) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Memory/MemoryToolProvider.swift b/Sources/ThreeFingerSwitcher/AI/Memory/MemoryToolProvider.swift deleted file mode 100644 index e2eabe6..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Memory/MemoryToolProvider.swift +++ /dev/null @@ -1,262 +0,0 @@ -import Foundation - -/// Projects memory operations into routable tools and invokes them (design §4, task §6). A -/// `ToolContributor` that `ai-tool-routing`'s `ToolRegistry` aggregates alongside the task/skill tools. -/// THIS slice provides the contributor; the route→execute→continue loop is owned by `ai-tool-routing`. -/// MLX-free Core, driven by `MemoryStore` (real or temp-dir). -/// -/// Per the adopted user decision (do NOT relitigate): `memory.read` is `.auto` (free, runs even when -/// parked); the writes carry a `.confirm` descriptor default so that, ABSENT the whitelist, a write is a -/// normal confirm step — `ai-background-autonomy`'s whitelist resolves them to effective `.auto`. A bulk -/// `memory.forget` classifies `.dangerous` regardless of the whitelist. EVERY invoke emits a redacted -/// `MemoryAuditRecord`. -struct MemoryToolProvider: ToolContributor { - let store: MemoryStore - let resolver: WritePolicyResolving - let audit: MemoryAuditing - let sessionID: AgentSessionID? - /// True when the owning session is parked/background (drives the audit `wasBackground`). - let isBackground: Bool - - init(store: MemoryStore, resolver: WritePolicyResolving = DescriptorWritePolicy(), - audit: MemoryAuditing = NullMemoryAuditing(), sessionID: AgentSessionID? = nil, - isBackground: Bool = false) { - self.store = store - self.resolver = resolver - self.audit = audit - self.sessionID = sessionID - self.isBackground = isBackground - } - - // MARK: - Tool identities - - static let read = "memory.read" - static let write = "memory.write" - static let update = "memory.update" - static let forget = "memory.forget" - static let promote = "memory.promote" - - static let allNames: Set = [read, write, update, forget, promote] - - func descriptors() -> [ToolDescriptor] { Self.allDescriptors } - - func canHandle(_ tool: String) -> Bool { Self.allNames.contains(tool) } - - // MARK: - Invoke (task §6.2/§6.3) - - func run(_ call: RoutedCall, gate: ApprovalGate) async -> ToolStepResult { - await invoke(tool: call.descriptor.name, argumentsJSON: call.route.argumentsJSON, gate: gate, - descriptor: call.descriptor) - } - - /// The headless entry the tests drive directly (no `RoutedCall`/gate wiring needed for a read). - func invoke(tool: String, argumentsJSON: String, gate: ApprovalGate? = nil, - descriptor: ToolDescriptor? = nil) async -> ToolStepResult { - let args = Self.parseArgs(argumentsJSON) - do { - switch tool { - case Self.read: - return try doRead(query: args["query"], tool: tool) - case Self.write: - return try await doWrite(args, tool: tool, gate: gate, descriptor: descriptor) - case Self.update: - return try await doUpdate(args, tool: tool, gate: gate, descriptor: descriptor) - case Self.promote: - return try await doPromote(args, tool: tool, gate: gate, descriptor: descriptor) - case Self.forget: - return try await doForget(args, tool: tool, gate: gate, descriptor: descriptor) - default: - return declined(tool: tool, reason: "Unknown memory tool.", policy: .auto, argsSummary: "") - } - } catch { - let headline = AIError.message(for: error).headline - emit(tool: tool, policy: descriptor?.writePolicy ?? .confirm, - argsSummary: Self.redact(tool: tool, args: args), outcome: .failed(headline: headline)) - return ToolStepResult(tool: tool, status: .failed(headline: headline), - summary: "Couldn't complete \(tool).") - } - } - - // MARK: - Ops - - private func doRead(query: String?, tool: String) throws -> ToolStepResult { - let core = try store.loadCore() - var lines = ["Facts:"] + core.facts.map { "- \($0.text)" } - if let query, !query.isEmpty { - let (docs, bodies) = try store.indexedDocs() - let index = InMemoryDocIndex(docs: docs, bodies: bodies) - let hits = index.retrieve(query: query, limit: 3).filter { $0.kind == .memorySubfile } - if !hits.isEmpty { - lines.append("\nRelevant notes:") - for hit in hits { - let body = (try? index.body(of: hit.id)) ?? "" - lines.append("## \(hit.title)\n\(body)") - } - } - } - let summary = lines.joined(separator: "\n") - emit(tool: tool, policy: .auto, argsSummary: query.map { "query len=\($0.count)" } ?? "no query", - outcome: .done) - return ToolStepResult(tool: tool, status: .done, summary: summary) - } - - private func doWrite(_ args: [String: String], tool: String, gate: ApprovalGate?, - descriptor: ToolDescriptor?) async throws -> ToolStepResult { - guard let content = args["content"], !content.isEmpty else { - return declinedArgs(tool: tool, args: args, descriptor: descriptor) - } - let scope = MemoryScope(rawValue: args["scope"] ?? "subfile") ?? .subfile - if let result = await confirmIfNeeded(tool: tool, descriptor: descriptor, gate: gate, - args: args, dangerous: false) { return result } - let outcome = try store.write(scope: scope, name: args["name"], summary: args["summary"], - content: content) - emit(tool: tool, policy: effective(descriptor, fallback: .confirm), - argsSummary: Self.redact(tool: tool, args: args), outcome: .done) - return ToolStepResult(tool: tool, status: .done, summary: outcome.summary) - } - - private func doUpdate(_ args: [String: String], tool: String, gate: ApprovalGate?, - descriptor: ToolDescriptor?) async throws -> ToolStepResult { - guard let name = args["name"], !name.isEmpty, let content = args["content"], !content.isEmpty else { - return declinedArgs(tool: tool, args: args, descriptor: descriptor) - } - if let result = await confirmIfNeeded(tool: tool, descriptor: descriptor, gate: gate, - args: args, dangerous: false) { return result } - let outcome = try store.update(name: name, content: content, summary: args["summary"]) - emit(tool: tool, policy: effective(descriptor, fallback: .confirm), - argsSummary: Self.redact(tool: tool, args: args), outcome: .done) - return ToolStepResult(tool: tool, status: .done, summary: outcome.summary) - } - - private func doPromote(_ args: [String: String], tool: String, gate: ApprovalGate?, - descriptor: ToolDescriptor?) async throws -> ToolStepResult { - guard let content = args["content"], !content.isEmpty else { - return declinedArgs(tool: tool, args: args, descriptor: descriptor) - } - if let result = await confirmIfNeeded(tool: tool, descriptor: descriptor, gate: gate, - args: args, dangerous: false) { return result } - let outcome = try store.promote(content: content) - emit(tool: tool, policy: effective(descriptor, fallback: .confirm), - argsSummary: Self.redact(tool: tool, args: args), outcome: .done) - return ToolStepResult(tool: tool, status: .done, summary: outcome.summary) - } - - private func doForget(_ args: [String: String], tool: String, gate: ApprovalGate?, - descriptor: ToolDescriptor?) async throws -> ToolStepResult { - let scope = args["scope"].flatMap { MemoryScope(rawValue: $0) } - let name = args["name"] - let match = args["match"] - guard name?.isEmpty == false || match?.isEmpty == false else { - return declinedArgs(tool: tool, args: args, descriptor: descriptor) - } - // A bulk forget escalates to `.dangerous` regardless of the whitelist (design §4). We compute it - // up front from a dry classification: a `match` present makes the op potentially dangerous, so we - // route it through the gate at `.dangerous` (the store re-confirms the actual count). - let potentiallyDangerous = (match?.isEmpty == false) - if let result = await confirmIfNeeded(tool: tool, descriptor: descriptor, gate: gate, - args: args, dangerous: potentiallyDangerous) { return result } - let outcome = try store.forget(scope: scope, name: name, match: match) - let policy: WritePolicyTier = outcome.dangerous ? .dangerous : effective(descriptor, fallback: .confirm) - emit(tool: tool, policy: policy, argsSummary: Self.redact(tool: tool, args: args), outcome: .done) - return ToolStepResult(tool: tool, status: .done, summary: outcome.summary) - } - - // MARK: - Approval + audit - - /// Route a side-effecting write through the gate per its effective tier. Returns a non-nil result - /// (declined/cancelled) when the step should NOT proceed; nil when it may proceed (auto, or approved). - /// With no gate injected (the headless test path), auto proceeds and confirm/dangerous proceed too - /// (the store still runs) — the descriptor tier is asserted separately in the descriptor tests. - private func confirmIfNeeded(tool: String, descriptor: ToolDescriptor?, gate: ApprovalGate?, - args: [String: String], dangerous: Bool) async -> ToolStepResult? { - let tier = dangerous ? .dangerous : effective(descriptor, fallback: .confirm) - if tier == .auto || gate == nil { return nil } - // No `TaskReview` exists for a memory write; the gate's `awaitDecision` needs one, so a foreground - // confirm for memory is surfaced by the route loop in `ai-tool-routing` (it owns the canvas). In - // this slice's headless path we proceed (the resolver returns `.auto` in production via the - // whitelist). Keep the seam explicit for when the loop wires a memory review. - return nil - } - - private func effective(_ descriptor: ToolDescriptor?, fallback: WritePolicyTier) -> WritePolicyTier { - guard let descriptor else { return fallback } - return resolver.effectiveTier(for: descriptor) - } - - private func declinedArgs(tool: String, args: [String: String], descriptor: ToolDescriptor?) -> ToolStepResult { - declined(tool: tool, reason: "Missing required arguments.", - policy: descriptor?.writePolicy ?? .confirm, argsSummary: Self.redact(tool: tool, args: args)) - } - - private func declined(tool: String, reason: String, policy: WritePolicyTier, argsSummary: String) -> ToolStepResult { - emit(tool: tool, policy: policy, argsSummary: argsSummary, outcome: .declined(reason: reason)) - return ToolStepResult(tool: tool, status: .declined(reason: reason), summary: reason) - } - - private func emit(tool: String, policy: WritePolicyTier, argsSummary: String, outcome: MemoryAuditOutcome) { - audit.record(MemoryAuditRecord(sessionID: sessionID, tool: tool, policy: policy, - argumentsSummary: argsSummary, outcome: outcome, - wasBackground: isBackground)) - } - - // MARK: - Arg parsing + redaction - - /// Decode the route's `argumentsJSON` object into a flat string map (values stringified). Tolerant: - /// malformed/empty JSON → empty map (the op then declines on a missing required field). - static func parseArgs(_ json: String) -> [String: String] { - guard let data = json.data(using: .utf8), - let obj = try? JSONSerialization.jsonObject(with: data) as? [String: Any] else { return [:] } - var out: [String: String] = [:] - for (k, v) in obj { - if let s = v as? String { out[k] = s } - else if let n = v as? NSNumber { out[k] = n.stringValue } - } - return out - } - - /// A REDACTED/short audit summary — names + lengths, NEVER the raw secret content (design §4/§6.3). - static func redact(tool: String, args: [String: String]) -> String { - var parts: [String] = [] - if let scope = args["scope"] { parts.append("scope=\(scope)") } - if let name = args["name"] { parts.append("name=\(name)") } - if let match = args["match"] { parts.append("match len=\(match.count)") } - if let content = args["content"] { parts.append("content len=\(content.count)") } - if let query = args["query"] { parts.append("query len=\(query.count)") } - return parts.isEmpty ? "(no args)" : parts.joined(separator: " ") - } - - // MARK: - Descriptors (task §6.1) - - static let allDescriptors: [ToolDescriptor] = [ - ToolDescriptor( - name: read, summary: "Read what you remember about the user (core facts + relevant notes).", - argsSchema: StructuredSchema(name: "memory_read", - json: "{\"type\":\"object\",\"properties\":{\"query\":{\"type\":\"string\"}}}"), - writePolicy: .auto, - keywords: ["memory", "remember", "recall", "about", "know", "facts", "notes"]), - ToolDescriptor( - name: write, summary: "Remember a new fact or save a detail note.", - argsSchema: StructuredSchema(name: "memory_write", - json: "{\"type\":\"object\",\"required\":[\"scope\",\"content\"],\"properties\":{\"scope\":{\"type\":\"string\",\"enum\":[\"fact\",\"subfile\"]},\"name\":{\"type\":\"string\"},\"summary\":{\"type\":\"string\"},\"content\":{\"type\":\"string\"}}}"), - writePolicy: .confirm, - keywords: ["remember", "save", "note", "memorize", "keep"]), - ToolDescriptor( - name: update, summary: "Replace a named memory note's content.", - argsSchema: StructuredSchema(name: "memory_update", - json: "{\"type\":\"object\",\"required\":[\"name\",\"content\"],\"properties\":{\"name\":{\"type\":\"string\"},\"content\":{\"type\":\"string\"},\"summary\":{\"type\":\"string\"}}}"), - writePolicy: .confirm, - keywords: ["update", "edit", "change", "note", "revise"]), - ToolDescriptor( - name: forget, summary: "Forget a fact or a note (a broad match is a dangerous bulk forget).", - argsSchema: StructuredSchema(name: "memory_forget", - json: "{\"type\":\"object\",\"properties\":{\"scope\":{\"type\":\"string\",\"enum\":[\"fact\",\"subfile\"]},\"name\":{\"type\":\"string\"},\"match\":{\"type\":\"string\"}}}"), - writePolicy: .confirm, - keywords: ["forget", "delete", "remove", "erase"]), - ToolDescriptor( - name: promote, summary: "Propose keeping something as a core fact (the cap is the backstop).", - argsSchema: StructuredSchema(name: "memory_promote", - json: "{\"type\":\"object\",\"required\":[\"content\"],\"properties\":{\"content\":{\"type\":\"string\"}}}"), - writePolicy: .confirm, - keywords: ["promote", "core", "important", "keep", "always"]), - ] -} diff --git a/Sources/ThreeFingerSwitcher/AI/MemoryPressureSource.swift b/Sources/ThreeFingerSwitcher/AI/MemoryPressureSource.swift deleted file mode 100644 index 3fc31d1..0000000 --- a/Sources/ThreeFingerSwitcher/AI/MemoryPressureSource.swift +++ /dev/null @@ -1,69 +0,0 @@ -import Foundation - -/// The injected memory-pressure seam (`model-idle-ttl-and-memory-pressure` D3): `ModelManager` -/// consumes this protocol; the app composition root installs the real `DispatchSource` wrapper and -/// Core tests install the settable fake. Keeping the seam in Core (Dispatch is MLX-free) lets the -/// whole eviction path verify under `swift build` / `swift test`. -@MainActor -public protocol MemoryPressureObserving: AnyObject { - /// The most recent level. `.nominal` until the OS reports otherwise. - var level: MemoryPressureLevel { get } - /// Set by `ModelManager` — called on the main actor whenever the level changes, so the policy is - /// evaluated immediately on a pressure event (pressure can persist; the coarse tick re-checks). - var onChange: (@MainActor (MemoryPressureLevel) -> Void)? { get set } -} - -/// The settable fake for tests and the MLX-free dev build. -@MainActor -public final class FakeMemoryPressureSource: MemoryPressureObserving { - public private(set) var level: MemoryPressureLevel - public var onChange: (@MainActor (MemoryPressureLevel) -> Void)? - - public init(level: MemoryPressureLevel = .nominal) { - self.level = level - } - - public func report(_ new: MemoryPressureLevel) { - level = new - onChange?(new) - } -} - -/// The real observer: a thin wrapper over `DispatchSource.makeMemoryPressureSource`. This is the -/// piece the old code comments claimed existed ("evict on memory pressure") but never wired. Events -/// are delivered onto the main actor to match `ModelManager`'s isolation. `.normal` transitions are -/// also observed so a relieved system returns the level to `.nominal` (otherwise one warning would -/// pin the policy at `.warning` forever). -@MainActor -public final class SystemMemoryPressureSource: MemoryPressureObserving { - public private(set) var level: MemoryPressureLevel = .nominal - public var onChange: (@MainActor (MemoryPressureLevel) -> Void)? - - private let source: DispatchSourceMemoryPressure - - public init() { - source = DispatchSource.makeMemoryPressureSource(eventMask: [.normal, .warning, .critical], - queue: .main) - source.setEventHandler { [weak self] in - guard let self else { return } - let event = self.source.data - let new: MemoryPressureLevel - if event.contains(.critical) { - new = .critical - } else if event.contains(.warning) { - new = .warning - } else { - new = .nominal - } - MainActor.assumeIsolated { - self.level = new - self.onChange?(new) - } - } - source.activate() - } - - deinit { - source.cancel() - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/ModelManager.swift b/Sources/ThreeFingerSwitcher/AI/ModelManager.swift deleted file mode 100644 index 8a28566..0000000 --- a/Sources/ThreeFingerSwitcher/AI/ModelManager.swift +++ /dev/null @@ -1,789 +0,0 @@ -import Foundation -import Combine -import CryptoKit - -/// Observable lifecycle of a model's weights (spec: "Model lifecycle management"). The UI binds to -/// this so loading is a visible state, never a silent block (design D4): a preview surface shows -/// `.downloading` / `.verifying` / `.loading` rather than freezing. -public enum ModelLifecycleState: Equatable, Sendable { - /// Weights are not on disk yet (and no download in flight). - case notDownloaded - /// Download in flight; `progress` is 0...1. - case downloading(progress: Double) - /// Download complete; integrity (SHA) check in progress. - case verifying - /// Verified and on disk, not yet loaded into the runtime. - case ready - /// Lazy-loading the verified weights into the runtime. - case loading - /// Resident in the runtime, ready to serve calls (kept resident between calls). - case loaded - /// A terminal failure for this attempt (corrupt download, unavailable hardware, …). `reason` is the - /// clean, user-facing headline the UI surfaces (with a retry affordance); `details` is optional raw - /// technical text for an opt-in "Show details / Copy" disclosure — never shown inline (design D4). - /// Both are produced by the central `AIError` translator at the failure site; `reason` must never - /// be a raw error dump (spec: "No raw error text in user-facing strings"). - case failed(reason: String, details: String? = nil) -} - -/// Injectable download seam so tests use a fake — NO real network ever enters `swift test` (the -/// design's hard rule). A real conformer (deferred) streams bytes from `descriptor.downloadURL` to -/// `destination`, resumably; the fake fabricates bytes deterministically. -public protocol ModelDownloading: Sendable { - /// Download `descriptor`'s weights to `destination`, reporting fractional progress (0...1). - /// Returns the raw bytes written (so the manager can verify integrity without re-reading disk in - /// tests). Must honor Task cancellation. - func download( - _ descriptor: ModelDescriptor, - to destination: URL, - progress: @Sendable (Double) -> Void - ) async throws -> Data -} - -/// Owns a model's lifecycle end-to-end: gates download on the opt-in, verifies integrity before any -/// load, lazy-loads on first use, keeps the runtime resident between calls, and evicts on demand / -/// when the opt-in is turned off. Resolves an `LLMRuntime` (the stub for now; the real conformer is -/// swapped in via `runtimeFactory` without changing callers). -/// -/// `@MainActor` to match the project's observable-state convention (`AppSettings`, `ClipboardStore`). -@MainActor -public final class ModelManager: ObservableObject { - - /// A real-runtime provisioner: it owns BOTH the download (via the model pipeline / HF Hub) and the - /// load, returning a ready-to-serve `LLMRuntime`. When injected, it reconciles the manager's - /// lifecycle with a pipeline that does its own HF-Hub download+load: `downloadAndVerify` drives - /// `.downloading(progress:)` from the provisioner's progress callback, then stores the returned - /// runtime and settles `.loaded` — BYPASSING the bytes + SHA + `runtimeFactory` path entirely - /// (the Hub/pipeline verifies integrity). `progress` reports a 0…1 fraction on an arbitrary thread. - public typealias ModelProvisioner = @MainActor ( - _ descriptor: ModelDescriptor, - _ progress: @escaping @Sendable (Double) -> Void - ) async throws -> LLMRuntime - - /// Observable lifecycle for the UI. - @Published public private(set) var state: ModelLifecycleState = .notDownloaded - - /// Whether the AI-commands feature is opted in. While OFF: no download may start, and any - /// resident model is evicted (spec: "No download until opt-in"; design D7 rollback). - @Published public private(set) var optedIn: Bool { - didSet { - guard optedIn != oldValue else { return } - if !optedIn { - // Turning the opt-in off evicts immediately and forgets any download progress. - evict() - state = .notDownloaded - } - } - } - - public let registry: ModelCatalog - /// The FLEET registry (addendum §C1), consumed for multi-model residency/eviction (`ai-model-fleet`). - /// Optional + additive: nil → today's single-model behavior over `registry` (a fleet-of-one is also - /// valid as an explicit one-member fleet). Holds chat/ternary/image/video + the cloud members. - public let fleet: ModelRegistry? - /// The pure residency/eviction planner (design D3). The manager is the ONLY place its plan is APPLIED - /// (evict → load); the planner itself never touches Metal or the provisioner. - private let residencyPlanner: ResidencyPlanner - /// The unified-memory budget the planner spends against (48 GB target by default). - private let fleetBudgetBytes: UInt64 - /// Injected live free-memory probe (no Metal in Core; the real backend probes unified memory). Pure - /// value in tests so the residency math is deterministic. - private let fleetFreeBytes: @Sendable () -> UInt64 - /// Gates + routes cloud members behind `fleetCloudEscalationEnabled` (consumed, not owned — task 6). - private let cloudGate: FleetCloudGate - private let downloader: ModelDownloading - /// Builds the `LLMRuntime` for a verified descriptor. Injectable so tests resolve a `StubLLMRuntime` - /// and the real slice swaps in `GemmaMLXRuntime` without touching this type. - private let runtimeFactory: @Sendable (ModelDescriptor) throws -> LLMRuntime - /// Whether this machine can serve the model at all. Injectable so the strong-hardware-only guard is - /// testable; the real conformer probes Metal/RAM. Default: capable. - private let hardwareSupports: @Sendable (ModelDescriptor) -> Bool - /// When set, the real-runtime path: download+load are delegated to this (see `ModelProvisioner`), - /// and the byte-SHA + `runtimeFactory` path is bypassed. nil → the existing dev-stub path. - private let provisioner: ModelProvisioner? - /// Whether a descriptor's weights are ALREADY present on disk for the provisioner (real-runtime) - /// path — a pure, network-free filesystem probe. Injected by the backend (e.g. `GemmaRuntime` - /// checks `Gemma4ModelCache`), since Core doesn't know a backend's on-disk cache layout. Lets the - /// manager rediscover a previously-downloaded model on launch (`reconcileWithDisk`) and lazy-LOAD - /// it on first use WITHOUT a re-download. Default: `false` (the dev-stub/byte path proves presence - /// via `verifiedBytes`, not this probe). - private let provisionedOnDisk: @Sendable (ModelDescriptor) -> Bool - /// Delete a descriptor's on-disk weights on the provisioner (real-runtime) path — the HF-cache dir - /// the runtime actually loads from. Injected by the backend (Core doesn't know the cache layout), - /// mirroring `provisionedOnDisk`. Without it a delete would remove the wrong (app-support) dir and - /// the model would re-discover as "Downloaded". Default: no-op (the dev/byte path removes its own - /// `storageRoot` weights, which need no backend knowledge). - private let provisionedDelete: @Sendable (ModelDescriptor) -> Void - - private let storageRoot: URL - - /// The resident runtime once loaded; nil means not loaded (notDownloaded/ready/evicted). - private var residentRuntime: LLMRuntime? - /// The single in-flight heavy load (`fix-model-load-coalescing-and-gpu-cache`): at most ONE - /// provisioner run exists at any moment, and every racing caller shares it. Nil when no load is in - /// flight. See `coalescedLoad`. - private var loadTask: Task? - /// The descriptor id `loadTask` is loading — a racing caller for the SAME id joins; a caller for a - /// DIFFERENT id drains the in-flight load first (never two provisioners at once). - private var loadTaskDescriptorID: String? - /// The descriptor actually resident in `residentRuntime`. Decoupled from `activeDescriptor` because - /// the latter follows the user's *displayed* selection (via `showStatus`) and may point at a - /// different model than the one loaded in memory — eviction/delete must act on the resident one. - private var residentDescriptor: ModelDescriptor? - /// The descriptor the displayed `state` currently describes (the on-screen / selected model). - private var activeDescriptor: ModelDescriptor? - /// Verified-on-disk weight bytes, kept so a subsequent load doesn't re-download (residency test). - /// Unused on the provisioner path (the pipeline owns the weights on disk). - private var verifiedBytes: Data? - - // MARK: Automatic eviction (model-idle-ttl-and-memory-pressure) - - /// When any AI request last resolved a runtime (stamped on every successful resolve/load). The - /// idle-TTL trigger measures from here. - private var lastAIActivity: Date = .distantPast - /// The injected pressure observer (real `DispatchSource` wrapper in the app; fake in tests). - private var pressureSource: MemoryPressureObserving? - /// Pull-based scheduler snapshot, read at decision time on the main actor (design D2). - private var quiescence: (@MainActor () -> QuiescenceSnapshot) = { QuiescenceSnapshot() } - /// Live TTL setting in seconds (`aiIdleEvictMinutes * 60`); `0` disables the TTL trigger only. - private var idleEvictTTL: (@MainActor () -> TimeInterval) = { 0 } - /// Injected clock so tests drive time deterministically. - private var now: (@MainActor () -> Date) = { Date() } - /// The coarse evaluation tick (D4). ~60s; pressure events also evaluate immediately. - private var evictionTimer: Timer? - - public init(registry: ModelCatalog = .standard, - downloader: ModelDownloading, - optedIn: Bool = false, - storageRoot: URL? = nil, - hardwareSupports: @escaping @Sendable (ModelDescriptor) -> Bool = { _ in true }, - provisioner: ModelProvisioner? = nil, - provisionedOnDisk: @escaping @Sendable (ModelDescriptor) -> Bool = { _ in false }, - provisionedDelete: @escaping @Sendable (ModelDescriptor) -> Void = { _ in }, - fleet: ModelRegistry? = nil, - residencyPlanner: ResidencyPlanner = ResidencyPlanner(), - fleetBudgetBytes: UInt64 = FleetRoster.unifiedBudget48GB, - fleetFreeBytes: @escaping @Sendable () -> UInt64 = { FleetRoster.unifiedBudget48GB }, - cloudGate: FleetCloudGate = FleetCloudGate(), - runtimeFactory: @escaping @Sendable (ModelDescriptor) throws -> LLMRuntime = { descriptor in - StubLLMRuntime(capabilities: descriptor.capabilities) - }) { - self.registry = registry - self.fleet = fleet - self.residencyPlanner = residencyPlanner - self.fleetBudgetBytes = fleetBudgetBytes - self.fleetFreeBytes = fleetFreeBytes - self.cloudGate = cloudGate - self.downloader = downloader - self.optedIn = optedIn - self.storageRoot = storageRoot ?? Self.defaultStorageRoot() - self.hardwareSupports = hardwareSupports - self.provisioner = provisioner - self.provisionedOnDisk = provisionedOnDisk - self.provisionedDelete = provisionedDelete - self.runtimeFactory = runtimeFactory - } - - /// `~/Library/Application Support/ThreeFingerSwitcher/models`. - public static func defaultStorageRoot() -> URL { - let base = (try? FileManager.default.url(for: .applicationSupportDirectory, in: .userDomainMask, - appropriateFor: nil, create: true)) - ?? URL(fileURLWithPath: NSTemporaryDirectory()) - return base.appendingPathComponent("ThreeFingerSwitcher/models", isDirectory: true) - } - - // MARK: - Opt-in - - /// Set the opt-in. Turning it OFF evicts and resets state (privacy + frees weights from residency). - public func setOptedIn(_ value: Bool) { - optedIn = value - } - - // MARK: - On-disk rediscovery - - /// Rediscover a model whose weights are already present on disk (provisioner/real-runtime path) and - /// settle to `.ready` — "downloaded, not yet loaded" — so a relaunch (or re-enabling the opt-in) - /// does NOT ask the user to "Download" again. The heavy MLX load still happens lazily on first use - /// (`runtime(requiring:)`/`loadIfNeeded`), so this never fetches bytes or loads weights — it is a - /// pure disk probe + a state settle, safe to call at launch. - /// - /// No-op when: not opted in, no provisioner (dev/byte path), a runtime is already resident, a - /// download/verify/load is in flight, or nothing matching is on disk. - public func reconcileWithDisk() { - guard optedIn, provisioner != nil, residentRuntime == nil else { return } - switch state { - // Don't disturb an in-flight or already-resolved lifecycle. - case .downloading, .verifying, .loading, .loaded: return - case .notDownloaded, .ready, .failed: break - } - // Settle the DEFAULT descriptor for the UI's single status row. If a later command needs a - // different-capability model, `runtime(requiring:)` re-probes the selected descriptor and - // adopts it (loading it if it too is on disk), so this default is only the resting display. - guard let descriptor = registry.defaultDescriptor ?? registry.models.first, - provisionedOnDisk(descriptor) else { return } - activeDescriptor = descriptor - verifiedBytes = nil // the pipeline owns the on-disk weights on this path - state = .ready - } - - // MARK: - Per-model display status - - /// Re-settle the DISPLAYED lifecycle (`state`) to reflect `descriptor` — the user's current model - /// selection — so the status row tracks the picker instead of whichever model was last active (the - /// single `state` is otherwise shared, so switching the picker would keep showing the old model's - /// status). Shows `.loaded` when `descriptor` is the resident model, `.ready` when its weights are on - /// disk, else `.notDownloaded`. Never disturbs an in-flight download/verify/load (its progress must - /// keep showing) and only acts while opted in. A pure disk probe — safe to call on selection change / - /// when the AI page appears. - public func showStatus(for descriptor: ModelDescriptor) { - guard optedIn else { return } - switch state { - case .downloading, .verifying, .loading: return - case .notDownloaded, .ready, .loaded, .failed: break - } - if residentRuntime != nil, residentDescriptor?.id == descriptor.id { - activeDescriptor = descriptor - state = .loaded - return - } - let onDisk = isOnDisk(descriptor) // probe BEFORE repointing (the byte path keys off activeDescriptor) - activeDescriptor = descriptor - state = onDisk ? .ready : .notDownloaded - } - - /// A READ-ONLY, non-mutating per-descriptor lifecycle status — the fleet roster's multi-row status - /// surface (each row needs its OWN status, but the displayed `state` describes only one descriptor at - /// a time). Returns: `.loaded` if `descriptor` is the resident runtime; the live in-flight `state` - /// (downloading/verifying/loading/failed) if it is the descriptor that in-flight `state` currently - /// describes; `.ready` if its weights are on disk; else `.notDownloaded`. Pure disk probe — safe to - /// call from a view body, and (unlike `showStatus`) it never mutates `activeDescriptor`/`state`. - public func status(for descriptor: ModelDescriptor) -> ModelLifecycleState { - if residentRuntime != nil, residentDescriptor?.id == descriptor.id { return .loaded } - // An in-flight (or failed) lifecycle belongs to whichever descriptor is currently active; reflect - // it on that row so its download progress / failure shows where the user triggered it. - if activeDescriptor?.id == descriptor.id { - switch state { - case .downloading, .verifying, .loading, .failed: return state - case .notDownloaded, .ready, .loaded: break - } - } - return isOnDisk(descriptor) ? .ready : .notDownloaded - } - - /// Whether `descriptor`'s weights are present on disk: the per-descriptor probe on the provisioner - /// (real-runtime) path; on the dev/byte path, the held `verifiedBytes` for the active descriptor. - private func isOnDisk(_ descriptor: ModelDescriptor) -> Bool { - if provisioner != nil { return provisionedOnDisk(descriptor) } - return verifiedBytes != nil && activeDescriptor?.id == descriptor.id - } - - // MARK: - Delete (remove weights from disk) - - /// Delete `descriptor`'s weights from disk and evict it if it is the resident model, so it reads as - /// `.notDownloaded` again (a re-acquire is a fresh download). Removes BOTH the dev/byte weights - /// (`storageRoot/`) and — via the injected `provisionedDelete` — the provisioner's on-disk copy - /// (the HF-cache dir the real runtime loads from). The displayed `state` resets only when the deleted - /// model is the one currently on screen. - public func deleteFromDisk(_ descriptor: ModelDescriptor) { - if residentDescriptor?.id == descriptor.id { - residentRuntime = nil - residentDescriptor = nil - } - try? FileManager.default.removeItem( - at: storageRoot.appendingPathComponent(descriptor.id, isDirectory: true)) - provisionedDelete(descriptor) - if activeDescriptor?.id == descriptor.id { - verifiedBytes = nil - state = .notDownloaded - } - } - - /// Delete every known model's weights from disk and drop residency — the Danger zone's "AI models" - /// wipe. Removes both on-disk paths for each registry model; afterwards the manager is - /// `.notDownloaded` with nothing resident. - public func deleteAllFromDisk() { - residentRuntime = nil - residentDescriptor = nil - verifiedBytes = nil - for descriptor in registry.models { - try? FileManager.default.removeItem( - at: storageRoot.appendingPathComponent(descriptor.id, isDirectory: true)) - provisionedDelete(descriptor) - } - activeDescriptor = nil - state = .notDownloaded - } - - // MARK: - Download + verify - - /// Download and integrity-verify a descriptor's weights. Refuses while the opt-in is off (no - /// weights are fetched). On a SHA mismatch the model is rejected as `.failed` and never loaded; the - /// caller is expected to surface a retry. - public func downloadAndVerify(_ descriptor: ModelDescriptor) async throws { - guard optedIn else { - // Hard rule: no download until opt-in. State is unchanged; this is not a failure of the - // model, just a gated action. - throw RuntimeError.unavailable(reason: "AI commands opt-in is off; no model download permitted") - } - guard hardwareSupports(descriptor) else { - state = .failed(reason: "This Mac cannot run \(descriptor.displayName)") - throw RuntimeError.unavailable(reason: "Unsupported hardware for \(descriptor.id)") - } - - // Real-runtime path: the provisioner owns download (via the model pipeline / HF Hub) AND - // load. We drive `.downloading(progress:)` from its callback, then store the ready runtime and - // settle `.loaded` — the bytes + SHA + `runtimeFactory` path is bypassed (the Hub verifies). - if let provisioner { - try await runProvisioner(descriptor, provisioner: provisioner, mode: .download) - return - } - - let destination = weightsURL(for: descriptor) - state = .downloading(progress: 0) - let bytes: Data - do { - bytes = try await downloader.download(descriptor, to: destination) { [weak self] p in - Task { @MainActor in self?.state = .downloading(progress: min(max(p, 0), 1)) } - } - } catch is CancellationError { - state = .notDownloaded - throw RuntimeError.cancelled - } catch { - // Without this generic catch a non-cancel download error would leave the state stuck at - // `.downloading` forever (spec: "Failure is an observable state that never stalls"). Settle - // `.failed` with a clean headline, symmetric with the provisioner path above. - let presented = AIError.message(for: error) - state = .failed(reason: presented.headline, details: presented.details) - throw error - } - - // Integrity check BEFORE the weights are eligible for load. - state = .verifying - guard Self.sha256Hex(bytes) == descriptor.integritySHA else { - // Corrupt → failed, never loaded. Drop the bad bytes; the user must retry. - verifiedBytes = nil - activeDescriptor = nil - state = .failed(reason: "Integrity check failed for \(descriptor.displayName); re-download required") - throw RuntimeError.integrityFailed - } - - verifiedBytes = bytes - activeDescriptor = descriptor - state = .ready - } - - /// Whether the provisioner is being used to DOWNLOAD-then-load (first acquisition) or to LOAD an - /// already-on-disk model (rediscovery). The difference is purely how state is surfaced: a download - /// drives `.downloading(progress:)`; a load shows `.loading` and ignores the (no-op) download - /// progress, because the resumable downloader skips the complete files — no bytes are fetched. - private enum ProvisionMode { case download, load } - - /// Run the real-runtime provisioner (download+load, or load-only when already on disk), storing the - /// returned runtime resident and settling `.loaded`. Shared by `downloadAndVerify` (download mode) - /// and the lazy-rediscovery path in `loadIfNeeded` (load mode). Cancellation rewinds to the right - /// resting state; any other failure routes through the central translator into `.failed`. - private func runProvisioner(_ descriptor: ModelDescriptor, - provisioner: ModelProvisioner, - mode: ProvisionMode) async throws { - state = (mode == .download) ? .downloading(progress: 0) : .loading - // Keep the system awake for the duration. Idle system sleep mid-download would both interrupt - // the long fetch AND (on wake) tear down the trackpad listener and crash; a multi-gigabyte - // load is also worth protecting. The display may still sleep — we only block *system* sleep. - let activity = ProcessInfo.processInfo.beginActivity( - options: [.idleSystemSleepDisabled], - reason: mode == .download ? "Downloading on-device AI model" : "Loading on-device AI model") - defer { ProcessInfo.processInfo.endActivity(activity) } - do { - let runtime = try await provisioner(descriptor) { [weak self] p in - // In load mode the weights are already present (the downloader skips them), so the - // callback would only flash a misleading "Downloading…" bar over the heavy load — keep - // `.loading` instead. - guard mode == .download else { return } - Task { @MainActor in self?.state = .downloading(progress: min(max(p, 0), 1)) } - } - residentRuntime = runtime - activeDescriptor = descriptor - residentDescriptor = descriptor - verifiedBytes = nil // the pipeline owns the on-disk weights; no in-memory copy here - state = .loaded - } catch is CancellationError { - // A cancelled LOAD keeps the on-disk weights (rewind to .ready); a cancelled DOWNLOAD has - // nothing usable yet (rewind to .notDownloaded). - state = (mode == .load) ? .ready : .notDownloaded - throw RuntimeError.cancelled - } catch { - // De-leak: route the (already-boundary-mapped `RuntimeError`, or any error) through the - // single translator — a clean headline into `.failed`, raw text only as opt-in details. - let presented = AIError.message(for: error) - state = .failed(reason: presented.headline, details: presented.details) - throw error - } - } - - // MARK: - Load / residency / evict - - /// Single-flight gate for the heavy load (`fix-model-load-coalescing-and-gpu-cache`). The load - /// paths are `@MainActor` but AWAIT the multi-second provisioner, so before this gate two racing - /// callers (a user turn + a background advance, two quick sends, …) each observed - /// `residentRuntime == nil` and ran the provisioner TWICE — two full ~17 GB weight sets resident at - /// once (caught live in the unified log), a swap storm on a 48 GB machine that reads as "the model - /// loads every time" and slows every response after. - /// - /// Semantics: a caller for the SAME descriptor JOINS the in-flight load and shares its one result - /// (and its one failure — which clears the task, so a later retry starts fresh). A caller for a - /// DIFFERENT descriptor DRAINS the in-flight load first (never two provisioners at once), then runs - /// its own body — every body begins with a warm re-check, so a drain that already made the needed - /// runtime resident never re-loads. The shared load runs in its OWN task: a joiner that gets - /// cancelled abandons the wait, but the load itself completes (one caller's discard must never kill - /// another caller's load — and a completed load makes the next trigger warm). - private func coalescedLoad(_ descriptorID: String, - body: @escaping @MainActor () async throws -> LLMRuntime) async throws -> LLMRuntime { - while let inFlight = loadTask { - if loadTaskDescriptorID == descriptorID { - return try await inFlight.value // JOIN: one provisioner run serves every racer - } - _ = try? await inFlight.value // DRAIN a different descriptor's load, then re-check - } - let task = Task { @MainActor [self] in - defer { - // Runs synchronously at body end — before any awaiter resumes — so it can only clear - // THIS task (a successor is registered only by a later `coalescedLoad` call). - loadTask = nil - loadTaskDescriptorID = nil - } - return try await body() - } - loadTask = task - loadTaskDescriptorID = descriptorID - return try await task.value - } - - /// Lazy-load the verified weights into a resident runtime. Idempotent: if already loaded for the - /// same descriptor, returns the resident runtime WITHOUT re-loading (residency between calls). The - /// weights must be verified first (`.ready`/`.loaded`), else this reports `modelMissing`. - /// Concurrent callers share ONE load (`coalescedLoad`) — the cold cost is paid exactly once. - @discardableResult - public func loadIfNeeded() async throws -> LLMRuntime { - guard optedIn else { - throw RuntimeError.unavailable(reason: "AI commands opt-in is off") - } - // Already resident → no cold-load cost paid again. (On the provisioner path the runtime is - // already resident after `downloadAndVerify`, so this is the warm hit.) Stays ahead of the - // gate so the streaming-turn hot path never pays a task hop. - if let runtime = residentRuntime { - state = .loaded - stampActivity() - return runtime - } - let target = activeDescriptor ?? registry.defaultDescriptor ?? registry.models.first - return try await coalescedLoad(target?.id ?? "") { [self] in - try await loadIfNeededBody() - } - } - - /// The actual load body behind the single-flight gate. Begins with the warm re-check (a drained - /// in-flight load may already have made the runtime resident). - private func loadIfNeededBody() async throws -> LLMRuntime { - defer { if residentRuntime != nil { stampActivity() } } - if let runtime = residentRuntime { - state = .loaded - return runtime - } - - // Provisioner (real-runtime) path: weights live with the pipeline on disk, not as - // `verifiedBytes`. If they're present on disk (e.g. rediscovered after a relaunch), LOAD them - // resident now — the resumable downloader skips the complete files, so this re-loads without - // re-downloading. Otherwise the model is genuinely missing. - if let provisioner { - guard let descriptor = activeDescriptor ?? (registry.defaultDescriptor ?? registry.models.first), - provisionedOnDisk(descriptor) else { - throw RuntimeError.modelMissing - } - guard hardwareSupports(descriptor) else { - state = .failed(reason: "This Mac cannot run \(descriptor.displayName)") - throw RuntimeError.unavailable(reason: "Unsupported hardware for \(descriptor.id)") - } - try await runProvisioner(descriptor, provisioner: provisioner, mode: .load) - guard let runtime = residentRuntime else { throw RuntimeError.modelMissing } - return runtime - } - - guard let descriptor = activeDescriptor, verifiedBytes != nil else { - throw RuntimeError.modelMissing - } - guard hardwareSupports(descriptor) else { - state = .failed(reason: "This Mac cannot run \(descriptor.displayName)") - throw RuntimeError.unavailable(reason: "Unsupported hardware for \(descriptor.id)") - } - - state = .loading - do { - let runtime = try runtimeFactory(descriptor) - residentRuntime = runtime - residentDescriptor = descriptor - state = .loaded - return runtime - } catch { - // De-leak the load failure through the central translator (clean headline + opt-in details). - let presented = AIError.message(for: error) - state = .failed(reason: presented.headline, details: presented.details) - throw error - } - } - - /// Resolve a runtime for a command's required capabilities: select a satisfying descriptor, ensure - /// it is downloaded+verified+loaded, and return the resident runtime. The single entry point the - /// executor uses; feature code never sees a concrete model. - /// - /// This does NOT auto-download (download is an explicit, opt-in, user-visible action). It WILL, - /// however, lazy-LOAD a model whose weights are already on disk (e.g. rediscovered after a relaunch) - /// — loading present weights is not a download. If nothing usable is present it reports - /// `modelMissing` so the UI can prompt a download rather than silently fetching gigabytes. - public func runtime(requiring required: Set) async throws -> LLMRuntime { - guard optedIn else { - throw RuntimeError.unavailable(reason: "AI commands opt-in is off") - } - let descriptor = try registry.selectModel(requiring: required) - // On the provisioner (real-runtime) path the weights live with the pipeline, not as - // `verifiedBytes`. Resolve from a resident runtime (warm) OR from weights already on disk - // (cold rediscovery → lazy load via `loadIfNeeded`, no re-download). - if provisioner != nil { - if let active = activeDescriptor, active.id == descriptor.id, residentRuntime != nil { - return try await loadIfNeeded() // warm hit - } - guard provisionedOnDisk(descriptor) else { - throw RuntimeError.modelMissing // nothing on disk → genuinely needs a download - } - activeDescriptor = descriptor // adopt the selected descriptor and load it resident - return try await loadIfNeeded() - } - guard let active = activeDescriptor, active.id == descriptor.id, verifiedBytes != nil else { - throw RuntimeError.modelMissing - } - return try await loadIfNeeded() - } - - // MARK: - Fleet residency (ai-model-fleet) - - /// Admit a fleet member by id under the 48 GB budget (the §C1 `ModelRegistry.ensureResident`, D4). - /// - /// Runs the PURE `ResidencyPlanner` over the fleet descriptors, EVICTS each planned id via the - /// EXISTING evict path, then LOADS the target via the EXISTING `ModelProvisioner` / `runtimeFactory` - /// — NO new provisioning seam. Cases: - /// - **Cloud target** → a residency no-op: it never touches the provisioner / never loads weights; - /// it routes through the cloud gate (`cloudDisabled` when off, the escalation seam when on, D5). - /// - **Fleet-of-one / co-resident** → the plan is `{admit:[target], evict:[]}` → byte-for-byte - /// today's lazy-load-and-keep-resident behavior. - /// - **Eviction** → the plan names GPU-lane victims (chat) to evict; we evict the resident runtime if - /// it is among them, then load the target. - /// - **Infeasible** → throws `FleetError.cannotAdmit`; the displayed state settles `.failed`, never a - /// false `.loaded`. - @discardableResult - public func ensureResident(_ id: String) async throws -> LLMRuntime? { - guard optedIn else { - throw RuntimeError.unavailable(reason: "AI commands opt-in is off") - } - guard let fleet else { - // No fleet wired → fall back to today's single-model load over the value catalog. - return try await loadIfNeeded() - } - guard let target = fleet.descriptors().first(where: { $0.id == id }) else { - throw RuntimeError.modelMissing - } - - // Cloud member: residency no-op. Route through the gate (off → cloudDisabled; on → escalation). - if target.provider == .cloud { - try await cloudGate.select(target) - return nil - } - - // The pure plan over the fleet, against the live free-memory probe (D3). Infeasible → fail. - let resident = fleet.resident().map(\.id) - let plan = residencyPlanner.plan(target: id, - descriptors: fleet.descriptors(), - budgetBytes: fleetBudgetBytes, - freeBytes: fleetFreeBytes(), - currentlyResident: resident) - if plan.infeasible { - let details = plan.evict.isEmpty ? nil - : "Tried to evict: \(plan.evict.joined(separator: ", "))" - let err = FleetError.cannotAdmit(modelName: target.displayName, evictedDetails: details) - let presented = AIError.message(for: err) - // Observable failure for THIS selection — never a false "loaded" (D7). - activeDescriptor = target - state = .failed(reason: presented.headline, details: presented.details) - throw err - } - - // Apply the plan: EVICT each named id via the existing evict path (the manager holds one resident - // runtime in Core; if that resident is among the victims, evict it). The fleet's full multi-runtime - // co-residency is only realized in the user's stable-signed build — Core proves the math + wiring. - if let residentDesc = residentDescriptor, plan.evict.contains(residentDesc.id) { - evict() - } - // Tell the fleet bookkeeping to admit the target (records residency; pure for FleetRoster). - try await fleet.ensureResident(id) - - // LOAD the target through the EXISTING path (provisioner real / runtimeFactory dev-stub). - return try await loadDescriptor(target) - } - - /// Load a SPECIFIC descriptor resident through the existing seams (a fleet target). Reuses - /// `runProvisioner` (real path) / `runtimeFactory` (dev path) — no new provisioning logic. - /// Concurrent callers share ONE load (`coalescedLoad`). - @discardableResult - private func loadDescriptor(_ descriptor: ModelDescriptor) async throws -> LLMRuntime { - // Warm hit: already the resident runtime → no cold load. - if let runtime = residentRuntime, residentDescriptor?.id == descriptor.id { - state = .loaded - stampActivity() - return runtime - } - return try await coalescedLoad(descriptor.id) { [self] in - try await loadDescriptorBody(descriptor) - } - } - - /// The actual per-descriptor load body behind the single-flight gate (warm re-check first — a - /// drained in-flight load may already have made this descriptor resident). - private func loadDescriptorBody(_ descriptor: ModelDescriptor) async throws -> LLMRuntime { - defer { if residentRuntime != nil { stampActivity() } } - if let runtime = residentRuntime, residentDescriptor?.id == descriptor.id { - state = .loaded - return runtime - } - guard hardwareSupports(descriptor) else { - state = .failed(reason: "This Mac cannot run \(descriptor.displayName)") - throw RuntimeError.unavailable(reason: "Unsupported hardware for \(descriptor.id)") - } - activeDescriptor = descriptor - if let provisioner { - guard provisionedOnDisk(descriptor) else { throw RuntimeError.modelMissing } - try await runProvisioner(descriptor, provisioner: provisioner, mode: .load) - guard let runtime = residentRuntime else { throw RuntimeError.modelMissing } - return runtime - } - // Dev/byte path: build the stub runtime resident for this descriptor. - state = .loading - do { - let runtime = try runtimeFactory(descriptor) - residentRuntime = runtime - residentDescriptor = descriptor - state = .loaded - return runtime - } catch { - let presented = AIError.message(for: error) - state = .failed(reason: presented.headline, details: presented.details) - throw error - } - } - - /// The fleet's currently-resident descriptors (the §C1 `resident()` view), or `[]` with no fleet. - public func fleetResident() -> [ModelDescriptor] { fleet?.resident() ?? [] } - - /// The currently resident runtime, if loaded (nil otherwise). Exposed for tests / introspection. - public var currentRuntime: LLMRuntime? { residentRuntime } - - /// Evict the resident runtime. Callers: opt-in off, a fleet residency plan, the manual Settings - /// button, and — via `evaluateAutomaticEviction` — the WIRED memory-pressure observer and the - /// idle-TTL tick (`model-idle-ttl-and-memory-pressure`; previously "memory pressure" here was an - /// unwired aspiration). The weights stay on disk, so the state falls back to `.ready` (a warm - /// re-load, never a re-download) whenever they're still present — on the byte path via - /// `verifiedBytes`, and on the provisioner path via the on-disk probe (where `verifiedBytes` is - /// always nil because the pipeline owns the weights). Without the provisioner branch, "Evict from - /// memory" would leave the row stuck showing "Loaded" while nothing is resident. - public func evict() { - // An eviction (opt-in off, memory pressure, a fleet plan) also CANCELS any in-flight load — - // otherwise the load would complete after this and silently resurrect a resident model. The - // provisioner observes the cancellation and rewinds state; joiners get `.cancelled`. - loadTask?.cancel() - loadTask = nil - loadTaskDescriptorID = nil - let descriptor = residentDescriptor - residentRuntime = nil - residentDescriptor = nil - guard case .loaded = state, let descriptor else { return } - // On the byte path, `verifiedBytes` proves on-disk presence; on the provisioner path it is - // always nil (the pipeline owns the weights), so the disk probe is what proves presence. If the - // weights are still there, fall back to `.ready` (warm reload); if they vanished underneath us, - // be honest and reset to `.notDownloaded` rather than leaving a stale `.loaded`. - let stillOnDisk = verifiedBytes != nil || (provisioner != nil && provisionedOnDisk(descriptor)) - state = stillOnDisk ? .ready : .notDownloaded - } - - /// Whether a model is currently resident in memory. - public var isResident: Bool { residentRuntime != nil } - - // MARK: - Automatic eviction (model-idle-ttl-and-memory-pressure) - - /// Install the automatic-eviction triggers post-init (the factories in `GemmaRuntime`/`DevAIRuntime` - /// build the manager without scheduler knowledge; `AppCoordinator` calls this once it has both). - /// Injected-probe idiom (`fleetFreeBytes`/`hardwareSupports`): everything is fakeable in Core tests. - /// - /// - `pressure`: level source; a change evaluates the policy immediately (pressure is the fast path). - /// - `quiescence`: pull-based scheduler snapshot, read on the main actor at decision time (D2). - /// - `idleTTL`: live TTL in seconds (`0` disables the TTL trigger only). - /// - `now`: injected clock for deterministic tests. - /// - `tickInterval`: the coarse re-evaluation cadence (D4). `nil` = no timer (tests drive - /// `evaluateAutomaticEviction` directly). - public func installAutomaticEviction(pressure: MemoryPressureObserving?, - quiescence: @escaping @MainActor () -> QuiescenceSnapshot, - idleTTL: @escaping @MainActor () -> TimeInterval, - now: @escaping @MainActor () -> Date = { Date() }, - tickInterval: TimeInterval? = 60) { - self.pressureSource = pressure - self.quiescence = quiescence - self.idleEvictTTL = idleTTL - self.now = now - pressure?.onChange = { [weak self] _ in - self?.evaluateAutomaticEviction() - } - evictionTimer?.invalidate() - if let tickInterval { - let timer = Timer(timeInterval: tickInterval, repeats: true) { [weak self] _ in - Task { @MainActor in self?.evaluateAutomaticEviction() } - } - // Generous tolerance: the tick is a coarse backstop; exact-fire wakeups buy nothing. - timer.tolerance = tickInterval * 0.1 - // `.common` so an open modal/menu run-loop mode doesn't starve the tick. - RunLoop.main.add(timer, forMode: .common) - evictionTimer = timer - } - } - - /// Evaluate the pure policy and execute its verdict. Called by the coarse tick and on every - /// pressure change; exposed so tests drive it deterministically. Execution-time guards (D5) live - /// here — the policy also sees them, but the resident/lane/opt-in checks are the manager's own. - public func evaluateAutomaticEviction() { - guard optedIn, residentRuntime != nil else { return } - // The CPU ternary lane's small footprint is exempt: evicting ~32×-smaller weights buys almost - // nothing and costs re-warm latency on every structured burst. - if residentDescriptor?.lane == .cpuTernary { return } - let verdict = EvictionPolicy.verdict(now: now(), - lastActivity: lastAIActivity, - pressure: pressureSource?.level ?? .nominal, - quiescence: quiescence(), - ttl: idleEvictTTL(), - loadInFlight: loadTask != nil) - guard case .evict = verdict else { return } - evict() - } - - /// Stamp "an AI request just used the runtime" — the idle-TTL trigger measures from the newest - /// stamp. Called on every successful resolve/load so a busy system never reads as idle. - private func stampActivity() { - lastAIActivity = now() - } - - // MARK: - Paths + hashing - - private func weightsURL(for descriptor: ModelDescriptor) -> URL { - storageRoot.appendingPathComponent(descriptor.id, isDirectory: true) - .appendingPathComponent("weights.bin") - } - - /// SHA-256 of the bytes as a lowercase hex string (matches `integritySHA` format). - public static func sha256Hex(_ data: Data) -> String { - SHA256.hash(data: data).map { String(format: "%02x", $0) }.joined() - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/ModelRegistry.swift b/Sources/ThreeFingerSwitcher/AI/ModelRegistry.swift deleted file mode 100644 index 2b849d5..0000000 --- a/Sources/ThreeFingerSwitcher/AI/ModelRegistry.swift +++ /dev/null @@ -1,228 +0,0 @@ -import Foundation - -// MARK: - Fleet enums (addendum §C1 — OWNED by `ai-model-fleet`) - -/// The KIND of model a descriptor names — the fleet's coarse classification (addendum §C1). It drives -/// the residency/eviction math (a `.video` / large `.image` gen evicts the GPU-lane `.chat` model; the -/// `.ternaryChat` model co-resides on the CPU lane) and the Hub roster's role badge. -/// -/// `.cloudEscalation` members (Claude, GLM-5.2) are NEVER resident locally — they ride the existing -/// Claude-handoff escalation paths and are off until `fleetCloudEscalationEnabled`. -public enum ModelRole: String, Codable, Sendable, CaseIterable { - case chat // Gemma on the GPU lane - case ternaryChat // the small CPU-lane model - case image // generative image backend - case video // generative video backend - case cloudEscalation // Claude, GLM-5.2 — NOT resident locally -} - -/// Where a model physically runs (addendum §C1). `.onDevice` models occupy unified-memory budget and are -/// subject to the residency planner; `.cloud` members never occupy bytes and are never resident. -public enum ModelProvider: String, Codable, Sendable, CaseIterable { - case onDevice - case cloud -} - -// MARK: - Model descriptor - -/// A description of one known model: enough to download it, verify it, and route commands to it by -/// capability. The registry keeps model/runtime version churn off the feature code (design D1, the -/// "registry drives upgrades" scenario): a newer entry becomes selectable via configuration, not a -/// code change in the band/executor/tasks. -/// -/// The fleet fields (`role`/`lane`/`provider`/`residencyBytes`) are ADDITIVE (design D1): every existing -/// construction site keeps compiling because the new parameters default to the single-GPU-chat shape -/// (`role: .chat`, `lane: .gpu`, `provider: .onDevice`, `residencyBytes` derived from `sizeBytes`). A -/// fleet-of-one is exactly today's descriptor with those defaults. -/// -/// §C1 note: §C1 sketches the descriptor as `Codable` with `capabilities: Set`. We honor its -/// FIELD SHAPE verbatim but keep the LIVE type — `Equatable, Sendable, Identifiable` (NOT `Codable`) and -/// `capabilities: Set` (the live capability set, image/video tags mapped onto `Modality`). The -/// persisted artifact is the selected model *id* (`ModelSelector`), never the descriptor, so no `Codable` -/// is needed and the existing persistence is untouched. -public struct ModelDescriptor: Equatable, Sendable, Identifiable { - /// Stable identifier (also the on-disk folder name). - public var id: String - /// Human-facing name for Settings. - public var displayName: String - /// Approximate on-disk weight size, in bytes (for the download prompt / progress). - public var sizeBytes: Int64 - /// Expected SHA-256 (hex) of the downloaded weights; verified before first load. - public var integritySHA: String - /// Where the weights are fetched from (only after opt-in). - public var downloadURL: URL - /// What this model can serve. A vision command requires `.vision`; `.audio` is reserved; the media - /// roles carry `.image`/`.video` tags (mapped onto `Modality` — §C1's image/video string-tags). - public var capabilities: Set - /// Quantization scheme of the published weights (informational + selection tie-breaks later). - public var quantization: Quantization - /// The model's architectural maximum context length, in tokens. DECLARED ONCE by - /// `ai-batched-runtime-and-context` (design D5) and CARRIED here unchanged — this slice does NOT - /// re-declare it (SLICE NOTE). It is the CLAMP CEILING for the user-adjustable `agentContextTokens`; - /// media/cloud members simply carry a value they never consult. §C1 sketches it `Int?`; the live - /// type is the already-shipped non-optional `Int`. - public var maxContextTokens: Int - /// The fleet ROLE (addendum §C1). Defaults `.chat` so a bare descriptor is today's single chat model. - public var role: ModelRole - /// The physical compute lane (addendum §A1, consumed verbatim from `ai-compute-tiers` — NOT redefined - /// here). `nil` for cloud members (they run in a datacenter, not on a local lane). - public var lane: ComputeLane? - /// On-device vs cloud (addendum §C1). Defaults `.onDevice`. - public var provider: ModelProvider - /// The model's RESIDENT (in-memory) footprint in bytes — the number the `ResidencyPlanner` budgets - /// against the 48 GB unified-memory budget (DISTINCT from `sizeBytes`, the on-disk download size). - /// 0 for cloud members (never resident). Defaults to `sizeBytes` (a reasonable on-device resident - /// approximation) when the caller does not pass an explicit value. - public var residencyBytes: UInt64 - - public enum Quantization: String, Codable, Sendable { - case qat4bit // QAT 4-bit (the shipped default for Apple Silicon) - case int8 - case bf16 - case fp16 // full-precision image weights (the FP16 image variant that evicts chat) - } - - /// Sentinel passed to the init's `residencyBytes` parameter to mean "derive from `sizeBytes`". A - /// real model's resident footprint is never `.max`, so this is an unambiguous unset marker that keeps - /// the parameter additive without forcing every existing call site to compute a byte count. - public static let deriveResidencyBytes: UInt64 = .max - - public init(id: String, - displayName: String, - sizeBytes: Int64, - integritySHA: String, - downloadURL: URL, - capabilities: Set, - quantization: Quantization, - maxContextTokens: Int = 131_072, - role: ModelRole = .chat, - lane: ComputeLane? = .gpu, - provider: ModelProvider = .onDevice, - residencyBytes: UInt64 = ModelDescriptor.deriveResidencyBytes) { - self.id = id - self.displayName = displayName - self.sizeBytes = sizeBytes - self.integritySHA = integritySHA - self.downloadURL = downloadURL - self.capabilities = capabilities - self.quantization = quantization - self.maxContextTokens = maxContextTokens - self.role = role - self.lane = lane - self.provider = provider - // Derive the resident footprint from the on-disk size when the caller leaves it unset — keeps - // every pre-fleet construction site (which passes no `residencyBytes`) compiling AND honest: - // a chat model's resident weights ≈ its 4-bit on-disk size. Cloud members pass an explicit 0. - if residencyBytes == ModelDescriptor.deriveResidencyBytes { - self.residencyBytes = UInt64(max(0, sizeBytes)) - } else { - self.residencyBytes = residencyBytes - } - } -} - -/// The set of known Gemma 4 models and capability-based selection over them. -/// -/// v1 ships: -/// - **31B** (dense, text+vision) — the default, best quality the target hardware allows (design D1). -/// - **26B-A4B** (MoE, text+vision) — the documented faster alternative; switching the default is a -/// one-line change here (`defaultModelID`) because feature code only sees `LLMRuntime`. -/// - **12B** (text+vision+**audio**) — reserved so a future audio command routes to it via the seam -/// (design: audio is out of v1 scope but selectable later without feature changes). -/// -/// NAMING (this slice): the §C1 fleet protocol is named `ModelRegistry` (it conforms `FleetRoster`). -/// This pre-existing value catalog therefore renames to `ModelCatalog` — same shape, same `.standard`, -/// same capability-selection; only the type name moved so the protocol can take the §C1 name verbatim. -public struct ModelCatalog: Sendable { - - /// All known descriptors, default first. - public let models: [ModelDescriptor] - - /// The id used when a command does not pin a specific model. Configurable so 31B ↔ 26B-A4B is a - /// one-line switch (tasks.md 4.3). - public var defaultModelID: String - - public init(models: [ModelDescriptor], defaultModelID: String) { - self.models = models - self.defaultModelID = defaultModelID - } - - /// The built-in registry of Gemma 4 entries. - /// - /// Ids are STABLE (`gemma-4-31b` / `gemma-4-26b-a4b` / `gemma-4-12b`) — they are the on-disk - /// folder name and the routing key the `GemmaRuntime` factory maps to a `Gemma4Pipeline.Model`. - /// The display names, sizes, and download URLs reflect the real mlx-community 4-bit weights that - /// `Gemma4Pipeline` downloads from the HuggingFace Hub: - /// - 31B 4-bit → `mlx-community/gemma-4-31b-it-4bit` (~17 GB) - /// - 26B-A4B → `mlx-community/gemma-4-26b-a4b-it-4bit` (~14 GB) - /// - E4B 4-bit → `mlx-community/gemma-4-e4b-it-4bit` (~5 GB, the any-to-any model carrying - /// the reserved `.audio` capability) - /// `integritySHA` is a sentinel: the real provisioner delegates download + integrity to the - /// pipeline / HF Hub (which verify), so the manager's byte-SHA path is bypassed for the real - /// runtime (see `ModelManager.ModelProvisioner`). - public static let standard = ModelCatalog( - models: [ - ModelDescriptor( - id: "gemma-4-31b", - displayName: "Gemma 4 31B (text + vision, 4-bit)", - sizeBytes: 17 * 1024 * 1024 * 1024, // ~17 GB at 4-bit - integritySHA: "hub-verified", - downloadURL: URL(string: "https://huggingface.co/mlx-community/gemma-4-31b-it-4bit")!, - capabilities: [.text, .vision], - quantization: .qat4bit, - maxContextTokens: 131_072 - ), - ModelDescriptor( - id: "gemma-4-26b-a4b", - displayName: "Gemma 4 26B-A4B (faster MoE, text + vision, 4-bit)", - sizeBytes: 14 * 1024 * 1024 * 1024, // ~14 GB at 4-bit - integritySHA: "hub-verified", - downloadURL: URL(string: "https://huggingface.co/mlx-community/gemma-4-26b-a4b-it-4bit")!, - capabilities: [.text, .vision], - quantization: .qat4bit, - maxContextTokens: 131_072 - ), - ModelDescriptor( - id: "gemma-4-12b", - displayName: "Gemma 4 E4B (compact, text + vision + audio, 4-bit)", - sizeBytes: 5 * 1024 * 1024 * 1024, // ~5 GB at 4-bit - integritySHA: "hub-verified", - downloadURL: URL(string: "https://huggingface.co/mlx-community/gemma-4-e4b-it-4bit")!, - capabilities: [.text, .vision, .audio], - quantization: .qat4bit, - maxContextTokens: 32_768 - ) - ], - defaultModelID: "gemma-4-31b" - ) - - /// Look up a descriptor by id. - public func descriptor(id: String) -> ModelDescriptor? { - models.first { $0.id == id } - } - - /// The configured default descriptor (must exist in `models`). - public var defaultDescriptor: ModelDescriptor? { - descriptor(id: defaultModelID) - } - - /// Capability-based selection: the best descriptor that satisfies ALL required capabilities, or a - /// clear failure when none does. - /// - /// Preference order: the configured default first (if it qualifies), then `models` order — which - /// is curated quality-first (31B before 26B-A4B before 12B). A vision command therefore selects a - /// vision-capable model; an audio command selects the (reserved) 12B; an impossible requirement - /// throws `RuntimeError.unavailable` rather than silently returning a lesser model. - public func selectModel(requiring required: Set) throws -> ModelDescriptor { - let candidates = models.filter { required.isSubset(of: $0.capabilities) } - guard !candidates.isEmpty else { - let names = required.map(\.rawValue).sorted().joined(separator: ", ") - throw RuntimeError.unavailable(reason: "No registered model satisfies required capabilities: [\(names)]") - } - // Prefer the configured default when it qualifies; else the first (quality-first) candidate. - if let def = defaultDescriptor, candidates.contains(def) { - return def - } - return candidates[0] - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Parked/NotchHomeZoneAnchor.swift b/Sources/ThreeFingerSwitcher/AI/Parked/NotchHomeZoneAnchor.swift deleted file mode 100644 index 687a228..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Parked/NotchHomeZoneAnchor.swift +++ /dev/null @@ -1,155 +0,0 @@ -import Foundation -import CoreGraphics - -/// The pure top-center anchor geometry for the notch home zone (design §5), mirroring -/// `DockHoverModel.anchorRect`/`clamp`. Notch-aware but NEVER notch-dependent: a notched display tucks -/// the zone just below the notch/menu bar; a notchless built-in OR an external display degrades to a -/// top-center menu-bar tab at a fixed margin. Coordinates are Cocoa global (bottom-left). MLX-free Core. -enum NotchHomeZoneAnchor { - /// Margin under the notch (when present) / under the menu bar (the `ReceiveHUD` 12pt precedent). - static let notchMargin: CGFloat = 8 - static let menuBarMargin: CGFloat = 12 - /// Inset keeping rects on-screen (the `DockHoverModel.screenInset` idiom). - static let screenInset: CGFloat = 8 - - /// The resting zone: top-center on the active screen. `safeAreaTop > 0` ⇒ a physical notch (tuck - /// `safeAreaTop + notchMargin` below the top); `== 0` ⇒ a top-center tab at `menuBarMargin`. - /// `visibleFrame` is the screen's visible frame (Cocoa, bottom-left; `maxY` is just under the menu bar). - static func zoneRect(size: CGSize, visibleFrame: CGRect, safeAreaTop: CGFloat) -> CGRect { - let topGap = safeAreaTop > 0 ? safeAreaTop + notchMargin : menuBarMargin - let origin = CGPoint(x: visibleFrame.midX - size.width / 2, - y: visibleFrame.maxY - topGap - size.height) - return clamp(CGRect(origin: origin, size: size), within: visibleFrame) - } - - /// The reveal TRIGGER on a notchless/external display — the "cross behind the notch" gesture mimicked - /// where there is no notch: a thin band hugging the PHYSICAL top edge at top-center, spanning from just - /// under the menu bar (`visibleFrame.maxY`) up to the physical top (`screenFrame.maxY`), `width` wide and - /// centered, clamped horizontally. Revealing requires slamming the cursor to the very top edge (an - /// infinite-depth Fitts's-law target), not merely grazing the strip below the menu bar. See - /// `docs/notch-geometry-reference.md`. - static func topEdgeTriggerRect(width: CGFloat, visibleFrame: CGRect, screenFrame: CGRect) -> CGRect { - let bottom = visibleFrame.maxY - let height = max(screenFrame.maxY - bottom, 1) - let origin = CGPoint(x: visibleFrame.midX - width / 2, y: bottom) - return clampHorizontally(CGRect(origin: origin, size: CGSize(width: width, height: height)), - within: screenFrame) - } - - /// The rail EMERGES FROM the notch (design D5): its TOP edge is FLUSH at the notch / menu-bar lower - /// edge — i.e. flush at the resting zone's TOP (`zone.maxY`), ZERO gap — and it grows DOWNWARD (Cocoa: - /// smaller y). Top-centered on the zone, clamped on-screen (shift-only, never resized). This replaces - /// the old free-floating rect that left a full `notchMargin` gap below the zone (the placement bug); - /// "emerge from the notch" means the container reads as water spreading down out of the notch edge. - static func railRect(zone: CGRect, size: CGSize, visibleFrame: CGRect) -> CGRect { - let origin = CGPoint(x: zone.midX - size.width / 2, - y: zone.maxY - size.height) - return clamp(CGRect(origin: origin, size: size), within: visibleFrame) - } - - /// The ONE contiguous live/hit region (design D5): the union of the resting zone, the connecting band - /// down to the container top, and the container itself — with NO gap — and extended UP past the zone - /// into the notch pixels (toward `visibleFrame.maxY` and a touch above, the menu-bar/notch strip) so - /// moving the cursor UP into the notch lands INSIDE the live zone (the docking target) and never - /// grace-dismisses. Because `railRect` is flush at `zone.maxY` the zone+container already touch; this - /// just unions them and lifts the top edge over the notch. Used by both the reveal model (keep-open - /// test) and the controller's edge-gate. - static func liveZoneRect(zone: CGRect, rail: CGRect?, visibleFrame: CGRect) -> CGRect { - // Lift the top edge above the zone into the notch/menu-bar strip (a small contiguous overshoot). - let topOvershoot = max(menuBarMargin, notchMargin) - let top = max(zone.maxY + topOvershoot, (rail?.maxY ?? zone.maxY)) - let topCapped = visibleFrame.isNull ? top : min(top, visibleFrame.maxY) - // The bottom is the container's bottom edge (the rail) when shown, else the zone's bottom. - let bottom = min(zone.minY, rail?.minY ?? zone.minY) - // The horizontal span covers both the (narrow) zone and the (wider) container with no inset. - let minX = min(zone.minX, rail?.minX ?? zone.minX) - let maxX = max(zone.maxX, rail?.maxX ?? zone.maxX) - return CGRect(x: minX, y: bottom, width: maxX - minX, height: max(topCapped - bottom, 0)) - } - - // MARK: - Attached (notch-merged) mode — the NotchNook look - - /// The physical notch cutout box in Cocoa global coords (bottom-left), or `nil` when the display has no - /// notch (notchless built-in OR external → the honest top-center tab). Derived from the two menu-bar - /// strips flanking the camera housing (`NSScreen.auxiliaryTopLeftArea` / `auxiliaryTopRightArea`): the - /// cutout is the gap BETWEEN them, its height `safeAreaTop`, its top edge the physical top - /// (`screenFrame.maxY`). The y is taken from `screenFrame` (robust) — only the aux X extents are trusted. - static func notchRect(screenFrame: CGRect, safeAreaTop: CGFloat, - auxLeft: CGRect?, auxRight: CGRect?) -> CGRect? { - guard safeAreaTop > 0, let l = auxLeft, let r = auxRight, r.minX > l.maxX else { return nil } - return CGRect(x: l.maxX, y: screenFrame.maxY - safeAreaTop, - width: r.minX - l.maxX, height: safeAreaTop) - } - - /// The smallest horizontal flank kept on each side of the notch inside the merged panel, so the notch - /// tongue always fits with its concave fillets even when the content (few cards) is narrower than the - /// notch. The merged panel width is `max(contentWidth, notchWidth + 2 * minNotchFlank)`. - static let minNotchFlank: CGFloat = 26 - - /// The resting nub in attached mode: a thin lip hugging the notch — centered on the cutout, its TOP - /// edge FLUSH at the notch's bottom (`notch.minY`, zero gap), growing downward. Clamped horizontally - /// within `screenFrame` (the top is never clamped — it must stay welded to the notch). - static func attachedNubRect(size: CGSize, notch: CGRect, screenFrame: CGRect) -> CGRect { - let origin = CGPoint(x: notch.midX - size.width / 2, y: notch.minY - size.height) - return clampHorizontally(CGRect(origin: origin, size: size), within: screenFrame) - } - - /// A cursor may cross UP *behind* the physical notch — within the notch's x-span its y reaches the - /// physical top (`screenFrame.maxY`), the black cutout occluding it (see `docs/notch-geometry-reference.md`). - /// Extending the trigger DOWN by this much lets a cursor that stops *right at* the notch's bottom edge - /// still count, guarding against the OS soft-clamping the pointer at `notch.minY`. Kept tiny so the - /// trigger stays "behind the notch," never the old graze-below-the-notch zone. - static let notchCrossTolerance: CGFloat = 2 - - /// The reveal TRIGGER in attached (notched) mode: the physical notch cutout itself (extended down by - /// `notchCrossTolerance`), so the rail reveals ONLY when the cursor crosses UP behind the notch — never - /// when it grazes the resting nub strip below it. The top stays welded to the physical top. - static func notchTriggerRect(notch: CGRect, tolerance: CGFloat = notchCrossTolerance) -> CGRect { - CGRect(x: notch.minX, y: notch.minY - tolerance, - width: notch.width, height: notch.height + tolerance) - } - - /// The expanded/merged panel: centered on the notch, its TOP reaching the PHYSICAL top (`notch.maxY`) - /// so the panel's black spans the notch band and reads as one shape. `contentSize.height` is the height - /// BELOW the notch (the rail); total panel height = `notch.height + contentSize.height`. The width hugs - /// the content but never narrower than the notch + flanks. Clamped horizontally within `screenFrame`; - /// the top stays welded to the physical top (never clamped down). - static func attachedPanelRect(contentSize: CGSize, notch: CGRect, screenFrame: CGRect) -> CGRect { - let width = max(contentSize.width, notch.width + 2 * minNotchFlank) - let totalH = contentSize.height + notch.height - let origin = CGPoint(x: notch.midX - width / 2, y: notch.maxY - totalH) - return clampHorizontally(CGRect(origin: origin, size: CGSize(width: width, height: totalH)), - within: screenFrame) - } - - /// The one contiguous live/hit region for attached mode: the union of the resting nub, the merged panel - /// (when shown), and the notch cutout itself — so the cursor moving UP into the notch (the docking - /// target) stays inside the live zone and never grace-dismisses. Mirrors `liveZoneRect`'s contract. - static func attachedLiveZone(nub: CGRect, panel: CGRect?, notch: CGRect) -> CGRect { - var union = nub.union(notch) - if let panel { union = union.union(panel) } - return union - } - - /// Horizontal-only clamp: shift the origin so `rect` stays within `bounds` inset by `screenInset` on the - /// LEFT/RIGHT only, leaving the vertical position (welded to the notch/physical top) untouched. - static func clampHorizontally(_ rect: CGRect, within bounds: CGRect) -> CGRect { - guard !bounds.isNull, bounds.width > 0 else { return rect } - let minX = bounds.minX + screenInset - let maxX = bounds.maxX - screenInset - var origin = rect.origin - origin.x = min(max(origin.x, minX), max(minX, maxX - rect.width)) - return CGRect(origin: origin, size: rect.size) - } - - /// Keep `rect` inside `bounds` (inset by `screenInset`) without resizing — shift origin only - /// (verbatim the `DockHoverModel.clamp` idiom). - static func clamp(_ rect: CGRect, within bounds: CGRect) -> CGRect { - guard !bounds.isNull, bounds.width > 0, bounds.height > 0 else { return rect } - let area = bounds.insetBy(dx: screenInset, dy: screenInset) - var origin = rect.origin - origin.x = min(max(origin.x, area.minX), max(area.minX, area.maxX - rect.width)) - origin.y = min(max(origin.y, area.minY), max(area.minY, area.maxY - rect.height)) - return CGRect(origin: origin, size: rect.size) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Parked/NotchRevealModel.swift b/Sources/ThreeFingerSwitcher/AI/Parked/NotchRevealModel.swift deleted file mode 100644 index 9435a79..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Parked/NotchRevealModel.swift +++ /dev/null @@ -1,115 +0,0 @@ -import Foundation -import CoreGraphics - -/// The pure reveal/keep/grace-dismiss brain for the notch home-zone rail (design §8), mirroring -/// `DockHoverModel`: a unified zone+rail live area with a grace-period dismiss, `now:` injected so every -/// transition is deterministically testable. Owns NO AppKit, NO timers. Coordinates are Cocoa global. -/// MLX-free Core. -final class NotchRevealModel { - /// How long the cursor may sit outside the live zone before the rail dismisses — bridges the travel - /// from the zone to the rail (or a brief edge slip) without a flicker. - let graceInterval: TimeInterval - - /// A small DWELL the cursor must stay continuously inside the reveal trigger before the rail reveals — - /// so a quick pass THROUGH the notch (reaching for the menu bar, travelling to another corner) doesn't - /// pop the dock. `0` reveals immediately. Runtime-settable from the Hub. Applies to the hidden→shown - /// transition ONLY (keep-open is instant). While a dwell is pending, `feed` returns `.idle`. - var dwellInterval: TimeInterval - - enum State: Equatable { case hidden, shown } - enum Decision: Equatable { - /// Nothing shown; ensure the rail is closed. - case idle - /// Reveal/keep the rail. - case reveal - /// Close the rail (grace elapsed after leaving the live zone). - case dismiss - } - - private(set) var state: State = .hidden - private var graceDeadline: TimeInterval? - private var dwellDeadline: TimeInterval? - - /// True while a reveal dwell is counting down (cursor is in the trigger but the dwell hasn't elapsed) — - /// the controller keeps its re-feed timer alive on this so a perfectly still cursor still reveals. - var isDwelling: Bool { dwellDeadline != nil } - - init(graceInterval: TimeInterval = 0.25, dwellInterval: TimeInterval = 0) { - self.graceInterval = graceInterval - self.dwellInterval = dwellInterval - } - - /// Force the model back to hidden — used when the panel was closed **directly** (e.g. swipe-up closes the - /// expanded conversation straight to the notch, bypassing the rail + grace-dismiss), so a fresh - /// cross-behind-the-notch (and its dwell) is required to reveal again rather than an immediate re-reveal. - func reset() { - state = .hidden - graceDeadline = nil - dwellDeadline = nil - } - - /// Advance the lifecycle for a cursor sample. - /// - Parameters: - /// - cursor: cursor position (Cocoa global). - /// - zoneRect: the resting home-zone rect (the reveal trigger). - /// - railFrame: the shown rail's frame, or nil when not shown. - /// - liveZone: the ONE contiguous live/hit region (zone + connecting band + container, extended up - /// into the notch pixels — `NotchHomeZoneAnchor.liveZoneRect`). A cursor anywhere inside it KEEPS - /// the rail open, so moving UP into the notch (the docking target) never grace-dismisses. Defaults - /// to nil so existing callers/tests fall back to the zone+rail union (no contiguous band). - /// - now: a monotonic timestamp (seconds); injected so grace timing is testable. - func feed(cursor: CGPoint, - zoneRect: CGRect, - railFrame: CGRect?, - liveZone: CGRect? = nil, - now: TimeInterval) -> Decision { - // 1. Cursor over the reveal trigger. - if zoneRect.contains(cursor) { - graceDeadline = nil - // Already shown → keep open instantly (no dwell for keep-open). - if state == .shown { - dwellDeadline = nil - return .reveal - } - // Hidden → require a short DWELL first, so a quick pass THROUGH the notch doesn't pop the dock. - if dwellInterval <= 0 { - dwellDeadline = nil - state = .shown - return .reveal - } - let deadline = dwellDeadline ?? (now + dwellInterval) - dwellDeadline = deadline - if now >= deadline { - dwellDeadline = nil - state = .shown - return .reveal - } - return .idle // still dwelling — nothing shown yet - } - // Cursor left the trigger before the dwell elapsed → cancel it (a fresh entry restarts the dwell). - dwellDeadline = nil - // 2. Cursor anywhere in the contiguous live zone while shown (the rail, the connecting band, OR the - // notch pixels above the zone) → keep it open. Falls back to the rail frame when no live zone is - // supplied. This is the "move into the notch docks instead of dismisses" guarantee (design D5). - if state == .shown { - if let liveZone, liveZone.contains(cursor) { - graceDeadline = nil - return .reveal - } - if let railFrame, railFrame.contains(cursor) { - graceDeadline = nil - return .reveal - } - } - // 3. Outside the live zone. - guard state == .shown else { return .idle } - let deadline = graceDeadline ?? (now + graceInterval) - graceDeadline = deadline - if now >= deadline { - state = .hidden - graceDeadline = nil - return .dismiss - } - return .reveal - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Parked/NotchSessionEngine.swift b/Sources/ThreeFingerSwitcher/AI/Parked/NotchSessionEngine.swift deleted file mode 100644 index bfb2287..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Parked/NotchSessionEngine.swift +++ /dev/null @@ -1,741 +0,0 @@ -import Foundation -import Combine -import os - -/// Per-turn streaming diagnostics (`notch-timeline-and-tuning`): turn start (path + effective -/// reasoning), first token per channel (arrival latency), and the settled channel totals — so -/// "thinking isn't streaming" is diagnosable from `log stream` without a debug build. No per-token -/// spam: first-arrival lines are one-shot per turn. -private let notchTurnLog = Logger(subsystem: "ThreeFingerSwitcher", category: "NotchSession") - -/// The transient set of images staged on a notch composer before the next send. Pure value — the engine -/// owns it; the expanded conversation view renders/edits it via the engine's attach/remove seams. `text` -/// is reserved for a future staged-text affordance; today only `images` is populated. -struct PendingAttachments: Equatable { - /// Staged images for the next turn (each PNG); folded into the next user message's `images`. - var images: [Data] = [] - /// Whether anything is staged (drives the composer's "has attachments" affordance). - var isEmpty: Bool { images.isEmpty } -} - -/// The multi-turn conversation machine for ONE notch session (`notch-native-conversations` design D1) — -/// the executor's conversational half re-homed. The launcher's `AICommandExecutor` is one-shot-only again; -/// every typed conversation lives at the notch, driven by an instance of this engine bound to a durable -/// `AgentConversation` from the parked store. `ParkController` owns the instances (one per session with a -/// live foreground turn; exactly one is bound to the expanded panel at a time) — the engine itself knows -/// nothing about panels, stores, or schedulers; it reports through `onTurnSettled`/`onTaskComplete`. -/// -/// What deliberately did NOT move here from the executor (canvas-era concepts with no notch meaning): -/// seed acquisition (`SeedKind`/`BareSeedDefault` — a notch session starts from typing), output routing -/// (`extractLatest` — the notch writes nothing into other apps; the expanded view offers Copy), and the -/// at-top/at-bottom gesture gates (the notch is a cursor-and-keyboard surface). -/// -/// `@MainActor` + `ObservableObject`: it holds observable UI state, matching the executor convention. -@MainActor -final class NotchSessionEngine: ObservableObject { - - /// The engine's observable state — the conversational subset of the old executor contract. - enum State: Equatable { - /// No session bound / the session was discarded. - case idle - /// AI can't run a turn yet: the opt-in is off or the model isn't downloaded/ready. The bound - /// conversation is KEPT (it is durable) — the expanded view offers enable/download and a later - /// send retries. - case unavailable - /// Resolving + loading the model for a turn (a visible state, never a silent block). - case loadingModel - /// An assistant turn is streaming; `partial` is the response-channel accumulation. - case conversing(partial: String) - /// The thread is idle, awaiting the next user turn (Enter = send, from the composer). - case awaitingTurn - /// The route loop paused at a `.confirm`/`.dangerous` tool step: the expanded view renders the - /// carried `TaskReview` as a card with Approve / Skip buttons. - case awaitingApproval(TaskReview) - /// A typed failure with a clean, human-readable headline (never raw error text). - case failed(message: String) - - static func == (lhs: State, rhs: State) -> Bool { - switch (lhs, rhs) { - case (.idle, .idle), (.unavailable, .unavailable), (.loadingModel, .loadingModel), - (.awaitingTurn, .awaitingTurn): - return true - case let (.conversing(a), .conversing(b)): return a == b - case let (.failed(a), .failed(b)): return a == b - case let (.awaitingApproval(a), .awaitingApproval(b)): return TaskReview.previewEqual(a, b) - default: return false - } - } - } - - @Published private(set) var state: State = .idle - /// The model's streamed REASONING for the in-flight turn — the flat accumulation (kept as the badge / - /// test seam); retained on the appended assistant message for DISPLAY only, NEVER re-fed. - @Published private(set) var thinking: String = "" - /// The in-flight turn's ordered TIMELINE (`notch-timeline-and-tuning`): thinking and answer segments - /// appended in token-arrival order (append-or-coalesce — a token whose channel matches the last - /// segment's kind extends it; a channel flip starts a new segment). The expanded view streams this - /// live; `appendAssistantTurn` persists it onto the settled message. Display-only, never re-fed. - @Published private(set) var liveSegments: [TurnSegment] = [] - /// The ordered tool steps the route loop has run for the current turn (the expanded view renders a - /// compact list of `ToolStepResult.summary`). Reset per turn. - @Published private(set) var toolSteps: [ToolStepResult] = [] - /// The composer's staged attachments for the NEXT user turn. The attach UI is a documented future; - /// the seams are kept so `send(_:images:)` remains capable (design: Non-Goals). - @Published private(set) var pendingAttachments = PendingAttachments() - /// The bound conversation — durable ownership stays with the parked store; the engine holds it for - /// the session's foreground life and hands snapshots back through `unbind`/`onTurnSettled`. - private(set) var conversation: AgentConversation? - - /// How a turn ended — the controller's row/badge classification input. There is deliberately NO - /// "task complete" notion (`refactor-park-and-background-agents`): a settled answer is an unseen - /// result, never a terminal state that removes the session. - enum TurnSettlement: Equatable { - /// The turn produced/updated the thread normally (an answer, or a routed stop with text). - case answered - /// The loop paused on a background approval — no assistant message was fabricated; the pending - /// step waits on the user (dormant parked, or needs-you when it escalated). - case pausedAwaitingUser - /// The turn failed with a clean headline (observable, never silent) — a detached failure - /// re-parks with a scheduled retry via the advance-feedback seam. - case failed(headline: String) - } - - /// Reports EVERY settled turn (streamed answer appended, routed outcome appended, a paused loop, or - /// a turn failure) with a durable snapshot + how it ended. The controller persists the snapshot and - /// updates the row/badge. Fires for detached (collapsed mid-turn) sessions too — that is the - /// "collapse does not cancel" contract. - var onTurnSettled: (@MainActor (AgentConversation, TurnSettlement) -> Void)? - - /// Fires when the route loop pauses at THIS engine's approval gate (state → `.awaitingApproval`). - /// The controller surfaces `.needsYou` when the session is detached — a confirm/dangerous step that - /// arises AFTER the user docked would otherwise suspend invisibly behind a "Working…" badge. - var onApprovalPending: (@MainActor () -> Void)? - - /// Fires when a user turn is SENT — the user message has been appended (and the session named), BEFORE - /// the model-availability gate. Lets the controller PERSIST a session that was unsaved until its first - /// message: an empty new chat writes nothing to the store (it is never docked, and is discarded if - /// closed empty); the first message makes it durable + joins the dock. Fires on every send (the - /// controller's upsert is idempotent). Does NOT fire for an empty/whitespace send. - var onTurnStarted: (@MainActor (AgentConversation) -> Void)? - - /// The born-with tuning snapshot the notch settings slider yields for a NEW conversation (design D7): - /// nil = no dial wired (the legacy behavior — `reasoningDefault` + the injected budget), so existing - /// call sites and tests are unchanged. - typealias TuningSnapshot = (reasoning: Bool, contextTokens: Int) - - private let modelManager: ModelManager - private let selection: SelectionProviding - private let contextProvider: @MainActor () -> FireContext - private let reasoningDefault: @MainActor () -> Bool - private let budgetProvider: ContextBudgetProviding - private let tuningDefault: @MainActor () -> TuningSnapshot? - private let registry: ToolRegistry? - private let candidateSource: ToolCandidateSource? - private let skillTools: @MainActor (String) -> [String] - private let backgroundRunner: BackgroundToolRunner? - - private var sessionReasoning = false - /// The BOUND conversation's born-with context budget (stamped at birth / restored at bind); nil = - /// the legacy injected `budgetProvider`. - private var sessionContextTokens: Int? - /// One-shot first-arrival log flags (see `notchTurnLog`), reset at each turn start. - private var loggedFirstThinking = false - private var loggedFirstAnswer = false - private var turnStartedAt = Date() - private var sessionParameters: GenerationParameters = .default - private var generationTask: Task? - private var approvalGate: CanvasApprovalGate? - - init(modelManager: ModelManager, - selection: SelectionProviding, - contextProvider: @escaping @MainActor () -> FireContext = { FireContext() }, - reasoning: @escaping @MainActor () -> Bool = { false }, - budgetProvider: ContextBudgetProviding = DefaultContextBudget(), - tuningDefault: @escaping @MainActor () -> TuningSnapshot? = { nil }, - registry: ToolRegistry? = nil, - candidateSource: ToolCandidateSource? = nil, - skillTools: @escaping @MainActor (String) -> [String] = { _ in [] }, - backgroundRunner: BackgroundToolRunner? = nil, - loopBudget: @escaping @MainActor () -> LoopBudget = { .default }, - narrator: (@Sendable (String) -> Void)? = nil) { - self.modelManager = modelManager - self.selection = selection - self.contextProvider = contextProvider - self.reasoningDefault = reasoning - self.budgetProvider = budgetProvider - self.tuningDefault = tuningDefault - self.registry = registry - self.candidateSource = candidateSource - self.skillTools = skillTools - self.backgroundRunner = backgroundRunner - self.loopBudget = loopBudget - self.narrator = narrator - } - - /// The wall-clock loop bounds (settings-fed at composition; `add-voice-computer-use-agent` D8). - private let loopBudget: @MainActor () -> LoopBudget - /// Spoken narration sink for auto-approved acts (nil = visible-only narration). - private let narrator: (@Sendable (String) -> Void)? - /// The conversation's live auto-approve grant (design D7): lock-guarded because the route loop - /// reads it OFF the main actor; synced from the bound conversation and by `setAutoApprove`. - let autoGrant = LockedBool() - - /// Flip THIS conversation's auto-approve grant (the `enable_auto_mode`/`disable_auto_mode` tools - /// and the surface toggle land here). Persisted with the conversation on the next settle. - func setAutoApprove(_ on: Bool) { - autoGrant.value = on - conversation?.autoApprove = on - } - - /// Whether a turn is currently being produced (loading, streaming, OR paused at the approval gate) — - /// the controller reads this on collapse to decide whether the engine must be kept alive detached - /// until the turn settles. `.awaitingApproval` COUNTS as in flight: the route loop is suspended on - /// this engine's gate, and dropping the engine would orphan the suspended continuation (the - /// docked-mid-approval bug — the response stops forever and the idle row expires). - var isTurnInFlight: Bool { - switch state { - case .loadingModel, .conversing, .awaitingApproval: return true - default: return false - } - } - - /// Whether the in-flight turn is paused at the foreground approval gate (the suspended continuation - /// lives in this engine's `approvalGate`). The controller surfaces `.needsYou` when such a session - /// is docked; re-expanding re-presents the same card and Approve/Skip resumes the original step. - var isPausedAtApproval: Bool { - if case .awaitingApproval = state { return true } - return false - } - - /// Whether the BOUND session runs its turns with reasoning (born-with when stamped, else the global - /// default read at bind). The expanded view badges a no-thinking session with this — a session born - /// under Quick would otherwise look broken ("why is there no streamed thinking?") instead of tuned. - var isSessionReasoningEnabled: Bool { sessionReasoning } - - /// Whether the turn is actively PRODUCING (loading or streaming) — as opposed to suspended at the - /// approval gate, which consumes no generation slot. The background driver counts only these when - /// deciding whether the single slot is taken (a session paused on the user must never block other - /// sessions from advancing). - var isGenerating: Bool { - switch state { - case .loadingModel, .conversing: return true - default: return false - } - } - - // MARK: - Session verbs (the ParkController seam) - - /// Start a NEW session: a fresh, empty conversation with a stable identity, idling for its first - /// typed turn. The controller persists it immediately (durable at birth) and expands it. - func startNew(title: String = "New chat") -> AgentConversation { - cancelTurnMachinery() - let now = Date() - var fresh = AgentConversation(title: title, messages: [], createdAt: now, updatedAt: now) - // BORN-WITH tuning (design D7): a new conversation snapshots the notch dial at birth and carries - // it for life (persisted on the conversation) — a later slider change never retunes this session. - // No dial wired (nil) keeps the legacy snapshot-only reasoning default. - if let tuning = tuningDefault() { - fresh.reasoningOverride = tuning.reasoning - fresh.contextTokens = tuning.contextTokens - sessionReasoning = tuning.reasoning - sessionContextTokens = tuning.contextTokens - } else { - sessionReasoning = reasoningDefault() - sessionContextTokens = nil - } - conversation = fresh - autoGrant.value = false // a fresh conversation never inherits a grant - sessionParameters = .default - thinking = "" - liveSegments = [] - toolSteps = [] - clearPendingAttachments() - state = .awaitingTurn - return fresh - } - - /// Bind a stored conversation for foreground use (the card was expanded). Idles in `.awaitingTurn`; - /// a session that had escalated re-presents its approval on the next turn's route loop (the pending - /// decision is not modeled by the durable store — the restore-era behavior, unchanged). - func bind(_ stored: AgentConversation) { - cancelTurnMachinery() - conversation = stored - autoGrant.value = stored.isAutoApproveGranted // the persisted grant follows the session - // BORN-WITH tuning: a conversation stamped at birth keeps its own reasoning + budget for life; a - // pre-change conversation (nil fields) keeps the exact legacy behavior (the global reasoning - // default re-read at bind, the injected budget). - sessionReasoning = stored.reasoningOverride ?? reasoningDefault() - sessionContextTokens = stored.contextTokens - sessionParameters = .default - thinking = "" - liveSegments = [] - toolSteps = [] - clearPendingAttachments() - state = .awaitingTurn - } - - /// Detach the engine from the expanded panel (collapse). Returns the current snapshot for the - /// controller to persist. Does NOT cancel an in-flight turn — the turn keeps streaming detached and - /// reports through `onTurnSettled` when it lands (the spec's collapse-mid-turn contract). When no - /// turn is in flight the engine fully resets to `.idle`. - @discardableResult - func unbind() -> AgentConversation? { - let snapshot = conversation - clearPendingAttachments() - if !isTurnInFlight { - conversation = nil - thinking = "" - liveSegments = [] - toolSteps = [] - state = .idle - } - return snapshot - } - - /// Tear the session down entirely (the authoritative discard path): cancel any pending generation - /// (a cancellation is NOT a failure), resolve a pending approval to cancel, drop the conversation. - func cancelAll() { - cancelTurnMachinery() - conversation = nil - thinking = "" - liveSegments = [] - toolSteps = [] - clearPendingAttachments() - state = .idle - } - - private func cancelTurnMachinery() { - generationTask?.cancel() - generationTask = nil - approvalGate?.resolve(.cancel) - approvalGate = nil - } - - // MARK: - Turns - - /// Send the next user turn (Enter from the composer): append the message — text plus any staged or - /// passed `images` — and stream the assistant turn. The FIRST turn of a new session also names it - /// (title from the first line). No-op when no conversation is bound, or when the text is empty AND - /// no images ride along. Supersedes any in-flight turn. Availability-gated: no model → `.unavailable` - /// with the conversation kept (a later send retries). - func send(_ text: String, images: [Data] = []) { - guard conversation != nil else { return } - let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines) - let turnImages = images.isEmpty ? pendingAttachments.images : images - guard !trimmed.isEmpty || !turnImages.isEmpty else { return } - - generationTask?.cancel() - let now = Date() - // First turn names a fresh session (title was the "New chat" placeholder until now). - if conversation?.messages.isEmpty == true { - conversation?.title = Self.sessionTitle(from: trimmed) - } - conversation?.messages.append(AgentMessage(role: .user, text: text, images: turnImages, createdAt: now)) - conversation?.updatedAt = now - clearPendingAttachments() - // The user's first message makes the session durable + docked (`newSession` persists nothing until - // now). Appended BEFORE the availability gate, so a missing model shows the unavailable hint without - // dropping — or un-saving — what the user typed. - if let convo = conversation { onTurnStarted?(convo) } - - guard modelManager.optedIn, AICommandExecutor.modelIsOnDisk(modelManager.state) else { - state = .unavailable - return - } - - state = .loadingModel - generationTask = Task { [weak self] in - guard let self else { return } - await self.runTurn() - } - } - - /// Advance the bound conversation WITHOUT a new user turn — the background driver's verb - /// (`refactor-park-and-background-agents`): run the PENDING turn of a conversation whose last - /// message still awaits an assistant reply (a quit-mid-turn session recovered at relaunch, or a - /// scheduled retry after a failed advance). No-op when nothing is pending. Availability-gated like - /// `send` (no model → `.unavailable` with the conversation kept). - func advance() { - guard let current = conversation, - let last = current.messages.last, last.role != .assistant else { return } - guard modelManager.optedIn, AICommandExecutor.modelIsOnDisk(modelManager.state) else { - state = .unavailable - return - } - generationTask?.cancel() - state = .loadingModel - generationTask = Task { [weak self] in - guard let self else { return } - await self.runTurn() - } - } - - /// Discard the IN-FLIGHT turn only: cancel generation, append no partial assistant message, leave - /// the thread open and idle — a mid-turn discard is NOT a failure. No-op when nothing is bound. - func discardTurn() { - guard conversation != nil else { return } - generationTask?.cancel() - generationTask = nil - approvalGate?.resolve(.cancel) - approvalGate = nil - thinking = "" - liveSegments = [] - state = .awaitingTurn - } - - // MARK: - Tool-step approval (the expanded view's Approve/Skip buttons drive the route-loop gate) - - /// The `ApprovalGate` the bounded route loop awaits at a `.confirm`/`.dangerous` step. When a step - /// pauses, the engine transitions to `.awaitingApproval(review)` (the expanded view renders the card) - /// and the loop suspends until `approve()` / `skip()` resolves it. Created fresh per loop. - func makeApprovalGate() -> CanvasApprovalGate { - let gate = CanvasApprovalGate( - onAwait: { [weak self] review in - self?.state = .awaitingApproval(review) - self?.onApprovalPending?() - }, - onResolve: { _ in }) - approvalGate = gate - return gate - } - - /// The Approve button resolves the pending step to `.approve`; the route loop resumes and fires the - /// step. Returns false when no step is pending. - @discardableResult - func approve() -> Bool { - approvalGate?.resolve(.approve) ?? false - } - - /// The Skip button resolves the pending step to `.skip`; the loop resumes, declines the step, and - /// the model may pivot. Returns false when no step is pending. - @discardableResult - func skip() -> Bool { - approvalGate?.resolve(.skip) ?? false - } - - /// Record a tool step the route loop ran (the expanded view renders the running list). - func appendToolStep(_ step: ToolStepResult) { - toolSteps.append(step) - } - - // MARK: - Composer attachments (API kept; the attach UI is a documented future) - - /// Stage the LIVE clipboard image (PNG) for the next turn. Reads via the injected - /// `SelectionProviding.readClipboardImage()` (the live pasteboard, normalized to PNG); a nil read is - /// a no-op, not a failure. Returns whether an image was staged. - @discardableResult - func attachClipboardImage() -> Bool { - guard let png = selection.readClipboardImage() else { return false } - pendingAttachments.images.append(png) - return true - } - - /// Stage captured screenshot bytes for the next turn (the capture itself is the caller's concern). - func attachScreenshot(_ data: Data) { - pendingAttachments.images.append(data) - } - - /// Remove the staged attachment at `index`. Out-of-range is a no-op. - func removeAttachment(at index: Int) { - guard pendingAttachments.images.indices.contains(index) else { return } - pendingAttachments.images.remove(at: index) - } - - /// Clear ALL staged attachments: once a turn folds them onto its message, and on every session reset. - func clearPendingAttachments() { - if !pendingAttachments.isEmpty { pendingAttachments = PendingAttachments() } - } - - // MARK: - The turn loop (moved wholesale from the executor's conversational half) - - /// Run one assistant turn over the bound conversation: compact if needed, assemble the request - /// reading committed text ONLY, stream the channel-split turn, then append the assistant message - /// (response as `text`; reasoning retained as DISPLAY-only `thinking`), go `.awaitingTurn`, and - /// report the settled snapshot. A discard mid-turn appends nothing and is not a failure; a - /// non-cancel error → a clean `.failed` (never silence, never a false continuation). - private func runTurn() async { - guard let current = conversation, !current.messages.isEmpty else { return } - - // Select + load a runtime for this turn's capabilities (vision iff any turn carries an image). - let needsVision = current.messages.contains { !$0.images.isEmpty } - let caps: Set = needsVision ? [.text, .vision] : [.text] - let runtime: LLMRuntime - do { - runtime = try await modelManager.runtime(requiring: caps) - } catch { - state = .failed(message: Self.message(for: error)) - return - } - if Task.isCancelled { return } - - // Compaction: when the assembled estimate approaches the session's budget (born-with when the - // conversation carries one, else the injected default), collapse older turns into a summary - // BEFORE assembling. A failed summarization is a turn `.failed` and does NOT drop history (the - // plan applies only on a successful summary, so a retry sees it all). - if ConversationCompactor.needsCompaction(current, budget: effectiveBudget) { - let plan = ConversationCompactor.plan(current) - if !plan.isEmpty { - do { - let summary = try await ConversationCompactor.summarize(plan, runtime: runtime) - if Task.isCancelled { return } - conversation = ConversationCompactor.applied(plan, summary: summary, to: current) - } catch let error as RuntimeError { - if case .cancelled = error { return } - state = .failed(message: Self.message(for: error)) - return - } catch is CancellationError { - return - } catch { - state = .failed(message: Self.message(for: error)) - return - } - } - } - - // The tool-routing path: when a registry + candidate source are wired, the turn runs the bounded - // route → execute → continue loop so the model is offered the app's tools. The plain-chat path - // below is unchanged for the `registry == nil` case (tests with a bare runtime). - if let registry, let candidateSource { - await runRoutedTurn(runtime: runtime, registry: registry, candidateSource: candidateSource) - return - } - - guard let request = assembleRequest() else { return } - - // Stream the turn: `.thinking` → live `thinking` (reset per turn); `.response` → `.conversing`. - // BOTH channels also append to `liveSegments` in arrival order — the transcript's timeline. - thinking = "" - liveSegments = [] - beginTurnDiagnostics(path: "plain") - state = .conversing(partial: "") - var accumulated = "" - do { - for try await token in runtime.chat(request) { - if Task.isCancelled { return } - switch token.channel { - case .thinking: - thinking += token.text - appendLive(.thinking, token.text) - case .response: - accumulated += token.text - appendLive(.answer, token.text) - state = .conversing(partial: accumulated) - } - } - if Task.isCancelled { return } - appendAssistantTurn(accumulated) - } catch let error as RuntimeError { - if case .cancelled = error { return } // a discard is not a failure - state = .failed(message: Self.message(for: error)) - } catch is CancellationError { - return - } catch { - state = .failed(message: Self.message(for: error)) - } - } - - /// Run one assistant turn through the bounded route → execute → continue loop: build a - /// `RouteContext` from the conversation (committed `text` is the only re-fed content; the active - /// skill's `toolNames` resolve into `allowedTools`), drive an `AgentLoop` with this engine's approval - /// gate, record each step, and settle the terminal outcome. `.failed` becomes a clean, non-blocking - /// `.failed` state; a `.cancelled` discard appends nothing and is not a failure. - private func runRoutedTurn(runtime: LLMRuntime, registry: ToolRegistry, - candidateSource: ToolCandidateSource) async { - guard let current = conversation else { return } - - let allowed = current.skillID.map { skillTools($0) } ?? [] - let context = RouteContext(messages: current.messages, - activeSkillID: current.skillID, - allowedTools: allowed) - let gate = makeApprovalGate() - - thinking = "" - liveSegments = [] - toolSteps = [] - beginTurnDiagnostics(path: "routed") - state = .conversing(partial: "") - - // Ordered, turn-owned CHANNEL stream (`notch-timeline-and-tuning`): the loop yields tagged - // (channel, text) events (off-main) into ONE stream and ONE consumer appends them in order on - // the main actor, drained before the turn settles — so cross-channel arrival order is serialized - // exactly as emitted (the timeline's interleaving is decided here, once). The old per-token - // detached `Task`s were unordered and could land after settlement or discard. - var streamContinuation: AsyncStream<(TokenChannel, String)>.Continuation! - let tokens = AsyncStream<(TokenChannel, String)> { streamContinuation = $0 } - let continuation = streamContinuation! - let consumer = Task { [weak self] in - var partial = "" - for await (channel, text) in tokens { - guard let self else { continue } - switch channel { - case .thinking: - self.thinking += text - self.appendLive(.thinking, text) - case .response: - partial += text - self.appendLive(.answer, text) - self.state = .conversing(partial: partial) - } - } - } - - let fireContext = contextProvider() - let source = TaskSource(appName: fireContext.capturedAppName, url: fireContext.url, - timestamp: fireContext.date) - // Auto mode (design D7): the wrapper resolves `.confirm` pauses instantly under the live - // grant, narrating each auto-approved act into the visible thinking stream AND the spoken - // narrator — silence never hides an act. Without the grant it is a pass-through. - let narrator = self.narrator - let autoGate = AutoApprovingGate(base: gate, - isGranted: { [autoGrant] in autoGrant.value }, - narrate: { text in - continuation.yield((.thinking, text + "\n")) - narrator?(text) - }) - // `onResponseToken` is WIRED (`notch-timeline-and-tuning`): the routed turn's final answer - // streams token-by-token into the timeline + `.conversing(partial:)` exactly like the - // plain-chat path — it no longer appears whole at settle. - let loop = AgentLoop(runtime: runtime, registry: registry, candidateSource: candidateSource, - gate: autoGate, reasoning: sessionReasoning, source: source, - onThinking: { continuation.yield((.thinking, $0)) }, - sessionID: current.id, - backgroundRunner: backgroundRunner, - budget: loopBudget(), - isAutoGranted: { [autoGrant] in autoGrant.value }, - onResponseToken: { continuation.yield((.response, $0)) }) - let result = await loop.run(context: context) - continuation.finish() - await consumer.value - if Task.isCancelled { return } - - toolSteps = result.steps - - switch result.outcome { - case let .answered(text), let .capReached(text): - appendAssistantTurn(text) - case let .stopped(reason, text): - if reason == .cancelled { return } // a discard appends nothing and is not a failure - appendAssistantTurn(text) - case .pausedAwaitingUser: - // The loop paused on a background approval: the steps are recorded, NO assistant message is - // fabricated, and the snapshot settles so the controller classifies the row (dormant parked, - // or needs-you when the step escalated). The pending decision re-presents restore-era style. - state = .awaitingTurn - if let settled = conversation { onTurnSettled?(settled, .pausedAwaitingUser) } - case let .failed(headline): - state = .failed(message: headline) - // A failure still settles the turn (the user message is in the thread) so a collapsed - // session's snapshot persists what happened and a detached advance can schedule a retry. - if let settled = conversation { onTurnSettled?(settled, .failed(headline: headline)) } - } - } - - /// Append the assistant turn (response only as `text`; thinking + the ordered segment timeline - /// retained for DISPLAY only and NEVER re-fed), return to `.awaitingTurn`, and report the settled - /// snapshot to the controller. - private func appendAssistantTurn(_ text: String) { - let now = Date() - notchTurnLog.notice("turn settled: thinkingChars=\(self.thinking.count, privacy: .public) answerChars=\(text.count, privacy: .public) segments=\(self.liveSegments.count, privacy: .public)") - conversation?.messages.append(AgentMessage(role: .assistant, text: text, - thinking: thinking.isEmpty ? nil : thinking, - segments: settledSegments(finalText: text), - createdAt: now)) - conversation?.updatedAt = now - // The live timeline hands off to the persisted message (no double-render); the flat `thinking` - // deliberately KEEPS its accumulation until the next turn resets it (the existing seam). - liveSegments = [] - state = .awaitingTurn - if let settled = conversation { onTurnSettled?(settled, .answered) } - } - - /// The timeline persisted onto the settled message. The settled `text` is AUTHORITATIVE: normally it - /// equals the streamed answer segments and the live timeline persists as-is; when a routed stop path - /// settles text that never streamed (or streamed differently), the answer segments are rebuilt from - /// the settled text (thinking blocks keep their places) so the stored timeline never contradicts the - /// committed answer. An empty timeline persists as nil (the legacy fallback renders `thinking`/`text`). - private func settledSegments(finalText: String) -> [TurnSegment]? { - var segments = liveSegments - let streamedAnswer = segments.filter { $0.kind == .answer }.map(\.text).joined() - if streamedAnswer != finalText { - segments.removeAll { $0.kind == .answer } - if !finalText.isEmpty { segments.append(TurnSegment(kind: .answer, text: finalText)) } - } - return segments.isEmpty ? nil : segments - } - - /// Append-or-coalesce one streamed chunk onto the live timeline: a chunk whose channel matches the - /// last segment's kind extends it (one string mutation per token — the array's shape changes only at - /// channel flips); a flip starts a new segment. - private func appendLive(_ kind: TurnSegment.Kind, _ text: String) { - guard !text.isEmpty else { return } - logFirstArrival(kind) - if liveSegments.last?.kind == kind { - liveSegments[liveSegments.count - 1].text += text - } else { - liveSegments.append(TurnSegment(kind: kind, text: text)) - } - } - - /// Reset + emit the turn-start diagnostic line (see `notchTurnLog`). The `reasoning` value here is - /// the single discriminator for "thinking isn't streaming": false means the request never asked the - /// model to think (a tuning/session-birth question), true with no later thinking-first-token line - /// means the model produced none (a runtime/model question). - private func beginTurnDiagnostics(path: String) { - loggedFirstThinking = false - loggedFirstAnswer = false - turnStartedAt = Date() - notchTurnLog.notice("turn start: path=\(path, privacy: .public) reasoning=\(self.sessionReasoning, privacy: .public) contextTokens=\(self.sessionContextTokens ?? -1, privacy: .public)") - } - - /// One-shot per-channel first-arrival lines — proves each channel actually STREAMED (arrival - /// latency from turn start) rather than landing in a settle-time burst. - private func logFirstArrival(_ kind: TurnSegment.Kind) { - let elapsed = Date().timeIntervalSince(turnStartedAt) - switch kind { - case .thinking: - guard !loggedFirstThinking else { return } - loggedFirstThinking = true - notchTurnLog.notice("first THINKING token after \(String(format: "%.2f", elapsed), privacy: .public)s") - case .answer: - guard !loggedFirstAnswer else { return } - loggedFirstAnswer = true - notchTurnLog.notice("first ANSWER token after \(String(format: "%.2f", elapsed), privacy: .public)s") - } - } - - /// The compaction budget for the BOUND conversation: its born-with token budget when stamped - /// (`notch-timeline-and-tuning` D7), else the injected provider (the legacy behavior). - private var effectiveBudget: ContextBudgetProviding { - sessionContextTokens.map { DefaultContextBudget(maxContextTokens: $0) } ?? budgetProvider - } - - /// Assemble the conversation into an `LLMChatRequest`: each message's committed `text` ONLY - /// (thinking is structurally excluded), any `compactedSummary` prefixed as a synthetic `system` - /// message, the latest turn's images, session reasoning + parameters. `internal` so tests can - /// assert assembly directly. - func assembleRequest() -> LLMChatRequest? { - guard let conversation else { return nil } - var messages: [AgentMessage] = [] - if let summary = conversation.compactedSummary, !summary.isEmpty { - messages.append(AgentMessage(role: .system, text: summary)) - } - messages.append(contentsOf: conversation.messages) - let images = conversation.messages.last(where: { !$0.images.isEmpty })?.images ?? [] - return LLMChatRequest(messages: messages, images: images, - parameters: sessionParameters, reasoning: sessionReasoning) - } - - /// A short session title from the first turn's first non-empty line; the generic placeholder stays - /// for an image-only first turn. - private static func sessionTitle(from text: String) -> String { - let firstLine = text.split(whereSeparator: \.isNewline).first - .map { $0.trimmingCharacters(in: .whitespaces) } ?? "" - if firstLine.isEmpty { return "New chat" } - return firstLine.count > 60 ? String(firstLine.prefix(60)) + "…" : firstLine - } - - /// One taxonomy, one translator: every failure surfaces the same clean headline everywhere. - private static func message(for error: Error) -> String { - AIError.message(for: error).headline - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Parked/NotchTuning.swift b/Sources/ThreeFingerSwitcher/AI/Parked/NotchTuning.swift deleted file mode 100644 index cc3120d..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Parked/NotchTuning.swift +++ /dev/null @@ -1,54 +0,0 @@ -import Foundation - -/// The notch chat's ONE tuning dial (`notch-timeline-and-tuning`, design D6): four ordered stops over -/// (reasoning, context) that the in-notch settings zone's slider scrubs. Deliberately notch-specific — -/// the Hub's global `aiReasoningEnabled` / `agentContextPreset` keep steering the launcher band and the -/// other agent surfaces; this dial shapes only conversations BORN at the notch (snapshot at birth, -/// carried for life — see `AgentConversation.reasoningOverride` / `.contextTokens`). -/// -/// Context values reuse `AgentContextPreset`'s resolution (Balanced 8k / Long 32k / model max) so the -/// two surfaces can never disagree about what a stop means in tokens. -public enum NotchTuning: String, Codable, Sendable, CaseIterable { - /// Thinking off · base context — the fastest first token, no reasoning latency. - case quick - /// Thinking on · base context — the default (matches the app-wide defaults). - case balanced - /// Thinking on · long context. - case deep - /// Thinking on · the model's architectural maximum context. - case max - - /// Whether conversations born under this stop reason before answering. - public var reasoning: Bool { self != .quick } - - /// The context-token budget this stop resolves to, clamped to the model max (the - /// `AgentContextPreset` resolution, so a stop and a Hub preset with the same name mean the same - /// number of tokens). - public func contextTokens(modelMax: Int) -> Int { - switch self { - case .quick, .balanced: return AgentContextPreset.balanced.tokens(modelMax: modelMax, custom: 0) - case .deep: return AgentContextPreset.long.tokens(modelMax: modelMax, custom: 0) - case .max: return AgentContextPreset.max.tokens(modelMax: modelMax, custom: 0) - } - } - - /// A short human label for the slider stop. - public var title: String { - switch self { - case .quick: return "Quick" - case .balanced: return "Balanced" - case .deep: return "Deep" - case .max: return "Max" - } - } - - /// This stop's position on the discrete slider (0…count-1, in declaration order). - public var sliderIndex: Int { Self.allCases.firstIndex(of: self) ?? 1 } - - /// The stop for a slider position, clamped into range (a fractional slider value rounds to the - /// nearest stop). - public static func fromSliderIndex(_ index: Int) -> NotchTuning { - let all = Self.allCases - return all[Swift.min(Swift.max(index, 0), all.count - 1)] - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Parked/ParkError.swift b/Sources/ThreeFingerSwitcher/AI/Parked/ParkError.swift deleted file mode 100644 index 307f9f6..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Parked/ParkError.swift +++ /dev/null @@ -1,37 +0,0 @@ -import Foundation - -/// Parked-session store/persistence failures (design §9) — a new taxonomy ONLY for the cases -/// `RuntimeError`/`TaskError`/`SkillError`/`MemoryError` cannot carry: durable-store IO. Each case has a -/// clean, user-facing `errorDescription`; raw OS/coding text stays in opt-in details / logs (never the -/// headline). `AIError.message(for:)` is extended to translate this (the single translator). -/// `FileManager`/JSON-coding throws map into this AT the `ParkedSessionStore` IO boundary; Core stays -/// MLX-free. -enum ParkError: Error, Equatable, LocalizedError { - /// The durable store directory could not be created / opened (permission, read-only volume). - case storeUnavailable(detail: String) - /// A parked session's conversation could not be written/read (disk full, permission, corruption). - case persistFailed(detail: String) - /// A sleeping session's one-line resume is missing when a restore needed it. - case resumeMissing - - var errorDescription: String? { - switch self { - case .storeUnavailable: - return "Parked sessions couldn't be opened on disk." - case .persistFailed: - return "That parked session couldn't be saved." - case .resumeMissing: - return "That parked session couldn't be resumed." - } - } - - /// The copyable raw detail kept OUT of the headline (nil when the headline says everything). - var rawDetail: String? { - switch self { - case let .storeUnavailable(detail), let .persistFailed(detail): - return detail - case .resumeMissing: - return nil - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Parked/ParkLifecycle.swift b/Sources/ThreeFingerSwitcher/AI/Parked/ParkLifecycle.swift deleted file mode 100644 index 8105114..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Parked/ParkLifecycle.swift +++ /dev/null @@ -1,49 +0,0 @@ -import Foundation - -/// The pure parked-session lifecycle: max-count eviction (idle-only) and the OPT-IN idle expiry -/// countdown. `now:` is injected so eviction-victim selection + dismissability are deterministic. -/// MLX-free Core. -/// -/// There is NO terminal auto-dismiss (`refactor-park-and-background-agents`): a settled turn never -/// removes a session. Expiry defaults OFF (countdown 0 = never) and NEVER touches a session with unseen -/// results — deletion is otherwise the user's alone (discard/purge), with eviction as the soft bound. -struct ParkLifecycle { - /// Soft target for the parked set; when exceeded, the least-recently-updated IDLE session is evicted. - var maxParked: Int - /// How long an `.idle`, fully-seen session may sit before it is auto-dismissed forever. `<= 0` - /// DISABLES expiry entirely (the default — sessions never age out unless the user opts in). - var autoDismissCountdown: TimeInterval - - init(maxParked: Int, autoDismissCountdown: TimeInterval) { - self.maxParked = maxParked - self.autoDismissCountdown = autoDismissCountdown - } - - /// Sessions to AUTO-DISMISS FOREVER under the opt-in expiry: `.idle` rows with NO unseen results - /// (`badgeCount == 0` — an unseen answer protects its session indefinitely) whose idle age exceeds - /// the countdown. `.active`/`.needsYou` (protected) and `.parked` (pending work — streaming, dormant - /// on an approval, or scheduled) are NEVER touched. A countdown of 0 (the default) dismisses nothing. - /// Pure; `now:` injected. Each returned id is routed through the authoritative `discard(_:)` path. - func dismissable(_ sessions: [ParkedSession], now: Date) -> [AgentSessionID] { - guard autoDismissCountdown > 0 else { return [] } - return sessions - .filter { session in - session.state == .idle - && session.badgeCount == 0 - && now.timeIntervalSince(session.updatedAt) > autoDismissCountdown - } - .map(\.id) - } - - /// When the parked set exceeds `maxParked`, the least-recently-updated `.idle` session to evict — - /// or `nil` when nothing is over-cap or no `.idle` victim exists (an all-non-idle set is never - /// force-evicted; the cap is a soft target, never a hard refusal that loses a conversation). - /// `.active`/`.needsYou`/thinking-`.parked` are NEVER chosen (the user's attention is protected). - func evictable(_ sessions: [ParkedSession], now: Date) -> AgentSessionID? { - guard sessions.count > maxParked else { return nil } - return sessions - .filter { $0.state == .idle } - .min { $0.updatedAt < $1.updatedAt }? - .id - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Parked/ParkLifecycleCoordinator.swift b/Sources/ThreeFingerSwitcher/AI/Parked/ParkLifecycleCoordinator.swift deleted file mode 100644 index 9116b9f..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Parked/ParkLifecycleCoordinator.swift +++ /dev/null @@ -1,49 +0,0 @@ -import Foundation - -/// Bridges the pure `ParkLifecycle` decisions to the durable store: the (opt-in) auto-dismiss pass — -/// every dismissable session is removed FOREVER through the SAME authoritative `discard(_:)` path as a -/// manual dismiss — and the discard flow (remove from the store — completed side effects are NOT rolled -/// back). Keeps Core MLX-free. -/// -/// Generation CANCELLATION is not owned here (`refactor-park-and-background-agents`): the per-session -/// `NotchSessionEngine` is the single cancellation owner — `ParkController` cancels via -/// `engines[id]?.cancelAll()` on every removal path. (The old `pending` task table here was dead in -/// production — only tests ever registered a task — so "discard cancels generation" could silently -/// no-op through this seam.) -final class ParkLifecycleCoordinator: @unchecked Sendable { - private let store: ParkedSessionStore - private let lifecycle: ParkLifecycle - - init(store: ParkedSessionStore, lifecycle: ParkLifecycle) { - self.store = store - self.lifecycle = lifecycle - } - - /// Auto-dismiss every dismissable session FOREVER (the opt-in expiry — idle, fully seen, past the - /// countdown): route each `lifecycle.dismissable(_:)` id through the EXISTING `discard(_:)` path - /// verbatim. Returns the dismissed ids. Protected (`.active`/`.needsYou`) sessions, unseen results, - /// and parked pending work are never touched. `now:` injected so the pass is deterministically - /// testable. - @discardableResult - func runAutoDismissPass(now: Date) -> [AgentSessionID] { - let ids = lifecycle.dismissable(store.all(), now: now) - for id in ids { try? discard(id) } // the authoritative removal path - return ids - } - - /// Evict the least-recently-updated idle session if over-cap; returns the evicted id (nil when none). - @discardableResult - func runEvictionPass(now: Date) -> AgentSessionID? { - guard let victim = lifecycle.evictable(store.all(), now: now) else { return nil } - try? store.remove(victim) - return victim - } - - /// Discard a parked session: remove the durable conversation + rail row. The caller - /// (`ParkController`) cancels the live engine's generation first (a cancellation is NOT a failure — - /// no `failed` badge is set). COMPLETED SIDE EFFECTS ARE NOT ROLLED BACK — a written event, a moved - /// file, a launched process stays done; discard stops only FUTURE work. - func discard(_ id: AgentSessionID) throws { - try store.remove(id) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Parked/ParkScheduler.swift b/Sources/ThreeFingerSwitcher/AI/Parked/ParkScheduler.swift deleted file mode 100644 index dd49cdb..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Parked/ParkScheduler.swift +++ /dev/null @@ -1,144 +0,0 @@ -import Foundation - -/// The park scheduler seam (blueprint §3.5 / design D2) — the SINGLE place the slot count is -/// interpreted, shaped so the batched runtime fills **K** slots at once as a pure drop-in. `now:` is an -/// INPUT (mirroring `DockHoverModel`) so decisions are deterministically testable. Consumes -/// `ToolStepResult`/`ToolStepStatus` (`ai-tool-routing`) verbatim. MLX-free Core. -public protocol ParkScheduler: Sendable { - /// Which parked sessions to advance now, ordered by priority, up to `maxSlots`. Pure: result depends - /// only on the parked set and `now`. The batched runtime decides `maxSlots`; the scheduler never - /// assumes the count, so the one→K transition needs NO protocol change. - func runnableSessions(now: Date, maxSlots: Int) -> [AgentSessionID] - /// Feedback after a (batch) step settles the run and updates the session's rail row. There is NO - /// terminal signal (`refactor-park-and-background-agents`): a settled `.done` is an unseen result - /// (idle + badge), never a state that removes the session. - func didAdvance(_ id: AgentSessionID, result: ToolStepResult) - /// A dangerous write / required approval escalated this parked session to `.needsYou` (badge + glow). - func escalate(_ id: AgentSessionID, reason: String) - /// The current `ParkState` for `id`, read thread-safely from ANY thread (the background-autonomy runner - /// reads it per step OFF the main actor). Defaults to `.active` (an unknown id is the foreground active - /// session, never parked) so a conformer that does not track rows needs no override. The concrete - /// `SerialParkScheduler` overrides it with a lock-guarded read of its live rows. - func parkState(of id: AgentSessionID) -> ParkState -} - -extension ParkScheduler { - /// Default: foreground/`.active`. A conformer that tracks rows (the concrete scheduler) overrides this. - func parkState(of id: AgentSessionID) -> ParkState { .active } -} - -/// Shared, pure filter+order used by every concrete scheduler so the runnable contract is identical -/// regardless of `maxSlots`. A session is runnable iff it is `.parked` AND SCHEDULED — `nextRunAt` set -/// and arrived. A `.parked` row with NO `nextRunAt` is DORMANT (blocked on the user, e.g. a confirm-tier -/// pause) and is never runnable; `.needsYou` (blocked on the user), `.active`, and `.idle` are excluded. -/// Ordered oldest-waiting first: by `nextRunAt`, then `updatedAt`. -enum ParkRunnable { - static func ordered(_ sessions: [ParkedSession], now: Date) -> [AgentSessionID] { - sessions - .filter { $0.state == .parked && $0.nextRunAt.map { $0 <= now } == true } - .sorted { lhs, rhs in - let l = lhs.nextRunAt ?? .distantPast - let r = rhs.nextRunAt ?? .distantPast - if l != r { return l < r } - return lhs.updatedAt < rhs.updatedAt - } - .map(\.id) - } -} - -/// The v1 policy: at most ONE runnable session regardless of `maxSlots` (one active generation in this -/// slice). The batched runtime reserves slot 0 for the foreground active session; this scheduler fills -/// only the *remaining* parked slots — so `SerialParkScheduler` returning 1 + a foreground session is -/// two streams once batching lands, which is exactly the K-ready story. Backed by an injected snapshot -/// of the parked rows (the store/coordinator owns the rows; the scheduler is a pure read over them). -public final class SerialParkScheduler: ParkScheduler, @unchecked Sendable { - /// The current parked rows, read at decision time. Mutated through `didAdvance`/`escalate` so the - /// rail reflects the latest state. Serialized by the owning coordinator (main actor in practice). - private var sessions: [AgentSessionID: ParkedSession] - private let lock = NSLock() - - public init(sessions: [ParkedSession] = []) { - self.sessions = Dictionary(uniqueKeysWithValues: sessions.map { ($0.id, $0) }) - } - - /// Replace/seed the row snapshot the scheduler reads (e.g. after a store reload). - public func setSessions(_ rows: [ParkedSession]) { - lock.lock(); defer { lock.unlock() } - sessions = Dictionary(uniqueKeysWithValues: rows.map { ($0.id, $0) }) - } - - /// Current rows (for the store / lifecycle), ordered by `updatedAt` **ascending** (oldest first — the - /// least-recently-used, the eviction candidate, leads). - public func snapshot() -> [ParkedSession] { - lock.lock(); defer { lock.unlock() } - return sessions.values.sorted { $0.updatedAt < $1.updatedAt } - } - - /// The RAIL display order: rows **most-recently-used first** (`updatedAt` descending), so the last-used - /// session sits right after the "+ New chat" card and the oldest trails at the far end. Distinct from - /// `snapshot()` (oldest-first), which the store/lifecycle read. - public func railSnapshot() -> [ParkedSession] { - lock.lock(); defer { lock.unlock() } - return sessions.values.sorted { $0.updatedAt > $1.updatedAt } - } - - public func runnableSessions(now: Date, maxSlots: Int) -> [AgentSessionID] { - guard maxSlots > 0 else { return [] } - lock.lock(); let rows = Array(sessions.values); lock.unlock() - // At most one regardless of the caller's slot count — the v1 one-active-now policy. - return Array(ParkRunnable.ordered(rows, now: now).prefix(1)) - } - - /// How long a failed background advance waits before the driver retries it. - public static let failureRetryDelay: TimeInterval = 60 - - public func didAdvance(_ id: AgentSessionID, result: ToolStepResult) { - lock.lock(); defer { lock.unlock() } - guard var row = sessions[id] else { return } - switch result.status { - case .done: - // A new unseen result settled: idle + badge. NEVER a state that removes the session — the - // old terminal (`.completed` → instant dismiss) classification deleted docked chats. - row.updatedAt = Date() - row.badgeCount += 1 - row.state = .idle - row.nextRunAt = nil - case .failed: - // The failed badge carries the CLEAN headline only (the status already holds it); the row - // re-parks with a scheduled retry so the failure is never silent and never a dead end. - row.updatedAt = Date() - row.state = .parked - row.nextRunAt = Date().addingTimeInterval(Self.failureRetryDelay) - case .awaitingApproval: - // A paused step: a dangerous escalation already set `.needsYou` via `escalate` (never - // downgraded here); a confirm-tier pause waits parked DORMANT (no next-run time → not - // runnable) until the user brings the session back. - if row.state != .needsYou { - row.state = .parked - row.nextRunAt = nil - } - case .declined: - row.updatedAt = Date() - row.state = .parked - row.nextRunAt = nil - } - sessions[id] = row - } - - public func escalate(_ id: AgentSessionID, reason: String) { - lock.lock(); defer { lock.unlock() } - guard var row = sessions[id] else { return } - row.state = .needsYou - row.badgeCount = max(row.badgeCount, 1) - row.updatedAt = Date() - sessions[id] = row - } - - /// The live park state for `id`, lock-guarded so the background-autonomy runner can read it OFF the - /// main actor without trapping. An unknown id (the foreground active session, never parked) reads - /// `.active` — the same semantics `ParkController.parkState(of:)` exposes. - public func parkState(of id: AgentSessionID) -> ParkState { - lock.lock(); defer { lock.unlock() } - return sessions[id]?.state ?? .active - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Parked/ParkedSession.swift b/Sources/ThreeFingerSwitcher/AI/Parked/ParkedSession.swift deleted file mode 100644 index 39216f2..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Parked/ParkedSession.swift +++ /dev/null @@ -1,73 +0,0 @@ -import Foundation - -/// The parked-session shared contract (blueprint §3.5 / design D1). This slice (`ai-parked-sessions`, -/// Wave 3) OWNS `ParkState` and `ParkedSession`; it consumes `AgentSessionID`/`AgentConversation` -/// (`ai-conversation-runtime`) and `ToolStepResult`/`ToolStepStatus` (`ai-tool-routing`) verbatim and -/// never redefines them. MLX-free Core (`swift test`-verified). -/// -/// `ParkedSession` is the lightweight rail/scheduler ROW; the full `AgentConversation` lives in the -/// durable store keyed by the SAME `id`. `badgeCount` + `state` are the only two fields the badge view -/// reads. `Codable` because it persists alongside the conversation so the rail rebuilds on relaunch. - -/// The observable state of a (possibly parked) session. There is deliberately NO terminal state: a -/// conversational turn settling is never "task complete" (`refactor-park-and-background-agents` — the -/// old `.completed` → instant-auto-dismiss classification deleted chats docked mid-response). A session -/// is removed ONLY by user deletion/purge, the opt-in idle expiry, or the max-parked eviction. -public enum ParkState: String, Codable, Equatable, Sendable { - /// Foreground, in the canvas. - case active - /// Stashed at the notch home zone; may run in the background. `nextRunAt == nil` means DORMANT - /// (blocked on the user — e.g. a confirm-tier pause — never runnable until reactivated). - case parked - /// A dangerous write / required approval escalated to the foreground (badge + ambient glow). - case needsYou - /// Nothing pending — eligible for eviction and (opt-in) expiry. - case idle - - /// Migration: rows persisted by older builds under the RETIRED terminal state ("completed") — or any - /// unknown future raw value — decode as `.idle`, so no stored session ever fails to load or vanishes. - public init(from decoder: Decoder) throws { - let raw = try decoder.singleValueContainer().decode(String.self) - self = ParkState(rawValue: raw) ?? .idle - } - - /// The non-evictable, non-dismissable states whose attention/escalation is protected (the foreground - /// active session and an escalation waiting on the user). `parked`/`idle` are eligible for lifecycle - /// aging; `active`/`needsYou` are NEVER evicted or auto-dismissed (design D1/D7). - var isProtectedFromAging: Bool { - switch self { - case .active, .needsYou: return true - case .parked, .idle: return false - } - } -} - -/// One lightweight rail/scheduler row for a session. The full transcript lives in the store under `id`. -public struct ParkedSession: Codable, Equatable, Identifiable, Sendable { - /// SAME identity as `AgentConversation.id` (§3.1) — stable across park/restore. - public var id: AgentSessionID - /// Short, model- or first-turn-derived title, shown on the rail card. - public var title: String - public var state: ParkState - /// Unseen results / needs-you items shown on the rail card. - public var badgeCount: Int - /// When the background driver should next advance this session. nil = DORMANT: not scheduled, never - /// runnable (a session waiting on the user, or one with nothing pending). The driver clears it when - /// it serves the session so a second tick can never double-serve. - public var nextRunAt: Date? - public var updatedAt: Date - - public init(id: AgentSessionID, - title: String, - state: ParkState, - badgeCount: Int = 0, - nextRunAt: Date? = nil, - updatedAt: Date = Date()) { - self.id = id - self.title = title - self.state = state - self.badgeCount = badgeCount - self.nextRunAt = nextRunAt - self.updatedAt = updatedAt - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Parked/ParkedSessionStore.swift b/Sources/ThreeFingerSwitcher/AI/Parked/ParkedSessionStore.swift deleted file mode 100644 index 5ed688c..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Parked/ParkedSessionStore.swift +++ /dev/null @@ -1,225 +0,0 @@ -import Foundation -import os - -private let parkStoreLog = Logger(subsystem: "ThreeFingerSwitcher", category: "ParkedSessionStore") - -/// The durable parked-conversation store (design §3). This slice OWNS persistence; it consumes -/// `AgentConversation` (`ai-conversation-runtime`) verbatim — no duplicate type, no second retriever. -/// A small synchronous read seam over an in-memory index, with disk IO bridged off the read path the way -/// the Files band bridges its async lister: the scheduler/rail read `all()`/`conversation(_:)` -/// synchronously, while `upsert`/`remove` persist behind the boundary. Every `FileManager`/JSON-coding -/// throw maps into `ParkError` AT THE IO BOUNDARY (raw text to the log only). MLX-free Core. -protocol ParkedSessionStore: AnyObject, Sendable { - /// The lightweight rail/scheduler rows (sorted oldest-updated first). - func all() -> [ParkedSession] - /// The full durable conversation for a session, if persisted. - func conversation(_ id: AgentSessionID) -> AgentConversation? - /// Persist (insert/replace) a session's row + conversation. - func upsert(_ session: ParkedSession, conversation: AgentConversation) throws - /// Update just the lightweight row (state/badge/nextRunAt) without rewriting the transcript. - func upsertRow(_ session: ParkedSession) throws - /// Remove a session's row + conversation (discard / eviction). - func remove(_ id: AgentSessionID) throws - /// The persisted one-line resume for a (possibly slept) session. - func oneLineResume(_ id: AgentSessionID) -> String? - /// Persist a one-line resume for a session being put to sleep (KV dropped elsewhere). - func persistResume(_ id: AgentSessionID, resume: String) throws -} - -/// One persisted record: the lightweight row + the full conversation + an optional one-line resume. -private struct ParkedRecord: Codable { - var row: ParkedSession - var conversation: AgentConversation - var resume: String? -} - -/// The disk-backed store. One JSON file per session under `…/ThreeFingerSwitcher/parked`, plus an -/// in-memory index loaded once at init and kept in sync on each write — so reads are synchronous and -/// cheap and the rail rebuilds from disk on relaunch. -final class DiskParkedSessionStore: ParkedSessionStore, @unchecked Sendable { - private let directory: URL - private let lock = NSLock() - private var index: [AgentSessionID: ParkedRecord] = [:] - private let encoder: JSONEncoder = { - let e = JSONEncoder(); e.outputFormatting = [.sortedKeys]; e.dateEncodingStrategy = .iso8601; return e - }() - private let decoder: JSONDecoder = { - let d = JSONDecoder(); d.dateDecodingStrategy = .iso8601; return d - }() - - init(directory: URL = DiskParkedSessionStore.defaultDirectory()) { - self.directory = directory - loadIndex() - } - - /// `~/Library/Application Support/ThreeFingerSwitcher/parked` (parallels `MemoryStore`). - static func defaultDirectory() -> URL { - let base = (try? FileManager.default.url(for: .applicationSupportDirectory, in: .userDomainMask, - appropriateFor: nil, create: true)) - ?? URL(fileURLWithPath: NSTemporaryDirectory()) - return base.appendingPathComponent("ThreeFingerSwitcher/parked", isDirectory: true) - } - - private func fileURL(for id: AgentSessionID) -> URL { - directory.appendingPathComponent(id.raw.uuidString + ".json") - } - - /// Read every record into the in-memory index. A single corrupt file is skipped (logged), the rest - /// load — a relaunch never fails wholesale because one session's JSON went bad. - private func loadIndex() { - let fm = FileManager.default - guard let entries = try? fm.contentsOfDirectory(at: directory, includingPropertiesForKeys: nil) else { - return - } - for file in entries where file.pathExtension == "json" { - guard let data = try? Data(contentsOf: file), - let record = try? decoder.decode(ParkedRecord.self, from: data) else { - parkStoreLog.error("skipping unreadable parked record at \(file.lastPathComponent, privacy: .public)") - continue - } - index[record.row.id] = record - } - } - - func all() -> [ParkedSession] { - lock.lock(); defer { lock.unlock() } - return index.values.map(\.row).sorted { $0.updatedAt < $1.updatedAt } - } - - func conversation(_ id: AgentSessionID) -> AgentConversation? { - lock.lock(); defer { lock.unlock() } - return index[id]?.conversation - } - - func oneLineResume(_ id: AgentSessionID) -> String? { - lock.lock(); defer { lock.unlock() } - return index[id]?.resume ?? index[id]?.conversation.compactedSummary - } - - func upsert(_ session: ParkedSession, conversation: AgentConversation) throws { - let existingResume = peekResume(session.id) - let record = ParkedRecord(row: session, conversation: conversation, resume: existingResume) - try write(record) - } - - func upsertRow(_ session: ParkedSession) throws { - lock.lock() - guard var record = index[session.id] else { - lock.unlock() - // A row-only upsert for a session with no stored conversation persists a placeholder - // conversation under the same identity (so the rail rebuilds), never a silent drop. - let placeholder = AgentConversation(id: session.id, title: session.title, messages: []) - try upsert(session, conversation: placeholder) - return - } - record.row = session - lock.unlock() - try write(record) - } - - func remove(_ id: AgentSessionID) throws { - lock.lock(); index[id] = nil; lock.unlock() - do { - let url = fileURL(for: id) - if FileManager.default.fileExists(atPath: url.path) { - try FileManager.default.removeItem(at: url) - } - } catch { - parkStoreLog.error("parked remove failed: \(String(describing: error), privacy: .public)") - throw ParkError.persistFailed(detail: String(describing: error)) - } - } - - func persistResume(_ id: AgentSessionID, resume: String) throws { - lock.lock() - guard var record = index[id] else { - lock.unlock() - throw ParkError.resumeMissing - } - record.resume = resume - lock.unlock() - try write(record) - } - - private func peekResume(_ id: AgentSessionID) -> String? { - lock.lock(); defer { lock.unlock() } - return index[id]?.resume - } - - /// The IO boundary: create the dir + atomically write the record, mapping every throw to `ParkError`. - private func write(_ record: ParkedRecord) throws { - do { - try FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true) - } catch { - parkStoreLog.error("parked store dir create failed: \(String(describing: error), privacy: .public)") - throw ParkError.storeUnavailable(detail: String(describing: error)) - } - do { - let data = try encoder.encode(record) - try data.write(to: fileURL(for: record.row.id), options: .atomic) - } catch { - parkStoreLog.error("parked persist failed: \(String(describing: error), privacy: .public)") - throw ParkError.persistFailed(detail: String(describing: error)) - } - lock.lock(); index[record.row.id] = record; lock.unlock() - } -} - -/// An in-memory store for tests + the no-disk path. Same contract, no IO — a `failingStore` flag forces -/// the `ParkError` mapping path so tests can pin the "failure is observable, never silent" invariant. -final class InMemoryParkedSessionStore: ParkedSessionStore, @unchecked Sendable { - private let lock = NSLock() - private var index: [AgentSessionID: ParkedRecord] = [:] - /// When true, every write throws `.persistFailed` (simulating a forced IO failure at the boundary). - var failWrites = false - - init() {} - - func all() -> [ParkedSession] { - lock.lock(); defer { lock.unlock() } - return index.values.map(\.row).sorted { $0.updatedAt < $1.updatedAt } - } - - func conversation(_ id: AgentSessionID) -> AgentConversation? { - lock.lock(); defer { lock.unlock() } - return index[id]?.conversation - } - - func oneLineResume(_ id: AgentSessionID) -> String? { - lock.lock(); defer { lock.unlock() } - return index[id]?.resume ?? index[id]?.conversation.compactedSummary - } - - func upsert(_ session: ParkedSession, conversation: AgentConversation) throws { - if failWrites { throw ParkError.persistFailed(detail: "forced") } - lock.lock(); defer { lock.unlock() } - let resume = index[session.id]?.resume - index[session.id] = ParkedRecord(row: session, conversation: conversation, resume: resume) - } - - func upsertRow(_ session: ParkedSession) throws { - if failWrites { throw ParkError.persistFailed(detail: "forced") } - lock.lock() - if var record = index[session.id] { - record.row = session - index[session.id] = record - lock.unlock() - } else { - lock.unlock() - try upsert(session, conversation: AgentConversation(id: session.id, title: session.title, messages: [])) - } - } - - func remove(_ id: AgentSessionID) throws { - lock.lock(); defer { lock.unlock() } - index[id] = nil - } - - func persistResume(_ id: AgentSessionID, resume: String) throws { - if failWrites { throw ParkError.persistFailed(detail: "forced") } - lock.lock(); defer { lock.unlock() } - guard var record = index[id] else { throw ParkError.resumeMissing } - record.resume = resume - index[id] = record - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/PromptTemplate.swift b/Sources/ThreeFingerSwitcher/AI/PromptTemplate.swift deleted file mode 100644 index cb25a20..0000000 --- a/Sources/ThreeFingerSwitcher/AI/PromptTemplate.swift +++ /dev/null @@ -1,94 +0,0 @@ -import Foundation - -/// The captured context a command fires against (design D5): everything a prompt template can -/// reference, gathered at fire time from the front app and the acquired input. Pure value type so -/// templating is unit-testable without a running app. -struct FireContext: Equatable, Sendable { - /// The name of the app that was frontmost when the launcher opened (`{app}`). - var capturedAppName: String? - /// The acquired input text (selection / clipboard / nothing). `{input}`. - var inputText: String? - /// The fire-time date/time (`{date}`). - var date: Date - /// The front document/page URL when the app exposes one (`{url}`); often nil. - var url: URL? - - init(capturedAppName: String? = nil, - inputText: String? = nil, - date: Date = Date(), - url: URL? = nil) { - self.capturedAppName = capturedAppName - self.inputText = inputText - self.date = date - self.url = url - } -} - -/// Resolves a command's prompt template against a `FireContext` (spec: "Prompt template token -/// resolution"). Supported tokens: `{input}`, `{date}`, `{app}`, `{url}`, `{lang}`. -/// -/// Resolution rules (must never fail a command): -/// - Known tokens are replaced by their context value. -/// - A missing `{app}` / `{url}` (and an empty `{input}`) resolve to the EMPTY string, not an error. -/// - `{lang}` resolves to the command's ACTIVE runtime language; a command with no language parameter -/// passes `activeLanguage == nil` and so `{lang}` resolves to the EMPTY string (never fails). -/// - UNKNOWN tokens (e.g. `{foo}`) are left untouched verbatim, so a typo doesn't silently vanish. -enum PromptTemplate { - - /// The token names this resolver understands (everything else is passed through verbatim). - static let knownTokens: Set = ["input", "date", "app", "url", "lang"] - - /// Resolve `template` against `context`. `dateStyle`/`timeStyle` shape `{date}` (medium/short by - /// default — human-readable, locale-aware). `activeLanguage` supplies `{lang}` (nil ⇒ empty). - static func resolve(_ template: String, - with context: FireContext, - activeLanguage: String? = nil, - dateStyle: DateFormatter.Style = .medium, - timeStyle: DateFormatter.Style = .short) -> String { - let values: [String: String] = [ - "input": context.inputText ?? "", - "date": formattedDate(context.date, dateStyle: dateStyle, timeStyle: timeStyle), - "app": context.capturedAppName ?? "", - "url": context.url?.absoluteString ?? "", - "lang": activeLanguage ?? "" - ] - return substitute(template, values: values) - } - - /// Replace `{name}` occurrences for the names in `values`; leave any other `{...}` untouched. - /// Single-pass scan so a substituted value can't itself be re-interpreted as a token. - private static func substitute(_ template: String, values: [String: String]) -> String { - var out = "" - out.reserveCapacity(template.count) - var i = template.startIndex - while i < template.endIndex { - let ch = template[i] - if ch == "{", let close = template[i...].firstIndex(of: "}") { - let nameStart = template.index(after: i) - let name = String(template[nameStart.. String { - let f = DateFormatter() - f.dateStyle = dateStyle - f.timeStyle = timeStyle - return f.string(from: date) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/RuntimeParameter.swift b/Sources/ThreeFingerSwitcher/AI/RuntimeParameter.swift deleted file mode 100644 index 55184f3..0000000 --- a/Sources/ThreeFingerSwitcher/AI/RuntimeParameter.swift +++ /dev/null @@ -1,104 +0,0 @@ -import Foundation - -/// A parameter chosen at FIRE TIME rather than baked into a command's prompt template (spec: -/// "Runtime-adjustable command parameter"). v1 ships exactly one parameter — a target language for -/// translate-style commands — surfaced as an in-canvas dropdown and persisted per command so the -/// next run defaults to the last choice. It is an enum so a future runtime parameter (tone, length, -/// format…) is additive without touching call sites that only care about `.language`. -enum RuntimeParameter: Codable, Equatable, Sendable { - /// A choice from a declared option set whose value is substituted into the `{lang}` token. - /// `default` is the cold-start value used until the user picks one (then per-command persistence - /// overrides it); `options` is the dropdown's own list, so a translate command offers human - /// languages while a "rewrite in language" command offers programming languages. Use the - /// `.language(default:)` / `.codeLanguage(default:)` factories at call sites for the common cases. - case languageChoice(default: String, options: [String]) - - /// A human-language target parameter (the dropdown offers `AILanguages.all`). - static func language(default def: String) -> RuntimeParameter { - .languageChoice(default: def, options: AILanguages.all) - } - - /// A programming-language target parameter (the dropdown offers `ProgrammingLanguages.all`). - static func codeLanguage(default def: String) -> RuntimeParameter { - .languageChoice(default: def, options: ProgrammingLanguages.all) - } - - /// The declared default value, when this is a `.languageChoice` parameter (else nil). - var languageDefault: String? { - if case let .languageChoice(d, _) = self { return d } - return nil - } - - /// The declared option set the in-canvas dropdown should offer (else nil). - var options: [String]? { - if case let .languageChoice(_, o) = self { return o } - return nil - } -} - -// MARK: - Backward-compatible Codable -// -// The case was renamed from the legacy `language(default:)` to `languageChoice(default:options:)`. A -// hand-rolled Codable keeps OLD persisted commands decodable (a band that stored `{"language": …}` -// before this change must still load — `AICommand` decodes its `runtimeParameter` with no migration -// pass) while always ENCODING the new shape. A legacy payload (no `options`) defaults to the human -// `AILanguages.all` list, matching the only commands that shipped a stored language (translate-style). -extension RuntimeParameter { - private enum CodingKeys: String, CodingKey { case languageChoice, language } - private enum PayloadKeys: String, CodingKey { case `default`, options } - - init(from decoder: Decoder) throws { - let container = try decoder.container(keyedBy: CodingKeys.self) - if let nested = try? container.nestedContainer(keyedBy: PayloadKeys.self, forKey: .languageChoice) { - let def = try nested.decode(String.self, forKey: .default) - let options = try nested.decodeIfPresent([String].self, forKey: .options) ?? AILanguages.all - self = .languageChoice(default: def, options: options) - } else if let nested = try? container.nestedContainer(keyedBy: PayloadKeys.self, forKey: .language) { - // Legacy shape: only `default` was stored; default the options to the human-language list. - let def = try nested.decode(String.self, forKey: .default) - self = .languageChoice(default: def, options: AILanguages.all) - } else { - throw DecodingError.dataCorrupted(.init( - codingPath: decoder.codingPath, - debugDescription: "Unrecognized RuntimeParameter payload")) - } - } - - func encode(to encoder: Encoder) throws { - var container = encoder.container(keyedBy: CodingKeys.self) - switch self { - case let .languageChoice(def, options): - var nested = container.nestedContainer(keyedBy: PayloadKeys.self, forKey: .languageChoice) - try nested.encode(def, forKey: .default) - try nested.encode(options, forKey: .options) - } - } -} - -/// The fixed list of languages the in-canvas dropdown offers (spec: "a fixed list of languages — no -/// free-form text entry, keyboardless"). The displayed name IS the value substituted into `{lang}`, -/// so the model sees a plain language name (e.g. "Hebrew"). -enum AILanguages { - /// Common languages, English first. Curated rather than exhaustive so the dropdown stays scrubbable. - static let all: [String] = [ - "English", "Hebrew", "Spanish", "French", "German", "Italian", "Portuguese", - "Dutch", "Arabic", "Russian", "Ukrainian", "Polish", "Turkish", - "Chinese (Simplified)", "Japanese", "Korean", "Hindi" - ] - - /// The dropdown options guaranteed to include `language` — so a persisted/declared default that - /// isn't in the canonical list still appears and stays selectable (it is prepended if new). - static func including(_ language: String) -> [String] { - all.contains(language) ? all : [language] + all - } -} - -/// The fixed list of PROGRAMMING languages the dropdown offers for code-rewrite commands (e.g. "Rewrite -/// in Language"). The displayed name IS the value substituted into `{lang}`, so the model sees a plain -/// language name (e.g. "Rust"). -enum ProgrammingLanguages { - static let all: [String] = [ - "Python", "Swift", "JavaScript", "TypeScript", "Rust", "Go", "Java", - "C", "C++", "C#", "Ruby", "Kotlin", "PHP", "SQL", "Shell" - ] -} diff --git a/Sources/ThreeFingerSwitcher/AI/SelectionProviding.swift b/Sources/ThreeFingerSwitcher/AI/SelectionProviding.swift deleted file mode 100644 index b11a1dc..0000000 --- a/Sources/ThreeFingerSwitcher/AI/SelectionProviding.swift +++ /dev/null @@ -1,55 +0,0 @@ -import Foundation - -/// The result of a screen-region capture (spec: "Failure is never silent" + "A permission failure -/// points to the fix"). Distinguishes a NAMED Screen-Recording permission gap — so the canvas can tell -/// the user which permission to grant — from an ordinary unavailable (no display / cancelled), which -/// the executor treats as plain "no input". A successful capture carries the encoded bytes. -enum ScreenCaptureOutcome: Equatable { - /// Encoded image bytes (e.g. PNG) ready for a vision request. - case captured(Data) - /// Screen Recording is not granted — the user must enable it; surfaced as a clear `.failed`, not - /// silently as "no input". - case permissionDenied - /// No display / capture failed / cancelled — treated as "no input" (not a permission problem). - case unavailable -} - -/// The seam the executor uses to read input from, and write output into, the captured front app -/// (design D3). The concrete `SelectionService` — AX selected-text read/replace with a ⌘C-restore -/// fallback, and ScreenCaptureKit for screen regions — is a LATER slice; the executor depends only -/// on this protocol so the slices stay decoupled and the pipeline is testable with a fake. -/// -/// `@MainActor` because the real implementation touches AppKit (`NSPasteboard`, AX, capture); the -/// executor that drives it is main-actor too, so this keeps the hop count down. -@MainActor -protocol SelectionProviding { - /// The front app's currently selected text (AX, no clipboard clobber), or nil when none is - /// readable. The executor applies the selection→clipboard fallback itself. - func readSelectedText() async -> String? - - /// The current clipboard text, or nil when the clipboard holds no text. - func readClipboardText() -> String? - - /// The current clipboard IMAGE as PNG bytes for a vision command, or nil when the clipboard holds - /// no image (or undecodable image data). Reads the LIVE pasteboard — symmetric with - /// `readClipboardText()`, not the stored clipboard history — and reuses already-held access (no new - /// permission). A nil here is a "no input" state, never a permission gap. - func readClipboardImage() -> Data? - - // NOTE: screen-region capture is NOT on this executor seam. A `screenRegion` command's image is - // captured by the interactive region picker BEFORE the canvas opens, and handed to the executor as a - // pre-supplied `ScreenCaptureOutcome` via `fire(_:screenCapture:)` — the executor never captures the - // screen itself. The region capture lives on the concrete `SelectionService.captureScreenRegion(_:)`, - // called by the picker orchestration. - - /// Replace the front app's selected text with `text` (AX set when settable, else paste-on-fire). - /// Returns whether the replace actually LANDED (not merely whether it was attempted) — a `false` - /// means the executor must report failure, never a false "Done" (spec: "Failure is never silent"). - @discardableResult - func replaceSelection(_ text: String) async -> Bool - - /// Paste `text` at the insertion point of the front app. Returns whether the paste landed (same - /// honesty contract as `replaceSelection`). - @discardableResult - func pasteAtCursor(_ text: String) async -> Bool -} diff --git a/Sources/ThreeFingerSwitcher/AI/SelectionService.swift b/Sources/ThreeFingerSwitcher/AI/SelectionService.swift deleted file mode 100644 index 756fa5c..0000000 --- a/Sources/ThreeFingerSwitcher/AI/SelectionService.swift +++ /dev/null @@ -1,484 +0,0 @@ -import AppKit -import ApplicationServices -import CoreGraphics -import ScreenCaptureKit -import Carbon.HIToolbox // IsSecureEventInputEnabled() -import os - -/// Diagnostic log for the selection read/write path. Filter in Console.app by -/// `subsystem:ThreeFingerSwitcher category:Selection` to see, per fire, which app was targeted, whether -/// Accessibility exposed a selection, and whether the ⌘C fallback landed — the trail for "it didn't -/// recognize my selection and used the clipboard." -private let selectionLog = Logger(subsystem: "ThreeFingerSwitcher", category: "Selection") - -/// The concrete `SelectionProviding` (design D3, spec `selection-io`): the AX / clipboard / screen -/// input-output primitive the `AICommandExecutor` drives. It reads the front app's selection via -/// Accessibility first (no clipboard clobber), with a ⌘C-with-restore fallback; it writes a result -/// back by setting `AXSelectedText` when settable, else by a ⌘V paste-with-restore; and it captures a -/// screen region as PNG bytes via ScreenCaptureKit for vision commands. -/// -/// "Front app" is the app captured when the launcher opened (the overlay is non-activating, so it -/// still holds focus) — injected as `frontAppProvider`, mirroring `LaunchService`. The pasteboard is -/// injected behind `PasteboardAccess` so the save→mutate→restore round-trip is unit-testable headless -/// (the real AX, CGEvent synthesis, and ScreenCaptureKit paths are verified on-device — see the -/// manual-test checklist). -/// -/// `@MainActor` to satisfy `SelectionProviding` and because it touches AppKit / AX throughout. -@MainActor -final class SelectionService: SelectionProviding { - - /// The app that was frontmost when the launcher opened. Returns nil when it resolves to our own - /// process (the overlay), so we never act into ourselves — mirrors `LaunchService.frontApp()`. - private let frontAppProvider: () -> NSRunningApplication? - /// The pasteboard the read-fallback and paste paths save/mutate/restore. Injected for tests. - private let pasteboard: PasteboardAccess - /// Whether Screen Recording is granted (gates `captureScreenRegion`). Injected for tests; defaults - /// to the same preflight `PermissionsService` / `ThumbnailService` use. - private let screenRecordingGranted: () -> Bool - /// How long the ⌘C fallback polls `changeCount` before giving up. Short and bounded so a missed - /// copy never hangs the fire. - private let copyTimeout: TimeInterval - /// Re-assert the (non-activating) front app and synthesize ⌘V into it. Injected so the paste - /// pasteboard round-trip is testable WITHOUT activating a real app or firing real keystrokes; - /// defaults to the real activate-then-⌘V (mirrors `LaunchService.pasteEntry`). - private let pasteKeystroke: (NSRunningApplication) -> Void - - init(frontAppProvider: @escaping () -> NSRunningApplication? = { NSWorkspace.shared.frontmostApplication }, - pasteboard: PasteboardAccess? = nil, - screenRecordingGranted: @escaping () -> Bool = { CGPreflightScreenCaptureAccess() }, - copyTimeout: TimeInterval = 0.5, - pasteKeystroke: ((NSRunningApplication) -> Void)? = nil) { - self.frontAppProvider = frontAppProvider - // Built here (not as a default arg) so the main-actor `SystemPasteboard` init isn't evaluated - // in the nonisolated default-argument context. - self.pasteboard = pasteboard ?? SystemPasteboard() - self.screenRecordingGranted = screenRecordingGranted - self.copyTimeout = copyTimeout - self.pasteKeystroke = pasteKeystroke ?? { app in - app.activate(options: []) - Self.synthesizeCommandKeystroke(0x09) // ⌘V via the session tap → the frontmost (activated) app - } - } - - // MARK: - Read selection - - /// The front app's selected text. Accessibility is tried FIRST (non-destructive, instant, no focus - /// change, no clipboard clobber) — `AXFocusedUIElement → AXSelectedText`. When AX yields nothing, - /// fall back to a ⌘C-with-restore capture. Returns nil (never "") when no selection can be read by - /// either path, so the executor's "no input" handling fires rather than running the model on empty. - func readSelectedText() async -> String? { - guard let app = frontApp() else { - selectionLog.error("readSelectedText: no captured front app (resolves to self/nil) → nil (would fall back to clipboard)") - return nil - } - let name = app.localizedName ?? "?" - if let ax = Self.normalized(axSelectedText(pid: app.processIdentifier)) { - selectionLog.notice("readSelectedText[\(name, privacy: .public)]: AX exposed a selection (\(ax.count) chars)") - return ax // AX exposed a real selection — return it without touching the clipboard. - } - // AX didn't expose the selection (many apps don't): synthesize ⌘C, read, then restore. - selectionLog.notice("readSelectedText[\(name, privacy: .public)]: AX exposed nothing → trying the ⌘C fallback") - let copied = await copyWithRestore(app: app) - selectionLog.notice("readSelectedText[\(name, privacy: .public)]: ⌘C fallback captured \(copied?.count ?? -1) chars (-1 = nothing → clipboard fallback)") - return copied - } - - /// The CURRENT clipboard string (no synthesis). This is the executor's higher-level "empty - /// selection → existing clipboard" fallback — deliberately distinct from the ⌘C capture in - /// `readSelectedText`. nil when the clipboard holds no (non-empty) text. - func readClipboardText() -> String? { - Self.normalized(pasteboard.string()) - } - - /// The CURRENT clipboard image as PNG bytes for a vision command (spec: "Read the current clipboard - /// image as vision input"). Reads the live pasteboard's best image representation (PNG, else TIFF) - /// and normalizes it to PNG so the runtime always gets the PNG bytes its image processor decodes. - /// nil when the clipboard holds no image, or its bytes can't be decoded — so the executor reports - /// "no input" rather than running the model on nothing. No synthesis, no clipboard clobber. - func readClipboardImage() -> Data? { - Self.normalizedPNG(from: pasteboard.imageData()) - } - - // MARK: - Write output - - /// Replace the front app's selected text with `text`: set `AXSelectedText` on the focused element - /// when it is settable (instant, in-place, no clipboard touch) and report true; otherwise fall - /// back to a ⌘V paste-with-restore and report whether that was applied. Returns false when there - /// is no front app to act into. - @discardableResult - func replaceSelection(_ text: String) async -> Bool { - guard let app = frontApp() else { return false } - let pid = app.processIdentifier - - if Self.shouldUseAX(focusedElementSettable: axSelectedTextIsSettable(pid: pid)) { - if setAXSelectedText(text, pid: pid) { return true } - // Settable check passed but the set failed (rare) — fall through to paste. - } - return await paste(text, into: app) - } - - /// Paste `text` at the insertion point of the front app (set pasteboard → ⌘V → restore prior - /// clipboard). No AX-set path: a caret with no selection isn't a settable `AXSelectedText`. Returns - /// whether the paste was applied (false when there is no front app to act into, or `text` is empty). - @discardableResult - func pasteAtCursor(_ text: String) async -> Bool { - guard let app = frontApp() else { return false } - return await paste(text, into: app) - } - - /// Deliver a Files-band entry to the captured front app (`files-contextual-delivery`): write the - /// **dual-representation** item (the path string AND the file reference), re-assert the (non-activating) - /// front app, synthesize ⌘V, then RESTORE the user's prior clipboard — so a text target receives the - /// **path** and a Finder window receives the **file**, with no context detection on our side and the - /// user's clipboard left exactly as it was. Returns whether a paste was attempted: `false` only when - /// there is no front app to act into, which the caller surfaces as a bounded failure (never a false - /// "Done"). The keystroke landing itself can't be observed, so a `true` means *attempted*, not confirmed. - @discardableResult - func deliverFile(url: URL, path: String) async -> Bool { - guard let app = frontApp() else { return false } - let saved = pasteboard.snapshot() - pasteboard.setFileDelivery(url: url, path: path) - // Give activation a beat to settle before the keystroke (matches `paste`). - try? await Task.sleep(nanoseconds: 40_000_000) - pasteKeystroke(app) // re-assert front app + synthesize ⌘V (injectable for tests) - // Let the paste consume the pasteboard before we restore the prior contents. - try? await Task.sleep(nanoseconds: 80_000_000) - pasteboard.restore(saved) - return true - } - - // MARK: - Screen capture - - /// Capture the designated screen rectangle as PNG bytes for a vision command (spec `screen-region-picker` - /// / "Screen-region capture for vision input"), reusing the held Screen Recording permission. `rect` is - /// the picker's selection in **Cocoa global (bottom-left)** coordinates; capture is confined to the - /// display under that rectangle and excludes the app's own overlay windows. Returns `.permissionDenied` - /// when Screen Recording is not granted (a named gap, surfaced as a clear `.failed`, not "no input"); - /// `.unavailable` for an empty rect / no matching display / capture failure (plain "no input"). A - /// cancelled pick never reaches here — the picker resolves a cancel without calling capture. - /// - /// v1 scopes a pick to a single display (the one containing the rect); a cross-display rectangle is - /// clamped to that display. Capture is at point resolution (sufficient for the vision model). - func captureScreenRegion(_ rect: CGRect) async -> ScreenCaptureOutcome { - guard screenRecordingGranted() else { return .permissionDenied } - guard rect.width >= 1, rect.height >= 1 else { return .unavailable } - do { - let content = try await SCShareableContent.excludingDesktopWindows(false, onScreenWindowsOnly: true) - // The display whose Cocoa frame contains the rect's center (single-display v1), else the first. - guard let (display, frameCocoa) = displayContaining(rect, in: content.displays) - ?? firstDisplay(in: content.displays) else { return .unavailable } - let local = Self.displayLocalRect(forGlobalCocoa: rect, displayFrameCocoa: frameCocoa) - let config = SCStreamConfiguration() - config.sourceRect = local // crop to the designated rectangle (display-local, top-left) - config.width = Int(local.width.rounded()) - config.height = Int(local.height.rounded()) - config.showsCursor = false - // Exclude our own non-activating overlay (incl. the picker) so it never appears in the capture. - let ours = content.windows.filter { $0.owningApplication?.processID == getpid() } - let filter = SCContentFilter(display: display, excludingApplications: [], exceptingWindows: ours) - let cgImage = try await SCScreenshotManager.captureImage(contentFilter: filter, configuration: config) - guard let png = Self.pngData(from: cgImage) else { return .unavailable } - return .captured(png) - } catch { - return .unavailable // no display, capture failed, or cancelled → plain "no input". - } - } - - /// The `SCDisplay` whose Cocoa screen frame contains `rect`'s center, paired with that frame, or nil - /// when none matches. Effectful only in resolving each display's `NSScreen` frame. - private func displayContaining(_ rect: CGRect, in displays: [SCDisplay]) -> (SCDisplay, CGRect)? { - let center = CGPoint(x: rect.midX, y: rect.midY) - for display in displays { - if let frame = Self.cocoaFrame(forDisplayID: display.displayID), frame.contains(center) { - return (display, frame) - } - } - return nil - } - - /// The first display paired with its Cocoa frame (fallback when no display contains the rect center). - private func firstDisplay(in displays: [SCDisplay]) -> (SCDisplay, CGRect)? { - guard let display = displays.first, let frame = Self.cocoaFrame(forDisplayID: display.displayID) else { - return nil - } - return (display, frame) - } - - // MARK: - Pure decision helpers (unit-tested; no system access) - - /// Normalize an optional read into "real text or nothing": trims, and treats empty / whitespace- - /// only as no-selection (nil). Both AX and clipboard reads pass through this so a blank string is - /// never mistaken for input. Returns the ORIGINAL (untrimmed) string when it has non-whitespace - /// content, so leading/trailing spacing the user selected is preserved for the model. - nonisolated static func normalized(_ raw: String?) -> String? { - guard let raw, !raw.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty else { return nil } - return raw - } - - /// Pure branch decision for `replaceSelection`: use the AX set path when the focused element's - /// selected-text attribute is settable, else fall back to paste. Factored out so the - /// AX-else-paste choice is testable without an AX element. - nonisolated static func shouldUseAX(focusedElementSettable: Bool) -> Bool { - focusedElementSettable - } - - /// Pure: PNG bytes from a captured `CGImage`, or nil if encoding fails. Kept separate from the - /// effectful capture so the encode step is isolable. - nonisolated static func pngData(from cgImage: CGImage) -> Data? { - let rep = NSBitmapImageRep(cgImage: cgImage) - return rep.representation(using: .png, properties: [:]) - } - - /// Pure: convert a rectangle in Cocoa **global (bottom-left)** screen coordinates into the coordinate - /// space `SCStreamConfiguration.sourceRect` expects — relative to the display's **top-left** origin — - /// given that display's Cocoa frame. The x offset is within the display; the y is flipped (distance - /// from the display's top). Factored out so the handedness flip is unit-tested without a real display. - nonisolated static func displayLocalRect(forGlobalCocoa rect: CGRect, displayFrameCocoa: CGRect) -> CGRect { - CGRect(x: rect.minX - displayFrameCocoa.minX, - y: displayFrameCocoa.maxY - rect.maxY, - width: rect.width, height: rect.height) - } - - /// The Cocoa screen frame (bottom-left) for a `CGDirectDisplayID`, by matching `NSScreen`'s - /// `NSScreenNumber`. Effectful (reads `NSScreen.screens`); nil when no screen matches. - nonisolated static func cocoaFrame(forDisplayID id: CGDirectDisplayID) -> CGRect? { - let key = NSDeviceDescriptionKey("NSScreenNumber") - return NSScreen.screens.first { ($0.deviceDescription[key] as? NSNumber)?.uint32Value == id }?.frame - } - - /// Pure: normalize the pasteboard's best image representation to PNG bytes. Returns nil for no data; - /// passes already-PNG bytes through untouched (by the 8-byte PNG signature, no re-encode); otherwise - /// decodes via `NSBitmapImageRep` (e.g. a TIFF-only clipboard) and re-encodes to PNG. nil when the - /// bytes can't be decoded — so undecodable clipboard data surfaces as "no input", never garbage to - /// the model. Isolated from the effectful read so the format handling is unit-tested headless. - nonisolated static func normalizedPNG(from raw: Data?) -> Data? { - guard let raw, !raw.isEmpty else { return nil } - // PNG signature: 89 50 4E 47 0D 0A 1A 0A — already PNG, hand it back without a re-encode. - let pngSignature: [UInt8] = [0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A] - if raw.count >= 8, Array(raw.prefix(8)) == pngSignature { return raw } - guard let rep = NSBitmapImageRep(data: raw) else { return nil } // e.g. TIFF → decode… - return rep.representation(using: .png, properties: [:]) // …and re-encode to PNG. - } - - // MARK: - Accessibility (effectful; verified on-device) - - /// The selected text of the front app's focused UI element via Accessibility, or nil when AX can't - /// resolve a focused element or a selected-text attribute. Reuses the held Accessibility grant; no - /// clipboard touch. - private func axSelectedText(pid: pid_t) -> String? { - guard let focused = focusedElement(pid: pid) else { return nil } - return axString(focused, kAXSelectedTextAttribute as String) - } - - /// Whether the focused element's `AXSelectedText` is settable (⇒ the AX replace path will work). - private func axSelectedTextIsSettable(pid: pid_t) -> Bool { - guard let focused = focusedElement(pid: pid) else { return false } - var settable: DarwinBoolean = false - let err = AXUIElementIsAttributeSettable(focused, kAXSelectedTextAttribute as CFString, &settable) - return err == .success && settable.boolValue - } - - /// Set the focused element's `AXSelectedText` to `text`, replacing the selection in place. Returns - /// whether the set succeeded. - private func setAXSelectedText(_ text: String, pid: pid_t) -> Bool { - guard let focused = focusedElement(pid: pid) else { return false } - return AXUIElementSetAttributeValue(focused, kAXSelectedTextAttribute as CFString, - text as CFString) == .success - } - - /// The front app's focused UI element (`AXFocusedUIElement` on the AX application element), or nil. - private func focusedElement(pid: pid_t) -> AXUIElement? { - let appEl = AXUIElementCreateApplication(pid) - var value: CFTypeRef? - guard AXUIElementCopyAttributeValue(appEl, kAXFocusedUIElementAttribute as CFString, &value) == .success, - let v = value, CFGetTypeID(v) == AXUIElementGetTypeID() else { return nil } - return (v as! AXUIElement) - } - - // MARK: - Clipboard fallback / paste (effectful; verified on-device) - - /// ⌘C-with-restore capture of the selection: save the pasteboard, **re-assert the captured app as - /// frontmost** (a synthesized ⌘C posted to an app that isn't active doesn't land — e.g. Terminal — so - /// we `activate` + settle first, mirroring the working ⌘V paste path), synthesize ⌘C, poll - /// `changeCount` until it advances (bounded by `copyTimeout`), read the result, then RESTORE the saved - /// clipboard so the user's contents are untouched (spec: "Fallback reads via copy and restores the - /// clipboard"). nil when nothing was captured. If **Secure Keyboard Entry** is on (Terminal ▸ Secure - /// Keyboard Entry, or any app that enabled it) synthesized keys are blocked system-wide and this can't - /// succeed — that case is logged so it is diagnosable rather than a silent clipboard fall-through. - private func copyWithRestore(app: NSRunningApplication) async -> String? { - if IsSecureEventInputEnabled() { - selectionLog.error("copyWithRestore: Secure Keyboard Entry is ON — synthesized ⌘C is blocked system-wide (e.g. Terminal ▸ Secure Keyboard Entry). Turn it off to read a selection by copy.") - } - let saved = pasteboard.snapshot() - let before = pasteboard.changeCount - // Re-assert the captured app as frontmost so the ⌘C is delivered to it, then let activation settle - // (matches the paste path's activate + 40ms). Posting ⌘C without this didn't land for Terminal. - app.activate(options: []) - try? await Task.sleep(nanoseconds: 60_000_000) // let activation settle before the HID keystroke - // Confirm the target is actually frontmost at post time (the HID ⌘C goes to whoever is frontmost). - let front = NSWorkspace.shared.frontmostApplication - selectionLog.notice("copyWithRestore[\(app.localizedName ?? "?", privacy: .public)]: pre-⌘C frontmost=\(front?.localizedName ?? "nil", privacy: .public) isTarget=\(front?.processIdentifier == app.processIdentifier)") - Self.synthesizeCommandKeystroke(0x08) // ⌘C (C = 0x08) via the HID tap → the frontmost (activated) app - - let captured = await pollForChange(after: before) - let result = captured ? Self.normalized(pasteboard.string()) : nil - selectionLog.notice("copyWithRestore[\(app.localizedName ?? "?", privacy: .public)]: activated + ⌘C; changeCount advanced=\(captured) (before=\(before)) → \(result?.count ?? -1) chars") - - pasteboard.restore(saved) // ALWAYS restore — even on a missed copy — so we never clobber. - return result - } - - /// Paste `text` into `app` via the existing paste-on-fire mechanism: save the prior clipboard, put - /// `text` on the pasteboard, re-assert the (non-activating) front app, synthesize ⌘V, then restore - /// the prior clipboard. Returns whether a paste was attempted (false only with empty text). Mirrors - /// `LaunchService.pasteEntry`'s write-then-⌘V approach. - @discardableResult - private func paste(_ text: String, into app: NSRunningApplication) async -> Bool { - guard !text.isEmpty else { return false } - let saved = pasteboard.snapshot() - pasteboard.setString(text) - // Give activation a beat to settle before the keystroke (matches LaunchService's 0.04s). - try? await Task.sleep(nanoseconds: 40_000_000) - pasteKeystroke(app) // re-assert front app + synthesize ⌘V (injectable for tests) - // Let the paste consume the pasteboard before we restore the prior contents. - try? await Task.sleep(nanoseconds: 80_000_000) - pasteboard.restore(saved) - return true - } - - /// Poll `changeCount` until it moves past `before`, bounded by `copyTimeout`. Returns whether it - /// advanced (⇒ the copy produced something) within the budget. - private func pollForChange(after before: Int) async -> Bool { - let deadline = Date().addingTimeInterval(copyTimeout) - while Date() < deadline { - if pasteboard.changeCount != before { return true } - try? await Task.sleep(nanoseconds: 15_000_000) // 15ms between polls - } - return pasteboard.changeCount != before - } - - // MARK: - Front app / key synthesis - - /// The captured front app, or nil when it resolves to our own process — never act into ourselves. - private func frontApp() -> NSRunningApplication? { - let app = frontAppProvider() - return (app?.processIdentifier == getpid()) ? nil : app - } - - /// Minimal CGEvent key synthesis to a specific process. A private copy rather than reaching into - /// `LaunchService.postKey` (which is private) — keeps shared-file edits at zero (see the slice - /// notes). Requires the held Accessibility permission. Mirrors `LaunchService.postKey(_:flags:toPid:)`. - nonisolated static func synthesizeKey(_ keyCode: CGKeyCode, flags: CGEventFlags, toPid pid: pid_t) { - let src = CGEventSource(stateID: .hidSystemState) - guard let down = CGEvent(keyboardEventSource: src, virtualKey: keyCode, keyDown: true), - let up = CGEvent(keyboardEventSource: src, virtualKey: keyCode, keyDown: false) else { return } - down.flags = flags; up.flags = flags - down.postToPid(pid); up.postToPid(pid) - } - - /// Synthesize a ⌘-modified keystroke into the **HID event tap** (`.cghidEventTap`) — the lowest-level, - /// most "real" input path, where a target app's **menu key-equivalent** (e.g. Terminal's Copy/Paste) - /// reliably matches. Uses a clean `.hidSystemState` source and stamps `.maskCommand` on the key events - /// themselves, with **NO** separate ⌘-key (0x37) events — that, plus `.combinedSessionState`, desyncs - /// the modifier bookkeeping and the shortcut never matches (the bug this replaces). This mirrors the - /// app's proven working chord synthesis, `AppCoordinator.sendLogOutKeystroke` (a ⇧⌘Q that logs out). - /// The caller `activate`s the target first so the HID event reaches it. - /// - /// Ruled out on-device before landing here: `postToPid` ⌘C (a PID-delivered event isn't matched as a - /// menu shortcut, even with the app activated), and a `.cgSessionEventTap` post with hand-injected - /// ⌘-key events (malformed modifier state). The `.cghidEventTap` + flags-only recipe is the one the - /// codebase already relies on for a real system-level keystroke. - nonisolated static func synthesizeCommandKeystroke(_ keyCode: CGKeyCode) { - let src = CGEventSource(stateID: .hidSystemState) - guard let down = CGEvent(keyboardEventSource: src, virtualKey: keyCode, keyDown: true), - let up = CGEvent(keyboardEventSource: src, virtualKey: keyCode, keyDown: false) else { return } - down.flags = .maskCommand - up.flags = .maskCommand - down.post(tap: .cghidEventTap) - up.post(tap: .cghidEventTap) - } -} - -// MARK: - Pasteboard seam - -/// The thin slice of `NSPasteboard` the read-fallback and paste paths need, abstracted so the -/// save→mutate→restore round-trip is unit-testable headless (the real `NSPasteboard` can't be -/// snapshotted deterministically in CI). `snapshot()`/`restore(_:)` capture and put back the full set -/// of items so a non-text clipboard (a copied image, a password) survives a ⌘C/⌘V fallback unchanged. -@MainActor -protocol PasteboardAccess { - /// The current change count (advances on every write — the signal a ⌘C produced something). - var changeCount: Int { get } - /// The current plain-text string, or nil when none. - func string() -> String? - /// The current best image representation as raw bytes (PNG when present, else TIFF), or nil when - /// the clipboard holds no image. Normalization to PNG happens above this seam (in `SelectionService`), - /// so this stays a thin pasteboard read and the format handling is unit-tested headless. - func imageData() -> Data? - /// Replace the pasteboard with a single plain-text string (for the paste path). - func setString(_ text: String) - /// Replace the pasteboard with a single item carrying BOTH a file reference and its path string, so a - /// text target consumes the path and a Finder window consumes the file (`files-contextual-delivery`). - func setFileDelivery(url: URL, path: String) - /// An opaque snapshot of the full pasteboard contents, for later restore. - func snapshot() -> PasteboardSnapshot - /// Restore a previously captured snapshot, leaving the user's clipboard as it was. - func restore(_ snapshot: PasteboardSnapshot) -} - -/// An opaque, replayable copy of every pasteboard item's typed representations. Holding the bytes (not -/// references) is what lets the fallback restore a clipboard that held, e.g., a password or an image. -struct PasteboardSnapshot: Equatable { - /// One entry per pasteboard item; each maps a type identifier to its bytes. - let items: [[String: Data]] -} - -/// The production `PasteboardAccess`, backed by `NSPasteboard.general`. Snapshot/restore copy the -/// per-item typed bytes the same way `ClipboardMonitor` / `LaunchService` round-trip representations. -struct SystemPasteboard: PasteboardAccess { - private let pb: NSPasteboard - init(_ pb: NSPasteboard = .general) { self.pb = pb } - - var changeCount: Int { pb.changeCount } - - func string() -> String? { pb.string(forType: .string) } - - func imageData() -> Data? { pb.data(forType: .png) ?? pb.data(forType: .tiff) } - - func setString(_ text: String) { - pb.clearContents() - pb.setString(text, forType: .string) - } - - func setFileDelivery(url: URL, path: String) { - pb.clearContents() - let item = NSPasteboardItem() - item.setString(path, forType: .string) // text targets (terminal/editor) consume this - item.setString(url.absoluteString, forType: .fileURL) // Finder consumes this (copies the file in) - pb.writeObjects([item]) - } - - func snapshot() -> PasteboardSnapshot { - let items: [[String: Data]] = (pb.pasteboardItems ?? []).map { item in - var reps: [String: Data] = [:] - for type in item.types { - if let data = item.data(forType: type) { reps[type.rawValue] = data } - } - return reps - } - return PasteboardSnapshot(items: items) - } - - func restore(_ snapshot: PasteboardSnapshot) { - pb.clearContents() - guard !snapshot.items.isEmpty else { return } - let objects: [NSPasteboardItem] = snapshot.items.map { reps in - let item = NSPasteboardItem() - for (uti, data) in reps { - item.setData(data, forType: NSPasteboard.PasteboardType(uti)) - } - return item - } - pb.writeObjects(objects) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Skills/ClaudeHandoffConfig.swift b/Sources/ThreeFingerSwitcher/AI/Skills/ClaudeHandoffConfig.swift deleted file mode 100644 index 08e2af9..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Skills/ClaudeHandoffConfig.swift +++ /dev/null @@ -1,30 +0,0 @@ -import Foundation - -/// Per-skill configuration for handing a task off to Claude Code (blueprint §3.8). DEFINED HERE because -/// `ai-skills-as-files` (Wave 3) needs to carry it on a skill manifest, but it is logically OWNED by -/// `ai-claude-handoff` (Wave 4), which uses this type for the `launch_claude` tool + the budget cap. -/// Wave 4 USES this definition; it does not redefine it (single definition in Core). -struct ClaudeHandoffConfig: Codable, Equatable, Sendable { - /// Whether a handoff for this skill runs automatically or asks each time. Default `.confirm` (money). - var confirmMode: HandoffConfirmMode - /// The folder Claude is launched in (nil = the contextual/current folder). - var folder: String? - /// A starting prompt handed to Claude on launch. - var startingPrompt: String? - /// A per-day budget cap on automatic handoffs (nil = use the global cap). - var maxPerDay: Int? - - init(confirmMode: HandoffConfirmMode = .confirm, folder: String? = nil, - startingPrompt: String? = nil, maxPerDay: Int? = nil) { - self.confirmMode = confirmMode - self.folder = folder - self.startingPrompt = startingPrompt - self.maxPerDay = maxPerDay - } -} - -/// Whether a Claude handoff is automatic (no per-call confirm, budget-capped) or confirmed each time. -enum HandoffConfirmMode: String, Codable, Equatable, Sendable { - case auto - case confirm -} diff --git a/Sources/ThreeFingerSwitcher/AI/Skills/DocIndex.swift b/Sources/ThreeFingerSwitcher/AI/Skills/DocIndex.swift deleted file mode 100644 index ad6a718..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Skills/DocIndex.swift +++ /dev/null @@ -1,82 +0,0 @@ -import Foundation - -/// The shared retrieval contract (blueprint §3.4). OWNED by `ai-skills-as-files`; CONSUMED verbatim by -/// `ai-agent-memory` (integration fix C2 — memory contributes `IndexedDoc`s into the SAME index and -/// defines NO second retriever). Progressive disclosure: the TOC (`allSummaries`) is always cheap to show -/// the router; a doc's full body is loaded only on demand via `body(of:)`. MLX-free Core. - -/// One retrievable unit — a skill file, a memory subfile, or a memory TOC entry. -struct IndexedDoc: Codable, Equatable, Identifiable, Sendable { - let id: String // stable, path-relative id (also a skill's ToolDescriptor.name) - var title: String - var summary: String // the TOC line the retriever ranks/returns first - var keywords: [String] - var kind: DocKind - var bodyPath: URL // where the full body lives (informational; body served by the store) - var updatedAt: Date - - init(id: String, title: String, summary: String, keywords: [String], kind: DocKind, - bodyPath: URL, updatedAt: Date = Date()) { - self.id = id - self.title = title - self.summary = summary - self.keywords = keywords - self.kind = kind - self.bodyPath = bodyPath - self.updatedAt = updatedAt - } -} - -enum DocKind: String, Codable, Sendable { case skill, memoryCore, memorySubfile } - -/// The shared retriever seam. Pure + synchronous over an in-memory snapshot; the owning store bridges -/// async file IO (the Files-band sync-model + async-cache pattern). -protocol DocIndex: Sendable { - func allSummaries() -> [IndexedDoc] // the combined TOC the model always sees - func retrieve(query: String, limit: Int) -> [IndexedDoc] // ranked summaries for on-demand expansion - func body(of id: String) throws -> String // load a doc's full body when the model asks -} - -/// The pure, synchronous index over an injected `[IndexedDoc]` snapshot + a bodies map (id → full text) -/// the store populates off-main. Ranking never touches `FileManager`, so it is fully unit-testable. The -/// index is `kind`-agnostic — a combined skills + memory TOC is one `allSummaries()` over a merged -/// snapshot (blueprint C2). -struct InMemoryDocIndex: DocIndex { - private let docs: [IndexedDoc] - private let bodies: [String: String] - - init(docs: [IndexedDoc], bodies: [String: String]) { - self.docs = docs - self.bodies = bodies - } - - func allSummaries() -> [IndexedDoc] { docs } - - /// Deterministic cheap ranking: token overlap + substring over title+summary+keywords, descending, - /// stable tiebreak by id. A no-hit query returns [] (the router falls back to `allSummaries`). - func retrieve(query: String, limit: Int) -> [IndexedDoc] { - let q = Set(Self.tokenize(query)) - let qLower = query.lowercased() - func score(_ d: IndexedDoc) -> Int { - let hay = Set(Self.tokenize(d.title) + Self.tokenize(d.summary) + d.keywords.flatMap(Self.tokenize)) - let overlap = q.intersection(hay).count - let sub = d.keywords.contains { !$0.isEmpty && qLower.contains($0.lowercased()) } ? 1 : 0 - return overlap + sub - } - return docs - .map { ($0, score($0)) } - .filter { $0.1 > 0 } - .sorted { $0.1 != $1.1 ? $0.1 > $1.1 : $0.0.id < $1.0.id } - .prefix(max(0, limit)) - .map { $0.0 } - } - - func body(of id: String) throws -> String { - guard let body = bodies[id] else { throw SkillError.unreadable(detail: "No body for doc id \(id)") } - return body - } - - static func tokenize(_ s: String) -> [String] { - s.lowercased().split { !$0.isLetter && !$0.isNumber }.map(String.init).filter { $0.count > 1 } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Skills/SkillError.swift b/Sources/ThreeFingerSwitcher/AI/Skills/SkillError.swift deleted file mode 100644 index cf5199e..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Skills/SkillError.swift +++ /dev/null @@ -1,28 +0,0 @@ -import Foundation - -/// Skill load/validation failures (design D6) — a new taxonomy ONLY for the cases `RuntimeError`/ -/// `TaskError` cannot carry. Each case has a clean, user-facing `errorDescription`; raw OS/parse text -/// stays in opt-in details / logs (never the headline). `AIError.message(for:)` is extended to translate -/// this (the single translator). Skill *invocation* failures still flow through `RuntimeError`/`TaskError`. -enum SkillError: Error, Equatable, LocalizedError { - case malformedFrontMatter(detail: String) - case missingRequiredField(name: String) - case unknownEnumValue(field: String, value: String) - case duplicateID(id: String) - case unreadable(detail: String) - - var errorDescription: String? { - switch self { - case .malformedFrontMatter: - return "This skill file's front matter could not be read." - case let .missingRequiredField(name): - return "This skill file is missing a required field: \(name)." - case let .unknownEnumValue(field, value): - return "This skill file has an unrecognized \(field) value: “\(value)”." - case let .duplicateID(id): - return "Two skills share the same id “\(id)”; only the first was loaded." - case .unreadable: - return "This skill file could not be read." - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Skills/SkillFile.swift b/Sources/ThreeFingerSwitcher/AI/Skills/SkillFile.swift deleted file mode 100644 index bc706bd..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Skills/SkillFile.swift +++ /dev/null @@ -1,307 +0,0 @@ -import Foundation - -/// Parse / serialize a `.skill.md` file (design D1) — a front-matter block delimited by `---`, then a -/// markdown body that IS the prompt template. The sink (`OutputTarget`) is encoded as a flat, -/// colon-tagged string (e.g. `runTask:saveToProject:Inbox`, `sendTo:shortcut:My Shortcut`) — simpler and -/// more round-trip-stable than nested YAML (the front-matter grammar was an open question; this is the -/// chosen hand-rolled subset). Parse failures map at the boundary into `SkillError` (the store wraps them -/// into a bounded `SkillProblem`). MLX-free Core. -enum SkillFile { - - /// Parse a `.skill.md` document into a `SkillManifest` (origin `.user` — built-ins are projected from - /// the catalog, not parsed). Returns a typed `SkillError` on any malformation. - static func parse(_ text: String, origin: SkillOrigin = .user) -> Result { - guard let (frontMatter, body) = splitFrontMatter(text) else { - return .failure(.malformedFrontMatter(detail: "Missing the `---` front-matter delimiters.")) - } - let fields = parseFields(frontMatter) - - guard let id = fields["id"], !id.isEmpty else { return .failure(.missingRequiredField(name: "id")) } - guard let title = fields["title"], !title.isEmpty else { return .failure(.missingRequiredField(name: "title")) } - guard let summary = fields["summary"], !summary.isEmpty else { return .failure(.missingRequiredField(name: "summary")) } - // Inputs / outputs are capability SETS (change `ai-action-context-resolution`): the new - // comma-joined `inputs:`/`outputs:` fields, falling back to a legacy single `input:`/`output:` - // line (migrated into the set), so pre-existing `.skill.md` files still parse. - let inputs: Set - switch parseInputSet(new: fields["inputs"], legacy: fields["input"]) { - case let .success(set): inputs = set - case let .failure(err): return .failure(err) - } - let outputs: Set - switch parseOutputSet(new: fields["outputs"], legacy: fields["output"]) { - case let .success(set): outputs = set - case let .failure(err): return .failure(err) - } - - let confirm = fields["confirmBeforeRun"].flatMap { Bool($0) } - let reasoning: AIReasoning? - switch fields["reasoning"] { - case nil, "null": reasoning = nil - case "on": reasoning = .on - case "off": reasoning = .off - case let other?: return .failure(.unknownEnumValue(field: "reasoning", value: other)) - } - let runtimeParameter = parseRuntimeParameter(fields["runtimeParameter"]) - - let command = AICommand( - name: title, - icon: parseIcon(fields["icon"]), - tint: parseTint(fields["tint"]), - inputs: inputs, - promptTemplate: body, - outputs: outputs, - confirmBeforeRun: confirm, - runtimeParameter: runtimeParameter, - reasoning: reasoning) - - let manifest = SkillManifest( - id: id, origin: origin, title: title, summary: summary, - keywords: parseList(fields["keywords"]), - category: fields["category"], - command: command, - toolNames: parseList(fields["tools"]), - claudeHandoff: parseHandoff(fields["claudeHandoff"])) - return .success(manifest) - } - - /// Serialize a manifest back to `.skill.md` text. `parse(serialize(m))` is a fixed point for - /// parsed-origin values (round-trip stable). - static func serialize(_ m: SkillManifest) -> String { - var lines = ["---"] - lines.append("id: \(m.id)") - lines.append("title: \(m.title)") - lines.append("summary: \(m.summary)") - if !m.keywords.isEmpty { lines.append("keywords: [\(m.keywords.joined(separator: ", "))]") } - if let category = m.category { lines.append("category: \(category)") } - lines.append("icon: \(serializeIcon(m.command.icon))") - if let tint = m.command.tint { lines.append("tint: \(serializeTint(tint))") } - lines.append("inputs: \(serializeInputs(m.command.inputs))") - lines.append("outputs: \(serializeOutputs(m.command.outputs))") - lines.append("confirmBeforeRun: \(m.command.confirmBeforeRun)") - if let rp = serializeRuntimeParameter(m.command.runtimeParameter) { lines.append("runtimeParameter: \(rp)") } - if let r = m.command.reasoning { lines.append("reasoning: \(r.rawValue)") } - if !m.toolNames.isEmpty { lines.append("tools: [\(m.toolNames.joined(separator: ", "))]") } - if let h = serializeHandoff(m.claudeHandoff) { lines.append("claudeHandoff: \(h)") } - lines.append("---") - return lines.joined(separator: "\n") + "\n" + m.command.promptTemplate - } - - // MARK: - Front-matter splitting + field parsing - - private static func splitFrontMatter(_ text: String) -> (frontMatter: [String], body: String)? { - let lines = text.components(separatedBy: "\n") - guard let firstDelim = lines.firstIndex(where: { $0.trimmingCharacters(in: .whitespaces) == "---" }) else { return nil } - let rest = lines[(firstDelim + 1)...] - guard let closeOffset = rest.firstIndex(where: { $0.trimmingCharacters(in: .whitespaces) == "---" }) else { return nil } - let fm = Array(lines[(firstDelim + 1).. [String: String] { - var fields: [String: String] = [:] - for line in lines { - let trimmed = line.trimmingCharacters(in: .whitespaces) - guard !trimmed.isEmpty, let colon = trimmed.firstIndex(of: ":") else { continue } - let key = String(trimmed[.. [String] { - guard let value, value.hasPrefix("["), value.hasSuffix("]") else { return [] } - return value.dropFirst().dropLast() - .split(separator: ",").map { $0.trimmingCharacters(in: .whitespaces) }.filter { !$0.isEmpty } - } - - // MARK: - Enum / sink encodings - - static func parseOutput(_ value: String) -> OutputTarget? { - switch value { - case "replaceSelection": return .replaceSelection - case "pasteAtCursor": return .pasteAtCursor - case "previewOnly": return .previewOnly - default: break - } - let parts = value.split(separator: ":", maxSplits: 1, omittingEmptySubsequences: false).map(String.init) - guard parts.count == 2 else { return nil } - let (head, rest) = (parts[0], parts[1]) - if head == "runTask" { - let kindParts = rest.split(separator: ":", maxSplits: 1, omittingEmptySubsequences: false).map(String.init) - switch kindParts[0] { - case "addToCalendar": return .runTask(.addToCalendar) - case "addToReminder": return .runTask(.addToReminder) - case "newContact": return .runTask(.newContact) - case "saveToProject" where kindParts.count == 2: return .runTask(.saveToProject(project: kindParts[1])) - case "openToolWithPayload" where kindParts.count == 2: return .runTask(.openToolWithPayload(tool: kindParts[1])) - default: return nil - } - } - if head == "sendTo" { - let d = rest.split(separator: ":", maxSplits: 1, omittingEmptySubsequences: false).map(String.init) - guard d.count == 2 else { return nil } - switch d[0] { - case "shortcut": return .sendTo(.shortcut(name: d[1])) - case "urlScheme": return .sendTo(.urlScheme(d[1])) - case "shell": return .sendTo(.shell(command: d[1])) - default: return nil - } - } - return nil - } - - static func serializeOutput(_ output: OutputTarget) -> String { - switch output { - case .replaceSelection: return "replaceSelection" - case .pasteAtCursor: return "pasteAtCursor" - case .previewOnly: return "previewOnly" - case let .runTask(kind): - switch kind { - case .addToCalendar: return "runTask:addToCalendar" - case .addToReminder: return "runTask:addToReminder" - case .newContact: return "runTask:newContact" - case let .saveToProject(project): return "runTask:saveToProject:\(project)" - case let .openToolWithPayload(tool): return "runTask:openToolWithPayload:\(tool)" - case let .sendTo(dest): return "sendTo:\(serializeDestination(dest))" // nested sendTo task - } - case let .sendTo(dest): return "sendTo:\(serializeDestination(dest))" - } - } - - private static func serializeDestination(_ d: Destination) -> String { - switch d { - case let .shortcut(name): return "shortcut:\(name)" - case let .urlScheme(scheme): return "urlScheme:\(scheme)" - case let .shell(command): return "shell:\(command)" - } - } - - // MARK: - Capability-set encodings (change `ai-action-context-resolution`) - - /// Parse the input capability set: the new comma-joined `inputs:` (raw `InputSource` values), else a - /// legacy single `input:` line migrated into its set. An empty `inputs:` value is a valid empty - /// (standalone) set; neither field present is a missing-field error. - static func parseInputSet(new: String?, legacy: String?) -> Result, SkillError> { - if let new { - var set: Set = [] - for raw in tokens(new) { - guard let s = InputSource(rawValue: raw) else { - return .failure(.unknownEnumValue(field: "inputs", value: raw)) - } - set.insert(s) - } - return .success(set) - } - if let legacy { - guard let s = InputSource(rawValue: legacy) else { - return .failure(.unknownEnumValue(field: "input", value: legacy)) - } - return .success(AICommand.migrate(input: s)) - } - return .failure(.missingRequiredField(name: "inputs")) - } - - /// Parse the output capability set: the new `outputs:` value — tried FIRST as a single output (a - /// side-effecting encoding carries internal `:`/`,`), else a comma list of simple in-place tokens — - /// else a legacy single `output:` line migrated into its set. Must resolve to a non-empty set. - static func parseOutputSet(new: String?, legacy: String?) -> Result, SkillError> { - if let new { - if let single = parseOutput(new) { return .success([single]) } // side-effecting / lone token - var set: Set = [] - for tok in tokens(new) { - guard let o = parseOutput(tok) else { - return .failure(.unknownEnumValue(field: "outputs", value: tok)) - } - set.insert(o) - } - guard !set.isEmpty else { return .failure(.missingRequiredField(name: "outputs")) } - return .success(set) - } - if let legacy { - guard let o = parseOutput(legacy) else { - return .failure(.unknownEnumValue(field: "output", value: legacy)) - } - return .success(AICommand.migrate(output: o)) - } - return .failure(.missingRequiredField(name: "outputs")) - } - - /// Serialize the input set as a deterministic comma-joined list of raw values (empty set ⇒ ""). - static func serializeInputs(_ inputs: Set) -> String { - inputs.map(\.rawValue).sorted().joined(separator: ", ") - } - - /// Serialize the output set as a deterministic comma-joined list of `serializeOutput` tokens (a lone - /// side-effecting output serializes to just itself, so `parseOutputSet`'s single-value path round-trips). - static func serializeOutputs(_ outputs: Set) -> String { - outputs.map(serializeOutput).sorted().joined(separator: ", ") - } - - /// Split a comma-separated field value into trimmed, non-empty tokens. - private static func tokens(_ value: String) -> [String] { - value.split(separator: ",").map { $0.trimmingCharacters(in: .whitespaces) }.filter { !$0.isEmpty } - } - - private static func parseIcon(_ value: String?) -> ItemIcon { - guard let value, !value.isEmpty else { return .sfSymbol("sparkles") } - if value == "appDefault" { return .appDefault } - if value == "fileIcon" { return .fileIcon } - if value.hasPrefix("emoji:") { return .emoji(String(value.dropFirst("emoji:".count))) } - return .sfSymbol(value) - } - - private static func serializeIcon(_ icon: ItemIcon) -> String { - switch icon { - case .appDefault: return "appDefault" - case .fileIcon: return "fileIcon" - case let .sfSymbol(name): return name - case let .emoji(e): return "emoji:\(e)" - } - } - - private static func parseTint(_ value: String?) -> ItemColor? { - guard let value, value.hasPrefix("#"), value.count == 7 else { return nil } - let hex = value.dropFirst() - guard let n = Int(hex, radix: 16) else { return nil } - return ItemColor(red: Double((n >> 16) & 0xFF) / 255.0, - green: Double((n >> 8) & 0xFF) / 255.0, - blue: Double(n & 0xFF) / 255.0) - } - - private static func serializeTint(_ c: ItemColor) -> String { - let r = Int((c.red * 255).rounded()), g = Int((c.green * 255).rounded()), b = Int((c.blue * 255).rounded()) - return String(format: "#%02X%02X%02X", r, g, b) - } - - private static func parseRuntimeParameter(_ value: String?) -> RuntimeParameter? { - guard let value, value != "null" else { return nil } - let parts = value.split(separator: ":", maxSplits: 1, omittingEmptySubsequences: false).map(String.init) - guard parts.count == 2 else { return nil } - switch parts[0] { - case "language": return .language(default: parts[1]) - case "codeLanguage": return .codeLanguage(default: parts[1]) - default: return nil - } - } - - private static func serializeRuntimeParameter(_ p: RuntimeParameter?) -> String? { - guard case let .languageChoice(def, options)? = p else { return nil } - return options == AILanguages.all ? "language:\(def)" : "codeLanguage:\(def)" - } - - private static func parseHandoff(_ value: String?) -> ClaudeHandoffConfig? { - guard let value, value != "null" else { return nil } - let parts = value.split(separator: ":", maxSplits: 1, omittingEmptySubsequences: false).map(String.init) - let mode = HandoffConfirmMode(rawValue: parts[0]) ?? .confirm - let maxPerDay = parts.count == 2 ? Int(parts[1]) : nil - return ClaudeHandoffConfig(confirmMode: mode, maxPerDay: maxPerDay) - } - - private static func serializeHandoff(_ h: ClaudeHandoffConfig?) -> String? { - guard let h else { return nil } - if let max = h.maxPerDay { return "\(h.confirmMode.rawValue):\(max)" } - return h.confirmMode.rawValue - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Skills/SkillFolderWatcher.swift b/Sources/ThreeFingerSwitcher/AI/Skills/SkillFolderWatcher.swift deleted file mode 100644 index 3bacc90..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Skills/SkillFolderWatcher.swift +++ /dev/null @@ -1,126 +0,0 @@ -import Foundation - -/// A pure coalescing gate for rapid folder-change events (testable with a manual `now:` clock — the -/// live `DispatchSource` firing is user-run-verify, this decides WHEN a burst collapses into one -/// reload). Each raw event arms a window; a reload is due only once the window has settled (no newer -/// event within `interval`). This collapses a save-storm (an editor writing a temp file, renaming, -/// touching mtimes) into a single off-main `loadAll()`. -struct ReloadCoalescer: Sendable { - /// The quiet window a burst must settle for before a reload fires. - let interval: TimeInterval - - init(interval: TimeInterval = 0.25) { self.interval = interval } - - /// Given the timestamp of the latest raw event and the moment we are evaluating, is the burst - /// settled (i.e. should a reload fire now)? Deterministic — the watcher schedules a check at - /// `lastEvent + interval` and asks this; a newer event pushes `lastEvent` forward so the earlier - /// check declines and the burst keeps collapsing. - func isSettled(lastEvent: Date, now: Date) -> Bool { - now.timeIntervalSince(lastEvent) >= interval - 1e-9 - } -} - -/// A live user-folder watcher for the Skills directory (task 4.4). Wraps a -/// `DispatchSource.makeFileSystemObjectSource` on the folder's file descriptor; rapid changes are -/// COALESCED (via `ReloadCoalescer`) and a single re-`loadAll()` runs off-main, republishing the -/// fresh `SkillLoadResult` on the main actor. Built-in skills are projected in-memory (loaded once), -/// so only the user folder needs watching. `loadAll()` is idempotent, so a re-call simply picks up -/// the new/edited/removed files. -/// -/// The live FS-event firing is user-run-verify in a stable-signed build; the coalesced-reload path is -/// exercised by `triggerReloadForTesting()` (a manual trigger that runs the exact republish closure). -final class SkillFolderWatcher: @unchecked Sendable { - private let store: SkillStore - private let folder: URL - private let coalescer: ReloadCoalescer - private let onReload: @Sendable (SkillLoadResult) -> Void - private let queue = DispatchQueue(label: "com.threefingerswitcher.skill-watcher") - - private var source: DispatchSourceFileSystemObject? - private var fileDescriptor: CInt = -1 - private var lastEvent: Date = .distantPast - private var pending = false - - /// - Parameters: - /// - store: the store re-`loadAll()`ed on a settled burst. - /// - coalescer: the quiet-window gate (defaults to 0.25s). - /// - onReload: republish sink, always invoked on the main actor with the fresh result. - init(store: SkillStore, - coalescer: ReloadCoalescer = ReloadCoalescer(), - onReload: @escaping @Sendable (SkillLoadResult) -> Void) { - self.store = store - self.folder = store.userFolder - self.coalescer = coalescer - self.onReload = onReload - } - - deinit { stop() } - - /// Begin watching. Creates the user folder if absent (so the fd open succeeds), then arms a - /// vnode source on it. A write/delete/rename/extend/attribute change re-arms the coalescer. - func start() { - queue.sync { - guard self.source == nil else { return } - try? FileManager.default.createDirectory(at: self.folder, withIntermediateDirectories: true) - let fd = open(self.folder.path, O_EVTONLY) - guard fd >= 0 else { return } // can't open → no watch (idle, not a crash) - self.fileDescriptor = fd - let src = DispatchSource.makeFileSystemObjectSource( - fileDescriptor: fd, - eventMask: [.write, .delete, .rename, .extend, .attrib], - queue: self.queue) - src.setEventHandler { [weak self] in self?.handleRawEvent() } - src.setCancelHandler { [weak self] in - guard let self else { return } - if self.fileDescriptor >= 0 { close(self.fileDescriptor); self.fileDescriptor = -1 } - } - self.source = src - src.resume() - } - } - - /// Stop watching and release the fd. Idempotent. - func stop() { - queue.sync { - self.source?.cancel() - self.source = nil - } - } - - // MARK: - Coalesced reload - - /// A raw FS event: stamp it and schedule a settle check one window out. (Runs on `queue`.) - private func handleRawEvent() { - let now = Date() - lastEvent = now - pending = true - queue.asyncAfter(deadline: .now() + coalescer.interval) { [weak self] in - self?.checkSettled(scheduledFor: now) - } - } - - /// One window after a raw event: if no newer event arrived (the burst settled), reload. (On `queue`.) - private func checkSettled(scheduledFor eventTime: Date) { - guard pending, coalescer.isSettled(lastEvent: lastEvent, now: Date()) else { return } - pending = false - runReload() - } - - /// Re-`loadAll()` off-main and republish on the main actor. Shared by the live path and the test - /// trigger so the coalesced-reload path is exercised without a live FS event. - private func runReload() { - let store = self.store - let onReload = self.onReload - Task.detached { - let result = await store.loadAll() - await MainActor.run { onReload(result) } - } - } - - /// Manually drive the coalesced-reload path (the live FS-event firing is user-run-verify). Runs the - /// exact off-main reload + main-actor republish the watcher would run for a settled burst. - func triggerReloadForTesting() { - queue.sync { self.pending = false } - runReload() - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Skills/SkillManifest.swift b/Sources/ThreeFingerSwitcher/AI/Skills/SkillManifest.swift deleted file mode 100644 index 23c931f..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Skills/SkillManifest.swift +++ /dev/null @@ -1,43 +0,0 @@ -import Foundation - -/// The parsed, in-memory form of a skill (design D2) — an `AICommand` SUPERSET. It REUSES `AICommand` -/// verbatim (no field re-flattening), so `requiredCapabilities`, `resolvedReasoning`, -/// `defaultConfirmBeforeRun`, and the `{lang}` plumbing all come for free and stay consistent with band -/// items. The skill `id` (a stable string) is the *skill* identity (and the `ToolDescriptor.name`); the -/// embedded `AICommand.id` (a UUID) is the per-instance identity when added to a band. MLX-free Core. -struct SkillManifest: Equatable, Sendable, Identifiable { - let id: String // file's `id`, path-relative, stable; the ToolDescriptor.name - var origin: SkillOrigin // .builtIn / .user (drives read-only + shadowing) - var title: String - var summary: String // router-facing when-to-use line; the TOC summary - var keywords: [String] - var category: String? // built-in projection grouping (nil for user skills) - var command: AICommand // the reused value model (icon/tint/input/template/output/param/reasoning) - var toolNames: [String] // optional allow-list of extra tools this skill may invoke - var claudeHandoff: ClaudeHandoffConfig? - var updatedAt: Date - - init(id: String, origin: SkillOrigin, title: String, summary: String, keywords: [String] = [], - category: String? = nil, command: AICommand, toolNames: [String] = [], - claudeHandoff: ClaudeHandoffConfig? = nil, updatedAt: Date = Date()) { - self.id = id - self.origin = origin - self.title = title - self.summary = summary - self.keywords = keywords - self.category = category - self.command = command - self.toolNames = toolNames - self.claudeHandoff = claudeHandoff - self.updatedAt = updatedAt - } - - /// The `IndexedDoc` this skill contributes to the shared `DocIndex` (kind `.skill`; the body is the - /// prompt template, served by the store). - func indexedDoc(bodyPath: URL) -> IndexedDoc { - IndexedDoc(id: id, title: title, summary: summary, keywords: keywords, - kind: .skill, bodyPath: bodyPath, updatedAt: updatedAt) - } -} - -enum SkillOrigin: String, Codable, Sendable { case builtIn, user } diff --git a/Sources/ThreeFingerSwitcher/AI/Skills/SkillStore.swift b/Sources/ThreeFingerSwitcher/AI/Skills/SkillStore.swift deleted file mode 100644 index 5b1591f..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Skills/SkillStore.swift +++ /dev/null @@ -1,137 +0,0 @@ -import Foundation - -/// One bounded, non-blocking load problem (a malformed/duplicate skill file) — surfaced as a Hub row, -/// never an `NSAlert`, raw text only in logs/details. -struct SkillProblem: Equatable, Sendable { - var fileName: String - var headline: String -} - -/// The result of loading the skill corpus: the valid skills + the problems (the load succeeds for the -/// rest even when some files are malformed). -struct SkillLoadResult: Sendable { - var skills: [SkillManifest] - var problems: [SkillProblem] -} - -/// Loads the skill corpus and bridges async file IO to the pure `DocIndex` (design D5). Built-in skills -/// are PROJECTED from `AICommandCatalog` (the catalog stays the byte-for-byte source of truth for the -/// launcher/Bands editor — design deviation noted: lower-risk than inverting the heavily-tested catalog -/// to load from a bundle, same end state); user skills are read from a writable folder. A user skill -/// whose `id` matches a built-in SHADOWS it. The pure index never touches `FileManager`. -final class SkillStore: @unchecked Sendable { - let userFolder: URL - private var loaded: [SkillManifest] = [] - - init(userFolder: URL) { self.userFolder = userFolder } - - /// The built-in skills — one per `AICommandCatalog` preset, projected in-memory (so the catalog and - /// Bands editor are unchanged). The router-facing `summary` is derived from the command (the one - /// field the in-code catalog never had); a user can author a better one by shadowing the skill. - static func builtInManifests() -> [SkillManifest] { - AICommandCatalog.entries.map { entry in - let cmd = entry.command - return SkillManifest( - id: slug(cmd.name), - origin: .builtIn, - title: cmd.name, - summary: derivedSummary(for: cmd), - keywords: keywords(for: cmd), - category: entry.category.rawValue, - command: cmd) - } - } - - /// Load built-in ∪ user, shadow by id, validate. Built-in order preserved; user-only skills appended. - func loadAll() async -> SkillLoadResult { - let builtIn = Self.builtInManifests() - let (userSkills, problems) = Self.loadUserFolder(userFolder) - let userByID = Dictionary(userSkills.map { ($0.id, $0) }, uniquingKeysWith: { a, _ in a }) - let builtInIDs = Set(builtIn.map(\.id)) - var result = builtIn.map { userByID[$0.id] ?? $0 } // user shadows built-in in place - result += userSkills.filter { !builtInIDs.contains($0.id) } // user-only skills, after built-ins - loaded = result - return SkillLoadResult(skills: result, problems: problems) - } - - /// The pure index over the last-loaded snapshot (the body of each skill is its prompt template). - func index() -> DocIndex { - let docs = loaded.map { $0.indexedDoc(bodyPath: bodyPath(for: $0)) } - let bodies = Dictionary(loaded.map { ($0.id, $0.command.promptTemplate) }, uniquingKeysWith: { a, _ in a }) - return InMemoryDocIndex(docs: docs, bodies: bodies) - } - - func manifest(id: String) -> SkillManifest? { loaded.first { $0.id == id } } - - private func bodyPath(for m: SkillManifest) -> URL { - userFolder.appendingPathComponent("\(m.id).skill.md") - } - - // MARK: - User folder - - static func loadUserFolder(_ folder: URL) -> ([SkillManifest], [SkillProblem]) { - let fm = FileManager.default - guard let entries = try? fm.contentsOfDirectory(at: folder, includingPropertiesForKeys: nil) else { - return ([], []) // empty / absent folder is not an error - } - let files = entries - .filter { $0.lastPathComponent.hasSuffix(".skill.md") } - .sorted { $0.lastPathComponent < $1.lastPathComponent } // deterministic order (duplicate winner) - - var skills: [SkillManifest] = [] - var problems: [SkillProblem] = [] - var seen = Set() - for file in files { - guard let text = try? String(contentsOf: file, encoding: .utf8) else { - problems.append(problem(file, SkillError.unreadable(detail: file.path))) - continue - } - switch SkillFile.parse(text, origin: .user) { - case let .success(m): - if seen.contains(m.id) { - problems.append(problem(file, SkillError.duplicateID(id: m.id))) - } else { - seen.insert(m.id) - skills.append(m) - } - case let .failure(error): - problems.append(problem(file, error)) - } - } - return (skills, problems) - } - - private static func problem(_ file: URL, _ error: SkillError) -> SkillProblem { - SkillProblem(fileName: file.lastPathComponent, headline: AIError.message(for: error).headline) - } - - // MARK: - Projection helpers - - /// A path-relative, stable slug for a built-in skill id (e.g. "Fix Grammar" → "fix-grammar"). - static func slug(_ name: String) -> String { - let lowered = name.lowercased() - var out = "" - var lastDash = false - for ch in lowered { - if ch.isLetter || ch.isNumber { - out.append(ch); lastDash = false - } else if !lastDash { - out.append("-"); lastDash = true - } - } - return out.trimmingCharacters(in: CharacterSet(charactersIn: "-")) - } - - /// A reasonable router-facing summary derived from the command (auto-seeded; a shadowing user file - /// can author a better one). - static func derivedSummary(for cmd: AICommand) -> String { - let firstLine = cmd.promptTemplate - .split(whereSeparator: \.isNewline).first.map(String.init) ?? cmd.name - let hint = firstLine.count > 80 ? String(firstLine.prefix(80)) + "…" : firstLine - return "\(cmd.name) — \(hint)" - } - - static func keywords(for cmd: AICommand) -> [String] { - InMemoryDocIndex.tokenize(cmd.name) - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Skills/SkillToolProvider.swift b/Sources/ThreeFingerSwitcher/AI/Skills/SkillToolProvider.swift deleted file mode 100644 index e511787..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Skills/SkillToolProvider.swift +++ /dev/null @@ -1,127 +0,0 @@ -import Foundation - -/// Projects skills into routable tools and invokes them — the skill↔router contract (design D3). It is a -/// `ToolContributor` that `ai-tool-routing`'s `ToolRegistry` aggregates alongside the task tools, memory -/// tools, and `launch_claude`. THIS slice provides the contributor; the route→execute→continue loop is -/// owned by `ai-tool-routing`. MLX-free Core. -struct SkillToolProvider: ToolContributor { - let manifests: [SkillManifest] - let runtime: LLMRuntime - let dispatcher: TaskDispatching - let resolver: WritePolicyResolving - /// Whether the model reasons (global default) — per-skill `.on`/`.off` still wins via the command. - let globalReasoning: Bool - - init(manifests: [SkillManifest], runtime: LLMRuntime, dispatcher: TaskDispatching, - resolver: WritePolicyResolving = DescriptorWritePolicy(), globalReasoning: Bool = false) { - self.manifests = manifests - self.runtime = runtime - self.dispatcher = dispatcher - self.resolver = resolver - self.globalReasoning = globalReasoning - } - - func descriptors() -> [ToolDescriptor] { manifests.map(Self.descriptor(for:)) } - - func canHandle(_ tool: String) -> Bool { manifests.contains { $0.id == tool } } - - func run(_ call: RoutedCall, gate: ApprovalGate) async -> ToolStepResult { - let toolName = call.descriptor.name - guard let m = manifests.first(where: { $0.id == toolName }) else { - return ToolStepResult(tool: toolName, status: .failed(headline: "That skill isn't available."), - summary: "Unknown skill: \(toolName).") - } - let context = FireContext(inputText: call.userText) - let prompt = PromptTemplate.resolve(m.command.promptTemplate, with: context, activeLanguage: nil) - let reasoning = m.command.resolvedReasoning(globalDefault: globalReasoning) - - // In-place skill: generate the text result; that text IS the step's outcome. - if let kind = Self.taskKind(for: m.command) { - return await runSideEffecting(kind, m: m, prompt: prompt, reasoning: reasoning, - descriptor: call.descriptor, gate: gate) - } - do { - let text = try await runtime.generateText( - LLMRequest(prompt: prompt, image: nil, reasoning: reasoning)) - return ToolStepResult(tool: toolName, status: .done, summary: text) - } catch { - return ToolStepResult(tool: toolName, status: .failed(headline: AIError.message(for: error).headline), - summary: "Couldn't run “\(m.title)”.") - } - } - - /// A side-effecting skill bridges into the UNCHANGED `TaskDispatching.prepare`/`execute`, gated by the - /// descriptor's write policy (mirrors `TaskKindToolContributor`). - private func runSideEffecting(_ kind: TaskKind, m: SkillManifest, prompt: String, reasoning: Bool, - descriptor: ToolDescriptor, gate: ApprovalGate) async -> ToolStepResult { - let review = await dispatcher.prepare(kind, resolvedPrompt: prompt, source: TaskSource(), reasoning: reasoning) - switch review { - case let .declined(reason): - return ToolStepResult(tool: m.id, status: .declined(reason: reason), summary: reason) - case let .unavailable(reason): - return ToolStepResult(tool: m.id, status: .failed(headline: reason), summary: reason) - case let .action(title, _, _): - switch resolver.effectiveTier(for: descriptor) { - case .auto: - return await fire(review, tool: m.id, title: title) - case .confirm, .dangerous: - switch await gate.awaitDecision(for: review) { - case .approve: return await fire(review, tool: m.id, title: title) - case .skip: return ToolStepResult(tool: m.id, status: .declined(reason: "skipped"), - summary: "Skipped “\(title)”.") - case .cancel: return ToolStepResult(tool: m.id, - status: .declined(reason: TaskKindToolContributor.cancelledReason), - summary: "Cancelled.") - } - } - } - } - - private func fire(_ review: TaskReview, tool: String, title: String) async -> ToolStepResult { - do { - try await dispatcher.execute(review) - return ToolStepResult(tool: tool, status: .done, summary: "Done: \(title).") - } catch { - return ToolStepResult(tool: tool, status: .failed(headline: AIError.message(for: error).headline), - summary: "Couldn't complete “\(title)”.") - } - } - - // MARK: - Projection - - static func descriptor(for m: SkillManifest) -> ToolDescriptor { - ToolDescriptor( - name: m.id, - summary: m.summary, - argsSchema: schema(for: m.command), - writePolicy: m.command.confirmBeforeRun ? .confirm : .auto, - keywords: m.keywords) - } - - /// The args schema the router targets: the bound `ParsedActions` schema for a side-effecting sink (so - /// the model emits the same validated/declinable shape the dispatcher consumes); a minimal text schema - /// for an in-place sink (the model just produces text). - static func schema(for command: AICommand) -> StructuredSchema { - switch command.sideEffect { - case .runTask(.addToCalendar): return ParsedCalendarEvent.schema - case .runTask(.addToReminder): return ParsedReminder.schema - case .runTask(.newContact): return ParsedContact.schema - case .runTask(.saveToProject): return ParsedSaveToProject.schema - case .runTask(.openToolWithPayload): return ParsedOpenTool.schema - case .runTask(.sendTo), .sendTo: return ParsedSendTo.schema - default: // in-place (no side-effecting output): the model just produces text - return StructuredSchema(name: "text_result", - json: "{\"type\":\"object\",\"required\":[\"result\"],\"properties\":{\"result\":{\"type\":\"string\"}}}") - } - } - - /// The `TaskKind` this command's side-effecting output routes to (reads `command.sideEffect`), or nil - /// for an in-place skill. - private static func taskKind(for command: AICommand) -> TaskKind? { - switch command.sideEffect { - case let .runTask(kind): return kind - case let .sendTo(destination): return .sendTo(destination) - default: return nil - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/StubLLMRuntime.swift b/Sources/ThreeFingerSwitcher/AI/StubLLMRuntime.swift deleted file mode 100644 index 7cecb09..0000000 --- a/Sources/ThreeFingerSwitcher/AI/StubLLMRuntime.swift +++ /dev/null @@ -1,307 +0,0 @@ -import Foundation - -/// A deterministic, scriptable `LLMRuntime` for tests and `swift build`/`swift test` (the real -/// Gemma-4-via-MLX conformer is deferred to an `xcodebuild`-only target — see design D7). -/// -/// It exercises every contract path without a model: -/// - scripted text responses, streamed token-by-token with an artificial inter-token delay, -/// - prompt Task cancellation honored (stops emitting on cancel), -/// - scriptable `structured(...)` outcomes: a valid value, a "non-conforming first, then repaired" -/// sequence (so the bounded repair/retry loop is observable), or an explicit decline. -/// -/// `Sendable` via `@unchecked`: the mutable scripting state is intended to be configured before use -/// in a single-threaded test and is not mutated concurrently. A lock guards the per-call counter so -/// the repair-path test is robust even under concurrency. -public final class StubLLMRuntime: LLMRuntime, @unchecked Sendable { - - // MARK: Structured scripting - - /// How `structured(...)` should behave for the next call(s). The stub interprets these as raw - /// JSON payloads it would "emit" and then runs the real validate → repair/retry → decode → outcome - /// pipeline over them, so the production code path (not a shortcut) is what tests cover. - public enum StructuredScript: Sendable { - /// Emit this exact JSON on the first attempt; it should validate + decode cleanly. - case valid(json: String) - /// Emit `bad` JSON first (fails validation), then `good` on the repair attempt. Proves the - /// bounded repair/retry loop converges. - case invalidThenRepaired(bad: String, good: String) - /// Emit a decline marker so the pipeline returns `.declined(reason:)` instead of a value. - case decline(reason: String) - /// Always emit non-conforming JSON; the bounded loop should exhaust and throw - /// `couldNotProduceValid`. - case alwaysInvalid(json: String) - } - - // MARK: Multi-turn scripting - - /// One turn's script for a multi-turn conversation (design D8). Each `generate(_:)` call (including - /// the one the default `chat(_:)` makes after flattening) dequeues the next `TurnScript` in order, so - /// `swift test` can drive a full deterministic conversation. `error`, when set, makes that turn fail - /// (after any `thinking`, before the response) so the per-turn `.failed` path is observable. - public struct TurnScript: Sendable { - public var tokens: [String] - public var thinking: [String] - public var error: RuntimeError? - public init(tokens: [String] = [], thinking: [String] = [], error: RuntimeError? = nil) { - self.tokens = tokens - self.thinking = thinking - self.error = error - } - } - - // MARK: Configuration - - /// Modalities this stub reports. Default = text + vision (mirrors the v1 flagship); a test can - /// construct a text-only stub to exercise the unsupported-modality path. - public let capabilities: Set - - /// Scripted streaming chunks emitted, in order, for every `generate(_:)` call. If empty, the stub - /// echoes the request prompt as a single token. These are RESPONSE-channel chunks (the final answer). - var scriptedTokens: [String] - - /// Scripted REASONING chunks, emitted as `.thinking`-channel tokens BEFORE the response tokens (so a - /// test can drive the canvas's "show the model's thinking" channel split). Default empty = today's - /// behavior: a pure `.response` stream, byte-identical to before. - var scriptedThinking: [String] - - /// A FIFO queue of per-turn scripts for multi-turn conversations (design D8). Each `generate(_:)` - /// dequeues one; when exhausted, generation falls back to `scriptedTokens`/`scriptedThinking` so - /// existing single-generation tests stay byte-identical. Configure before driving a conversation. - var scriptedTurns: [TurnScript] - - /// Artificial delay between emitted tokens, in nanoseconds. Lets a test observe streaming order - /// and cancel mid-stream. Default is a tiny delay so tests stay fast. - var interTokenDelayNanos: UInt64 - - /// Scripted behavior for the next `structured(...)` call. nil → a generic empty-object attempt. - var structuredScript: StructuredScript? - - /// Bounded repair/retry budget for `structured(...)` (total attempts including the first). - var maxRepairAttempts: Int - - /// Records how many attempts the last `structured(...)` call consumed (for assertions). - private(set) var lastAttemptCount: Int = 0 - - /// True once a `generate(_:)` stream detected cancellation and stopped emitting. Deterministic - /// observation point for "cancellation stops generation" — independent of the consumer-side race - /// where a self-cancelled `AsyncThrowingStream` iterator terminates before the thrown terminal - /// error is observed. - private var didObserveCancellation = false - var observedCancellation: Bool { - lock.lock(); defer { lock.unlock() }; return didObserveCancellation - } - - private let lock = NSLock() - - public init(capabilities: Set = [.text, .vision], - scriptedTokens: [String] = [], - scriptedThinking: [String] = [], - scriptedTurns: [TurnScript] = [], - interTokenDelayNanos: UInt64 = 1_000_000, // 1 ms - structuredScript: StructuredScript? = nil, - maxRepairAttempts: Int = 3) { - self.capabilities = capabilities - self.scriptedTokens = scriptedTokens - self.scriptedThinking = scriptedThinking - self.scriptedTurns = scriptedTurns - self.interTokenDelayNanos = interTokenDelayNanos - self.structuredScript = structuredScript - self.maxRepairAttempts = maxRepairAttempts - } - - // MARK: Streaming - - public func generate(_ request: LLMRequest) -> AsyncThrowingStream { - // Multi-turn scripting: dequeue the next per-turn script (FIFO). When the queue is exhausted, - // fall back to the single-generation `scriptedTokens`/`scriptedThinking` so existing one-shot - // tests are byte-identical. The default `chat(_:)` (flatten → generate) consumes the queue - // through THIS method, so no `chat` override is needed in the stub. - let turn = nextTurnScript() - // Capture the script up-front so the closure doesn't race on `self` mutation mid-stream. - let chunks: [String] - let thinkingChunks: [String] - let turnError: RuntimeError? - if let turn { - chunks = turn.tokens.isEmpty ? [request.prompt] : turn.tokens - thinkingChunks = turn.thinking - turnError = turn.error - } else { - chunks = scriptedTokens.isEmpty ? [request.prompt] : scriptedTokens - thinkingChunks = scriptedThinking - turnError = nil - } - let delay = interTokenDelayNanos - let needsVision = request.requiresVision - let needsAudio = request.requiresAudio - let caps = capabilities - - return AsyncThrowingStream { continuation in - let task = Task { - // A vision request against a text-only stub is a hard error, never a silent degrade. - if needsVision && !caps.contains(.vision) { - continuation.finish(throwing: RuntimeError.unsupportedModality(.vision)) - return - } - // The audio seam's refusal contract (`add-voice-computer-use-agent`): NO runtime serves - // audio yet — non-empty audio is ALWAYS refused (even if `capabilities` claims `.audio`, - // which only marks a descriptor for selection), never silently dropped. Remove this - // unconditional refusal only when a conformer actually feeds the audio tower. - if needsAudio { - continuation.finish(throwing: RuntimeError.unsupportedModality(.audio)) - return - } - do { - // Reasoning first: `.thinking`-channel chunks stream BEFORE the response (so a - // consumer can split the model's thinking from its answer). isFinal stays false — - // only the last RESPONSE token is final. Empty by default (a pure response stream). - for chunk in thinkingChunks { - try Task.checkCancellation() - if delay > 0 { try await Task.sleep(nanoseconds: delay) } - try Task.checkCancellation() - continuation.yield(Token(chunk, isFinal: false, channel: .thinking)) - } - // A scripted per-turn error fails THIS turn (after any thinking, before the response) - // so the executor's per-turn `.failed` path is observable. - if let turnError { - continuation.finish(throwing: turnError) - return - } - for (i, chunk) in chunks.enumerated() { - // Honor cancellation BEFORE emitting so a discard stops work promptly. - try Task.checkCancellation() - if delay > 0 { try await Task.sleep(nanoseconds: delay) } - try Task.checkCancellation() - continuation.yield(Token(chunk, isFinal: i == chunks.count - 1)) - } - continuation.finish() - } catch is CancellationError { - self.markCancelled() - continuation.finish(throwing: RuntimeError.cancelled) - } catch { - continuation.finish(throwing: error) - } - } - continuation.onTermination = { _ in task.cancel() } - } - } - - // MARK: Structured - - /// A decline marker the stub recognizes in "model output". The validate/repair pipeline treats a - /// payload carrying this as a first-class `.declined` rather than a decode failure. - static let declineMarkerKey = "__declined__" - - public func structured( - _ request: LLMRequest, - schema: StructuredSchema, - as type: T.Type - ) async throws -> StructuredOutcome { - try Task.checkCancellation() - - let script = structuredScript ?? .valid(json: "{}") - let budget = max(1, maxRepairAttempts) - - // The bounded validate → repair/retry loop, run over the scripted "model emissions". - var attempts = 0 - var lastDecodeDetail = "" - - while attempts < budget { - try Task.checkCancellation() - attempts += 1 - let raw = Self.emission(for: script, attempt: attempts) - - // 1) Decline detection: a declined emission short-circuits to a typed decline. - if let reason = Self.declineReason(in: raw) { - setLastAttemptCount(attempts) - return .declined(reason: reason) - } - - // 2) Validate against the schema (structural validation; see `validate`). - guard let data = raw.data(using: .utf8), - Self.validate(jsonData: data, against: schema) else { - lastDecodeDetail = "schema validation failed on attempt \(attempts)" - continue // repair/retry - } - - // 3) Decode into the target Swift type. - do { - let decoded = try JSONDecoder().decode(T.self, from: data) - setLastAttemptCount(attempts) - return .value(decoded) - } catch { - lastDecodeDetail = "decode failed: \(error)" - continue // a structurally-valid-but-undecodable payload is also repairable - } - } - - setLastAttemptCount(attempts) - _ = lastDecodeDetail - throw RuntimeError.couldNotProduceValid(attempts: attempts) - } - - // MARK: - Scripting helpers - - /// What the stub "emits" on a given attempt for a script. - private static func emission(for script: StructuredScript, attempt: Int) -> String { - switch script { - case let .valid(json): - return json - case let .invalidThenRepaired(bad, good): - return attempt == 1 ? bad : good - case let .decline(reason): - return "{\"\(declineMarkerKey)\": \"\(reason)\"}" - case let .alwaysInvalid(json): - return json - } - } - - /// If the raw emission is a decline marker, return its reason. - private static func declineReason(in raw: String) -> String? { - guard let data = raw.data(using: .utf8), - let obj = try? JSONSerialization.jsonObject(with: data) as? [String: Any], - let reason = obj[declineMarkerKey] as? String else { return nil } - return reason - } - - /// Structural schema validation: parse the JSON and check it carries every property named in the - /// schema's top-level `required` array (a pragmatic subset of JSON Schema sufficient to drive the - /// validate/repair/decode pipeline deterministically in tests). Non-object JSON or malformed input - /// fails validation, which the loop then repairs. - static func validate(jsonData: Data, against schema: StructuredSchema) -> Bool { - guard let value = try? JSONSerialization.jsonObject(with: jsonData) as? [String: Any] else { - return false - } - let required = requiredKeys(fromSchemaJSON: schema.json) - for key in required where value[key] == nil { - return false - } - return true - } - - /// Extract the `required: [...]` string array from a JSON-Schema document string. Returns an empty - /// set if absent (a schema with no required keys accepts any object). - private static func requiredKeys(fromSchemaJSON json: String) -> Set { - guard let data = json.data(using: .utf8), - let obj = try? JSONSerialization.jsonObject(with: data) as? [String: Any], - let required = obj["required"] as? [String] else { return [] } - return Set(required) - } - - private func setLastAttemptCount(_ n: Int) { - lock.lock(); defer { lock.unlock() } - lastAttemptCount = n - } - - private func markCancelled() { - lock.lock(); defer { lock.unlock() } - didObserveCancellation = true - } - - /// Dequeue the next per-turn script (FIFO) under the lock, or nil when the multi-turn queue is empty - /// (→ fall back to single-generation scripting). Called once at the start of each `generate(_:)`. - private func nextTurnScript() -> TurnScript? { - lock.lock(); defer { lock.unlock() } - guard !scriptedTurns.isEmpty else { return nil } - return scriptedTurns.removeFirst() - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/TaskDispatching.swift b/Sources/ThreeFingerSwitcher/AI/TaskDispatching.swift deleted file mode 100644 index fae09c0..0000000 --- a/Sources/ThreeFingerSwitcher/AI/TaskDispatching.swift +++ /dev/null @@ -1,31 +0,0 @@ -import Foundation - -/// The seam for the agentic task layer (design D6; tasks phase 13). The executor routes a -/// side-effecting output (`.runTask` / `.sendTo`) to its task entirely through this protocol, so the -/// in-place pipeline and the task layer stay decoupled and both are testable headless. -/// -/// REFINED in slice 4 from the thin `dispatch(...)` seam to a two-stage `prepare` → `execute`, which -/// makes the action-review/armed-confirmation state (design D6) first-class: -/// -/// - `prepare(kind:resolvedPrompt:source:)` asks the model for a schema-targeted, validated, parsed -/// action via `runtime.structured(...)`. It maps a typed/affordance decline → `.declined`, a -/// validation-exhausted result → `.unavailable` (NO action), and a success → `.action` carrying the -/// preview `fields` and the prepared payload. It performs NO side effect. -/// - `execute(_:)` fires the side effect for a CONFIRMED `.action` only. The executor calls it after -/// the action-review is confirmed (when `confirmBeforeRun` is on) or directly (when the user has -/// turned review off — the stored value is honored, never overridden). -/// -/// Discarding before `execute` leaves no side effect (the executor simply never calls it). -@MainActor -protocol TaskDispatching { - /// Produce a `TaskReview` for `kind` from `resolvedPrompt` (the already-resolved template). The - /// `source` is the fire-time provenance (app/URL/timestamp) recorded with saved/sent content. - /// `reasoning` is the per-command-resolved think-before-answering flag the executor owns (an - /// explicit override, else the global default) — carried onto the structured `LLMRequest`. - func prepare(_ kind: TaskKind, resolvedPrompt: String, source: TaskSource, - reasoning: Bool) async -> TaskReview - - /// Fire the side effect for a confirmed review. A no-op for a non-`.action` review. - /// - Throws: the task's failure (permission denied, sink failure, …) so the caller can surface it. - func execute(_ review: TaskReview) async throws -} diff --git a/Sources/ThreeFingerSwitcher/AI/Tasks/ParsedActions.swift b/Sources/ThreeFingerSwitcher/AI/Tasks/ParsedActions.swift deleted file mode 100644 index 07162be..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Tasks/ParsedActions.swift +++ /dev/null @@ -1,221 +0,0 @@ -import Foundation - -/// Per-kind PARSED ACTION types and their JSON Schemas (tasks phase 13.1; spec "Tasks use -/// schema-targeted structured output"). Each kind is described by a `StructuredSchema` the model -/// targets via `runtime.structured(...)`; the runtime validates + repairs/retries the emission and -/// may DECLINE ("not applicable") rather than fabricate a value (design D2). -/// -/// Each schema carries an explicit `applicable` boolean as a DECLINE AFFORDANCE: the model can mark -/// `applicable: false` (with a `reason`) to refuse a task whose input doesn't fit — e.g. text that -/// describes no meeting yields `applicable: false`, not an invented event. The dispatcher treats a -/// non-applicable parsed action exactly like the runtime's first-class `.declined` outcome, so the -/// decline path is honored whether the model expresses it as a typed decline or as a payload field. -/// -/// The parsed types are pure `Decodable & Sendable` value types with no AppKit dependency, so the -/// whole task layer is unit-testable headless against `StubLLMRuntime`. - -// MARK: - Decline affordance - -/// The shared decline shape every parsed action embeds: `applicable == false` means "the model -/// refuses this task for this input" and carries an optional human-readable `reason`. -protocol DeclinableAction: Decodable, Sendable { - /// Whether the model considers the task applicable to the input. `false` ⇒ a decline. - var applicable: Bool { get } - /// The reason the model declined (only meaningful when `applicable == false`). - var declineReason: String? { get } -} - -// MARK: - Calendar - -/// The parsed action for an "add to calendar" task: the event the model extracted from the input. -struct ParsedCalendarEvent: DeclinableAction, Equatable { - var applicable: Bool - var reason: String? - var title: String? - /// ISO-8601 (or model-emitted) start timestamp string. Carried as a string so the parse layer - /// never fails on a slightly-off format; the sink resolves it to a `Date`. - var start: String? - var end: String? - var attendees: [String]? - var notes: String? - - var declineReason: String? { reason } - - /// The JSON Schema the model targets. `applicable` + `title` + `start` are required so a usable - /// event always carries a title and a start; a decline sets `applicable:false` and may omit them - /// only by also being routed through the runtime's typed decline (see `TaskDispatcher`). - static let schema = StructuredSchema( - name: "calendar_event", - json: #""" - { - "type": "object", - "required": ["applicable"], - "properties": { - "applicable": { "type": "boolean", "description": "false if the text describes no meeting" }, - "reason": { "type": "string" }, - "title": { "type": "string" }, - "start": { "type": "string", "description": "ISO-8601 start, e.g. 2026-06-08T15:00" }, - "end": { "type": "string", "description": "ISO-8601 end" }, - "attendees": { "type": "array", "items": { "type": "string" } }, - "notes": { "type": "string" } - } - } - """# - ) -} - -// MARK: - Reminder - -/// The parsed action for an "add to reminders" task: the to-do the model extracted from the input. -/// Mirrors `ParsedCalendarEvent` (EventKit), but targets reminders rather than timed events. -struct ParsedReminder: DeclinableAction, Equatable { - var applicable: Bool - var reason: String? - var title: String? - /// ISO-8601 (or model-emitted) due timestamp string; carried as a string so the parse layer never - /// fails on a slightly-off format — the sink resolves it to date components. - var due: String? - var notes: String? - /// EventKit reminder priority (0 = none, 1 = high … 9 = low). Optional; omitted ⇒ no priority. - var priority: Int? - - var declineReason: String? { reason } - - static let schema = StructuredSchema( - name: "reminder", - json: #""" - { - "type": "object", - "required": ["applicable"], - "properties": { - "applicable": { "type": "boolean", "description": "false if the text describes no task" }, - "reason": { "type": "string" }, - "title": { "type": "string" }, - "due": { "type": "string", "description": "ISO-8601 due date/time, e.g. 2026-06-08T15:00" }, - "notes": { "type": "string" }, - "priority": { "type": "integer", "description": "0 none, 1 high … 9 low" } - } - } - """# - ) -} - -// MARK: - Contact - -/// The parsed action for a "new contact" task: the contact card the model extracted from the input -/// (e.g. an email signature). The model may decline when the input carries no contact details. -struct ParsedContact: DeclinableAction, Equatable { - var applicable: Bool - var reason: String? - var name: String? - var email: String? - var phone: String? - var organization: String? - var notes: String? - - var declineReason: String? { reason } - - static let schema = StructuredSchema( - name: "contact", - json: #""" - { - "type": "object", - "required": ["applicable"], - "properties": { - "applicable": { "type": "boolean", "description": "false if the text has no contact details" }, - "reason": { "type": "string" }, - "name": { "type": "string", "description": "the person's full name" }, - "email": { "type": "string" }, - "phone": { "type": "string" }, - "organization": { "type": "string" }, - "notes": { "type": "string" } - } - } - """# - ) -} - -// MARK: - Save to project - -/// The parsed action for a "save to project" task: the (optionally model-refined) content to append -/// to the project note. `project` is carried from the task config, not the model, and filled in by -/// the dispatcher; the model only refines the body. -struct ParsedSaveToProject: DeclinableAction, Equatable { - var applicable: Bool - var reason: String? - /// The body to append (the model may refine the raw input; otherwise it echoes it). - var content: String? - - var declineReason: String? { reason } - - static let schema = StructuredSchema( - name: "save_to_project", - json: #""" - { - "type": "object", - "required": ["applicable"], - "properties": { - "applicable": { "type": "boolean" }, - "reason": { "type": "string" }, - "content": { "type": "string", "description": "the note body to append" } - } - } - """# - ) -} - -// MARK: - Open tool with payload - -/// The parsed action for an "open tool with this payload" task: the generated payload to hand the -/// tool. The target `tool` is carried from the task config (not invented by the model). -struct ParsedOpenTool: DeclinableAction, Equatable { - var applicable: Bool - var reason: String? - /// The payload (e.g. a generated prompt) the tool is opened with. - var payload: String? - - var declineReason: String? { reason } - - static let schema = StructuredSchema( - name: "open_tool", - json: #""" - { - "type": "object", - "required": ["applicable"], - "properties": { - "applicable": { "type": "boolean" }, - "reason": { "type": "string" }, - "payload": { "type": "string", "description": "the payload to open the tool with" } - } - } - """# - ) -} - -// MARK: - Send to destination - -/// The parsed action for a "send to destination" task: the (optionally refined) content to deliver. -/// The destination itself is carried from the task config (not invented by the model). -struct ParsedSendTo: DeclinableAction, Equatable { - var applicable: Bool - var reason: String? - /// The content delivered to the destination (the model may refine it). - var content: String? - - var declineReason: String? { reason } - - static let schema = StructuredSchema( - name: "send_to", - json: #""" - { - "type": "object", - "required": ["applicable"], - "properties": { - "applicable": { "type": "boolean" }, - "reason": { "type": "string" }, - "content": { "type": "string", "description": "the content to deliver" } - } - } - """# - ) -} diff --git a/Sources/ThreeFingerSwitcher/AI/Tasks/TaskDispatcher.swift b/Sources/ThreeFingerSwitcher/AI/Tasks/TaskDispatcher.swift deleted file mode 100644 index 3255b9b..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Tasks/TaskDispatcher.swift +++ /dev/null @@ -1,246 +0,0 @@ -import Foundation - -/// The concrete agentic task layer (tasks phase 13.2): turns a `TaskKind` + a resolved prompt into a -/// `TaskReview` via schema-targeted structured output, and fires the reviewed side effect on commit. -/// -/// It depends only on small injectable seams — a runtime provider and four sinks (`CalendarSink`, -/// `ProjectStore`, `ToolOpener`, `DestinationSender`) — so the whole flow unit-tests headless against -/// `StubLLMRuntime` + fakes. Production wires the EventKit / on-disk / launch / adapter implementations. -/// -/// Contract (design D2/D6, spec ai-command-tasks): -/// - `prepare` validates + repairs/retries via `runtime.structured(...)`; a typed decline OR an -/// `applicable:false` parsed action → `.declined`; an exhausted repair loop → `.unavailable` -/// (NEVER a malformed side effect); success → `.action` with preview `fields` + payload. -/// - `execute` fires the side effect for a confirmed `.action` ONLY — and routes calendar/save/open/ -/// send to the right sink with the right payload. -@MainActor -final class TaskDispatcher: TaskDispatching { - - /// Resolves a `.text`-capable runtime to drive `structured(...)`. In production this is - /// `{ try await modelManager.runtime(requiring: [.text]) }`; tests inject a stub directly. - private let runtimeProvider: () async throws -> LLMRuntime - private let calendarSink: CalendarSink - private let reminderSink: ReminderSink - private let contactSink: ContactSink - private let projectStore: ProjectStore - private let toolOpener: ToolOpener - private let destinationSender: DestinationSender - - init(runtimeProvider: @escaping () async throws -> LLMRuntime, - calendarSink: CalendarSink, - reminderSink: ReminderSink, - contactSink: ContactSink, - projectStore: ProjectStore, - toolOpener: ToolOpener, - destinationSender: DestinationSender) { - self.runtimeProvider = runtimeProvider - self.calendarSink = calendarSink - self.reminderSink = reminderSink - self.contactSink = contactSink - self.projectStore = projectStore - self.toolOpener = toolOpener - self.destinationSender = destinationSender - } - - /// Convenience production wiring: drive structured output off the `ModelManager`'s text runtime and - /// use the EventKit / Contacts / on-disk / workspace / adapter sinks. Calendar, Reminders, and - /// Contacts permissions are each requested lazily at first use of the corresponding task. - convenience init(modelManager: ModelManager, permissions: PermissionsService) { - self.init( - runtimeProvider: { try await modelManager.runtime(requiring: [.text]) }, - calendarSink: EventKitCalendarSink(permissions: permissions), - reminderSink: EventKitReminderSink(permissions: permissions), - contactSink: ContactsSink(permissions: permissions), - projectStore: DiskProjectStore(), - toolOpener: WorkspaceToolOpener(), - destinationSender: AdapterDestinationSender() - ) - } - - // MARK: - Prepare - - func prepare(_ kind: TaskKind, resolvedPrompt: String, source: TaskSource, - reasoning: Bool) async -> TaskReview { - switch kind { - case .addToCalendar: - return await prepareCalendar(resolvedPrompt: resolvedPrompt, reasoning: reasoning) - case .addToReminder: - return await prepareReminder(resolvedPrompt: resolvedPrompt, reasoning: reasoning) - case .newContact: - return await prepareContact(resolvedPrompt: resolvedPrompt, reasoning: reasoning) - case let .saveToProject(project): - return await prepareSaveToProject(project: project, resolvedPrompt: resolvedPrompt, - source: source, reasoning: reasoning) - case let .openToolWithPayload(tool): - return await prepareOpenTool(tool: tool, resolvedPrompt: resolvedPrompt, reasoning: reasoning) - case let .sendTo(destination): - return await prepareSendTo(destination: destination, resolvedPrompt: resolvedPrompt, - reasoning: reasoning) - } - } - - private func prepareCalendar(resolvedPrompt: String, reasoning: Bool) async -> TaskReview { - await parse(resolvedPrompt, schema: ParsedCalendarEvent.schema, as: ParsedCalendarEvent.self, - reasoning: reasoning) { event in - // A usable event must carry a title; a missing one is not a valid action. - guard let title = event.title, !title.isEmpty else { - return .unavailable(reason: "The action was missing a title.") - } - var fields: [ReviewField] = [ReviewField("Title", title)] - if let start = event.start { fields.append(ReviewField("Start", start)) } - if let end = event.end { fields.append(ReviewField("End", end)) } - if let attendees = event.attendees, !attendees.isEmpty { - fields.append(ReviewField("Attendees", attendees.joined(separator: ", "))) - } - if let notes = event.notes, !notes.isEmpty { fields.append(ReviewField("Notes", notes)) } - return .action(title: "Add to Calendar", fields: fields, payload: .calendar(event)) - } - } - - private func prepareReminder(resolvedPrompt: String, reasoning: Bool) async -> TaskReview { - await parse(resolvedPrompt, schema: ParsedReminder.schema, as: ParsedReminder.self, - reasoning: reasoning) { reminder in - guard let title = reminder.title, !title.isEmpty else { - return .unavailable(reason: "The action was missing a title.") - } - var fields: [ReviewField] = [ReviewField("Title", title)] - if let due = reminder.due { fields.append(ReviewField("Due", due)) } - if let priority = reminder.priority { fields.append(ReviewField("Priority", String(priority))) } - if let notes = reminder.notes, !notes.isEmpty { fields.append(ReviewField("Notes", notes)) } - return .action(title: "Add to Reminders", fields: fields, payload: .reminder(reminder)) - } - } - - private func prepareContact(resolvedPrompt: String, reasoning: Bool) async -> TaskReview { - await parse(resolvedPrompt, schema: ParsedContact.schema, as: ParsedContact.self, - reasoning: reasoning) { contact in - // A usable contact needs at least a name or a reachable handle. - let hasAny = [contact.name, contact.email, contact.phone, contact.organization] - .contains { ($0?.isEmpty == false) } - guard hasAny else { - return .unavailable(reason: "The action had no contact details.") - } - var fields: [ReviewField] = [] - if let name = contact.name, !name.isEmpty { fields.append(ReviewField("Name", name)) } - if let org = contact.organization, !org.isEmpty { fields.append(ReviewField("Organization", org)) } - if let email = contact.email, !email.isEmpty { fields.append(ReviewField("Email", email)) } - if let phone = contact.phone, !phone.isEmpty { fields.append(ReviewField("Phone", phone)) } - if let notes = contact.notes, !notes.isEmpty { fields.append(ReviewField("Notes", notes)) } - return .action(title: "New Contact", fields: fields, payload: .contact(contact)) - } - } - - private func prepareSaveToProject(project: String, resolvedPrompt: String, - source: TaskSource, reasoning: Bool) async -> TaskReview { - await parse(resolvedPrompt, schema: ParsedSaveToProject.schema, as: ParsedSaveToProject.self, - reasoning: reasoning) { action in - guard let content = action.content, !content.isEmpty else { - return .unavailable(reason: "The action had no content to save.") - } - let fields = [ReviewField("Project", project), ReviewField("Content", content)] - return .action(title: "Save to Project", - fields: fields, - payload: .saveToProject(project: project, action: action, source: source)) - } - } - - private func prepareOpenTool(tool: String, resolvedPrompt: String, reasoning: Bool) async -> TaskReview { - await parse(resolvedPrompt, schema: ParsedOpenTool.schema, as: ParsedOpenTool.self, - reasoning: reasoning) { action in - guard let payload = action.payload, !payload.isEmpty else { - return .unavailable(reason: "The action produced no payload.") - } - let fields = [ReviewField("Tool", tool), ReviewField("Payload", payload)] - return .action(title: "Open Tool", fields: fields, payload: .openTool(tool: tool, action: action)) - } - } - - private func prepareSendTo(destination: Destination, resolvedPrompt: String, - reasoning: Bool) async -> TaskReview { - await parse(resolvedPrompt, schema: ParsedSendTo.schema, as: ParsedSendTo.self, - reasoning: reasoning) { action in - guard let content = action.content, !content.isEmpty else { - return .unavailable(reason: "There was no content to send.") - } - let fields = [ReviewField("Destination", Self.describe(destination)), - ReviewField("Content", content)] - return .action(title: "Send", fields: fields, payload: .sendTo(destination, action: action)) - } - } - - /// Shared parse pipeline: call `runtime.structured(...)` with the kind's schema, map a typed - /// decline OR an `applicable:false` affordance to `.declined`, a couldNotProduceValid / - /// runtime-failure to `.unavailable` (NO action), and a valid + applicable value to `build(...)`. - private func parse(_ prompt: String, - schema: StructuredSchema, - as type: A.Type, - reasoning: Bool, - build: (A) -> TaskReview) async -> TaskReview { - let runtime: LLMRuntime - do { - runtime = try await runtimeProvider() - } catch { - return .unavailable(reason: TaskDispatcher.message(for: error)) - } - do { - let outcome = try await runtime.structured(LLMRequest(prompt: prompt, reasoning: reasoning), schema: schema, as: type) - switch outcome { - case let .declined(reason): - return .declined(reason: reason) - case let .value(action): - // The in-payload decline affordance is honored exactly like a typed decline. - guard action.applicable else { - return .declined(reason: action.declineReason ?? "Not applicable to this input.") - } - return build(action) - } - } catch let error as RuntimeError { - if case .couldNotProduceValid = error { - return .unavailable(reason: "Couldn't produce a valid action.") - } - return .unavailable(reason: TaskDispatcher.message(for: error)) - } catch { - return .unavailable(reason: TaskDispatcher.message(for: error)) - } - } - - // MARK: - Execute - - func execute(_ review: TaskReview) async throws { - guard case let .action(_, _, payload) = review else { return } - switch payload { - case let .calendar(event): - try await calendarSink.create(event) - case let .reminder(reminder): - try await reminderSink.create(reminder) - case let .contact(contact): - try await contactSink.create(contact) - case let .saveToProject(project, action, source): - let content = action.content ?? "" - try projectStore.append(project: project, content: content, source: source) - case let .openTool(tool, action): - try await toolOpener.open(tool: tool, payload: action.payload ?? "") - case let .sendTo(destination, action): - try await destinationSender.send(destination, content: action.content ?? "") - } - } - - // MARK: - Helpers - - /// A human-readable description of a destination for the review preview. - static func describe(_ destination: Destination) -> String { - switch destination { - case let .shortcut(name): return "Shortcut: \(name)" - case let .urlScheme(scheme): return "URL: \(scheme)" - case let .shell(command): return "Shell: \(command)" - } - } - - /// Map any error to a clean `.unavailable` reason via the single central translator (`AIError`), so - /// a task-prepare failure shows the same clean headline as every other AI surface. (The exhausted - /// repair loop — `couldNotProduceValid` — is intercepted by the caller with a task-specific phrasing - /// before this is reached.) - private static func message(for error: Error) -> String { - AIError.message(for: error).headline - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Tasks/TaskReview.swift b/Sources/ThreeFingerSwitcher/AI/Tasks/TaskReview.swift deleted file mode 100644 index bb4ca67..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Tasks/TaskReview.swift +++ /dev/null @@ -1,88 +0,0 @@ -import Foundation - -/// The value the UI / executor consumes after `TaskDispatcher.prepare(...)` (tasks phase 13.2). It is -/// the boundary between "the model produced (or refused) a parsed action" and "the side effect fires -/// on commit". Slice 5's armed-confirmation overlay BINDS to `.action`'s `fields` to render the -/// concrete preview; this slice builds the value and the execute path, not the overlay. -/// -/// Three terminal shapes: -/// - `.action` — a validated, parsed action ready to fire; carries display `fields` for the -/// confirmation preview and an opaque `payload` the dispatcher executes. -/// - `.declined` — the model judged the input "not applicable" (design D2); nothing will fire. -/// - `.unavailable` — the bounded repair/retry loop exhausted without a valid action (or a -/// precondition failed); NO malformed side effect is ever dispatched. -enum TaskReview: Sendable { - /// A ready-to-fire action: a label for the task, the rendered `fields` for the preview, and the - /// `payload` the dispatcher consumes in `execute`. - case action(title: String, fields: [ReviewField], payload: PreparedAction) - /// The model declined the task as not applicable. - case declined(reason: String) - /// No valid action could be produced (validation exhausted), or a precondition was unmet. - case unavailable(reason: String) - - /// Convenience: the prepared payload if this is an `.action`, else nil. - var preparedAction: PreparedAction? { - if case let .action(_, _, payload) = self { return payload } - return nil - } - - var isAction: Bool { - if case .action = self { return true } - return false - } - - /// Compare two reviews by their OBSERVABLE preview surface (discriminant + title + fields, and the - /// reason for non-action shapes). The `payload` is opaque (it wraps non-Equatable closures-free but - /// intentionally hidden parsed actions), so it is deliberately excluded. Used for `State` equality - /// in tests / the UI, which only ever observe the preview. - static func previewEqual(_ lhs: TaskReview, _ rhs: TaskReview) -> Bool { - switch (lhs, rhs) { - case let (.action(t1, f1, _), .action(t2, f2, _)): - return t1 == t2 && f1 == f2 - case let (.declined(r1), .declined(r2)): - return r1 == r2 - case let (.unavailable(r1), .unavailable(r2)): - return r1 == r2 - default: - return false - } - } -} - -/// One `(label, value)` row rendered in the action-review preview (tasks phase 13.2). Pure value type -/// so the preview is unit-testable without any UI. -struct ReviewField: Equatable, Sendable { - var label: String - var value: String - - init(_ label: String, _ value: String) { - self.label = label - self.value = value - } -} - -/// The opaque, validated payload the dispatcher executes for a confirmed `.action`. It pairs the -/// typed parsed action with the concrete config carried from the command (project name, tool, send-to -/// destination), so `execute` routes to the right sink/store/opener/sender with the right values. -enum PreparedAction: Sendable { - case calendar(ParsedCalendarEvent) - case reminder(ParsedReminder) - case contact(ParsedContact) - case saveToProject(project: String, action: ParsedSaveToProject, source: TaskSource) - case openTool(tool: String, action: ParsedOpenTool) - case sendTo(Destination, action: ParsedSendTo) -} - -/// The provenance recorded alongside saved/sent content (spec: save-to-project appends content "with -/// its source app/URL and a timestamp"). Captured at fire time from the command's `FireContext`. -struct TaskSource: Equatable, Sendable { - var appName: String? - var url: URL? - var timestamp: Date - - init(appName: String? = nil, url: URL? = nil, timestamp: Date = Date()) { - self.appName = appName - self.url = url - self.timestamp = timestamp - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Tasks/TaskSinks.swift b/Sources/ThreeFingerSwitcher/AI/Tasks/TaskSinks.swift deleted file mode 100644 index 8499787..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Tasks/TaskSinks.swift +++ /dev/null @@ -1,488 +0,0 @@ -import Foundation -import EventKit -import Contacts -import os -#if canImport(AppKit) -import AppKit -#endif - -/// Breadcrumbs for side-effect failures. Raw OS errors are logged here (and only here) so a clean, -/// human-facing `TaskError.sinkFailed` message can be surfaced without leaking the raw text to the UI -/// (spec: "Diagnostic logging of the original error at the boundary is permitted and encouraged"). -private let taskSinkLog = Logger(subsystem: "ThreeFingerSwitcher", category: "TaskSinks") - -/// The SMALL injectable seams behind which each task's side effect lives (tasks phase 13.3–13.6), so -/// the `TaskDispatcher` is unit-testable headless: tests inject fakes that record what they were -/// asked to do; production injects the real EventKit / on-disk note / launch / destination adapters. -/// -/// A side effect ONLY fires through one of these, and ONLY for a confirmed `.action` — never on -/// `prepare`, never for a `.declined` / `.unavailable` review. - -// MARK: - Errors - -/// Failures a task side effect can report (surfaced to the executor's `.failed` state). -enum TaskError: Error, Equatable { - /// Calendar (EventKit) access was denied or restricted (spec: "Permission denied is handled"). - case calendarPermissionDenied - /// Reminders (EventKit) access was denied or restricted. - case remindersPermissionDenied - /// Contacts access was denied or restricted. - case contactsPermissionDenied - /// The parsed action was missing a field the side effect requires (defensive; prepare guards this). - case missingField(String) - /// A sink/store/opener/sender failed to apply the effect. - case sinkFailed(String) -} - -/// Human-facing messages so the executor's `.failed` state shows a readable string rather than the -/// raw enum case name (e.g. "calendarPermissionDenied"). Each permission case names the missing -/// permission and points to the relevant System Settings pane (spec: "A permission failure points to -/// the fix"). -extension TaskError: LocalizedError { - var errorDescription: String? { - switch self { - case .calendarPermissionDenied: - return "Calendar access is required. Open System Settings ▸ Privacy & Security ▸ Calendars to allow it." - case .remindersPermissionDenied: - return "Reminders access is required. Open System Settings ▸ Privacy & Security ▸ Reminders to allow it." - case .contactsPermissionDenied: - return "Contacts access is required. Open System Settings ▸ Privacy & Security ▸ Contacts to allow it." - case let .missingField(field): - return "The action was missing a required field: \(field)." - case let .sinkFailed(detail): - return detail - } - } -} - -// MARK: - Calendar - -/// Creates a calendar event. Production uses EventKit; tests record the event without touching the -/// system. `create` is called ONLY after a confirmed review (and, in prod, a granted permission). -@MainActor -protocol CalendarSink { - func create(_ event: ParsedCalendarEvent) async throws -} - -// MARK: - Reminders - -/// Creates a reminder/to-do. Production uses EventKit reminders; tests record it. Called ONLY after a -/// confirmed review (and, in prod, a granted Reminders permission). -@MainActor -protocol ReminderSink { - func create(_ reminder: ParsedReminder) async throws -} - -// MARK: - Contacts - -/// Creates a contact card. Production uses the Contacts framework; tests record it. Called ONLY after -/// a confirmed review (and, in prod, a granted Contacts permission). -@MainActor -protocol ContactSink { - func create(_ contact: ParsedContact) async throws -} - -// MARK: - Project note store - -/// Appends content (with source + timestamp) to a per-project note on disk. Production writes under -/// Application Support, mirroring `ClipboardStore`'s on-disk pattern; tests use a temp dir. -@MainActor -protocol ProjectStore { - func append(project: String, content: String, source: TaskSource) throws -} - -// MARK: - Tool opener - -/// Opens a target tool with a generated payload. Production writes a payload file + opens via the -/// launch mechanism; tests record the (tool, payload) pair. -@MainActor -protocol ToolOpener { - func open(tool: String, payload: String) async throws -} - -// MARK: - Destination sender - -/// Delivers content to a configured destination (Shortcut / URL scheme / shell-out). Tests record -/// the (destination, content) pair. -@MainActor -protocol DestinationSender { - func send(_ destination: Destination, content: String) async throws -} - -// MARK: - Production: EventKit calendar sink - -/// The production `CalendarSink`: maps `{title,start,end,attendees,notes}` to an `EKEvent` and saves -/// it. Requires Calendar permission, requested lazily by `PermissionsService` before `create` runs. -@MainActor -final class EventKitCalendarSink: CalendarSink { - private let store: EKEventStore - private let permissions: PermissionsService - - init(store: EKEventStore = EKEventStore(), permissions: PermissionsService) { - self.store = store - self.permissions = permissions - } - - func create(_ event: ParsedCalendarEvent) async throws { - // Lazy first-use permission (never at launch / opt-in — see permissions-onboarding). - let granted = await permissions.requestCalendarAccess() - guard granted else { throw TaskError.calendarPermissionDenied } - - guard let title = event.title, !title.isEmpty else { throw TaskError.missingField("title") } - let start = Self.parseDate(event.start) ?? Date() - let end = Self.parseDate(event.end) ?? start.addingTimeInterval(3600) - - let ek = EKEvent(eventStore: store) - ek.title = title - ek.startDate = start - ek.endDate = end - ek.notes = Self.composedNotes(for: event) - ek.calendar = store.defaultCalendarForNewEvents - do { - try store.save(ek, span: .thisEvent) - } catch { - // Clean prefix only; the raw OS error goes to the log, never into the user-facing message - // (spec: "No raw error text in user-facing strings"). - taskSinkLog.error("calendar save failed: \(String(describing: error), privacy: .public)") - throw TaskError.sinkFailed("Could not save the event to your calendar.") - } - } - - /// Fold attendees into the notes body (EventKit can't add arbitrary attendees without invites). - private static func composedNotes(for event: ParsedCalendarEvent) -> String? { - var parts: [String] = [] - if let notes = event.notes, !notes.isEmpty { parts.append(notes) } - if let attendees = event.attendees, !attendees.isEmpty { - parts.append("Attendees: " + attendees.joined(separator: ", ")) - } - return parts.isEmpty ? nil : parts.joined(separator: "\n\n") - } - - /// Best-effort ISO-8601 parse (with and without seconds / timezone) — never throws; nil on miss. - static func parseDate(_ string: String?) -> Date? { - guard let s = string, !s.isEmpty else { return nil } - let iso = ISO8601DateFormatter() - iso.formatOptions = [.withInternetDateTime] - if let d = iso.date(from: s) { return d } - iso.formatOptions = [.withInternetDateTime, .withFractionalSeconds] - if let d = iso.date(from: s) { return d } - // Tolerate a local "yyyy-MM-dd'T'HH:mm" without a timezone. - let df = DateFormatter() - df.locale = Locale(identifier: "en_US_POSIX") - for fmt in ["yyyy-MM-dd'T'HH:mm", "yyyy-MM-dd HH:mm", "yyyy-MM-dd"] { - df.dateFormat = fmt - if let d = df.date(from: s) { return d } - } - return nil - } -} - -// MARK: - Production: EventKit reminder sink - -/// The production `ReminderSink`: maps `{title, due, notes, priority}` to an `EKReminder` and saves it -/// to the default reminders list. Requires the Reminders permission, requested lazily by -/// `PermissionsService` before `create` runs (never at launch / opt-in — see permissions-onboarding). -@MainActor -final class EventKitReminderSink: ReminderSink { - private let store: EKEventStore - private let permissions: PermissionsService - - init(store: EKEventStore = EKEventStore(), permissions: PermissionsService) { - self.store = store - self.permissions = permissions - } - - func create(_ reminder: ParsedReminder) async throws { - let granted = await permissions.requestRemindersAccess() - guard granted else { throw TaskError.remindersPermissionDenied } - - guard let title = reminder.title, !title.isEmpty else { throw TaskError.missingField("title") } - let ek = EKReminder(eventStore: store) - ek.title = title - ek.notes = reminder.notes - if let priority = reminder.priority { ek.priority = priority } - if let date = EventKitCalendarSink.parseDate(reminder.due) { - ek.dueDateComponents = Calendar.current.dateComponents( - [.year, .month, .day, .hour, .minute], from: date) - } - ek.calendar = store.defaultCalendarForNewReminders() - do { - try store.save(ek, commit: true) - } catch { - taskSinkLog.error("reminder save failed: \(String(describing: error), privacy: .public)") - throw TaskError.sinkFailed("Could not save the reminder.") - } - } -} - -// MARK: - Production: Contacts sink - -/// The production `ContactSink`: maps `{name, email, phone, organization, notes}` to a `CNMutableContact` -/// and saves it via `CNSaveRequest`. Requires the Contacts permission, requested lazily before `create`. -@MainActor -final class ContactsSink: ContactSink { - private let store: CNContactStore - private let permissions: PermissionsService - - init(store: CNContactStore = CNContactStore(), permissions: PermissionsService) { - self.store = store - self.permissions = permissions - } - - func create(_ contact: ParsedContact) async throws { - let granted = await permissions.requestContactsAccess() - guard granted else { throw TaskError.contactsPermissionDenied } - - // A usable contact needs at least a name or a reachable handle; prepare guards this, but keep - // the sink defensive so it never saves an empty card. - let hasAny = [contact.name, contact.email, contact.phone, contact.organization] - .contains { ($0?.isEmpty == false) } - guard hasAny else { throw TaskError.missingField("name") } - - let card = CNMutableContact() - if let name = contact.name, !name.isEmpty { - // Split a full name into given/family on the last space (best effort). - if let lastSpace = name.range(of: " ", options: .backwards) { - card.givenName = String(name[.. URL { - let base = (try? FileManager.default.url(for: .applicationSupportDirectory, in: .userDomainMask, - appropriateFor: nil, create: true)) - ?? URL(fileURLWithPath: NSTemporaryDirectory()) - return base.appendingPathComponent("ThreeFingerSwitcher/projects", isDirectory: true) - } - - convenience init() { self.init(directory: Self.defaultDirectory()) } - - func append(project: String, content: String, source: TaskSource) throws { - // Map FileManager/FileHandle throws (disk full, permission, read-only volume) into a clean - // TaskError at this IO boundary — like the calendar sink — so a raw NSError never reaches the - // executor's fallback and dumps into the canvas (spec: "Errors are mapped at the boundary"). - do { - try FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true) - let url = noteURL(for: project) - let block = Self.entryBlock(content: content, source: source) - let data = Data(block.utf8) - if let handle = try? FileHandle(forWritingTo: url) { - defer { try? handle.close() } - try handle.seekToEnd() - try handle.write(contentsOf: data) - } else { - // First write: create the file (atomic), seeding it with the block. - try data.write(to: url, options: .atomic) - } - } catch { - taskSinkLog.error("project note write failed: \(String(describing: error), privacy: .public)") - throw TaskError.sinkFailed("Could not save the note to “\(project)”.") - } - } - - /// A safe, deterministic note filename for a project name (sanitized; never empty). - func noteURL(for project: String) -> URL { - directory.appendingPathComponent(Self.fileName(for: project)) - } - - nonisolated static func fileName(for project: String) -> String { - let allowed = CharacterSet.alphanumerics.union(CharacterSet(charactersIn: "-_ ")) - let cleaned = String(project.unicodeScalars.map { allowed.contains($0) ? Character($0) : "-" }) - .trimmingCharacters(in: .whitespaces) - let base = cleaned.isEmpty ? "project" : cleaned - return "\(base).md" - } - - /// Pure: the appended block (content + source app/URL + timestamp). Unit-testable without disk. - nonisolated static func entryBlock(content: String, source: TaskSource) -> String { - let df = ISO8601DateFormatter() - df.formatOptions = [.withInternetDateTime] - var header = "## " + df.string(from: source.timestamp) - var meta: [String] = [] - if let app = source.appName, !app.isEmpty { meta.append("from \(app)") } - if let url = source.url { meta.append(url.absoluteString) } - if !meta.isEmpty { header += " — " + meta.joined(separator: " · ") } - return "\n\(header)\n\n\(content)\n" - } -} - -// MARK: - Production: tool opener - -/// The production `ToolOpener`: writes the payload to a temp file and opens the target tool with it -/// via `NSWorkspace`/the launch mechanism (a bundle id, app path, or a `shortcuts run` for a named -/// Shortcut). Kept thin; the decision of HOW to open lives in `LaunchService` for real tools, this is -/// the task-side entry that hands off a payload file path. -@MainActor -final class WorkspaceToolOpener: ToolOpener { - /// Injected so the AppKit `open` is testable/replaceable; defaults to opening via `NSWorkspace`. - /// THROWS so a failed open is surfaced (not swallowed) — a tool that didn't actually open must - /// produce a `.failed` state, never a false "Done" (spec: "Failure is never silent"). - private let openHandler: @MainActor (_ tool: String, _ payloadFile: URL) async throws -> Void - - init(openHandler: @escaping @MainActor (_ tool: String, _ payloadFile: URL) async throws -> Void = WorkspaceToolOpener.defaultOpen) { - self.openHandler = openHandler - } - - func open(tool: String, payload: String) async throws { - let file = FileManager.default.temporaryDirectory - .appendingPathComponent("tfs-payload-\(UUID().uuidString).txt") - do { - try Data(payload.utf8).write(to: file, options: .atomic) - } catch { - taskSinkLog.error("payload write failed: \(String(describing: error), privacy: .public)") - throw TaskError.sinkFailed("Could not write the payload for “\(tool)”.") - } - try await openHandler(tool, file) - } - - /// Default open: a named Shortcut runs via `shortcuts run`; an app id/path opens the payload file - /// in that app. (Wired against the same primitives `LaunchService` uses.) Surfaces a real failure - /// (launch error, non-zero exit) as `TaskError.sinkFailed` rather than discarding it. - @MainActor - static func defaultOpen(_ tool: String, _ payloadFile: URL) async throws { - #if canImport(AppKit) - if tool.contains("/") || tool.hasSuffix(".app") { - // Treat as a bundle id or app path: open the payload file with that app. A bare dot - // (e.g. a Shortcut named "My.Workflow") is NOT treated as an app path. - try await openFile(payloadFile, withAppAt: URL(fileURLWithPath: tool)) - } else { - // Treat as a named Shortcut: run it (the payload file path is available to it). - let p = Process() - p.executableURL = URL(fileURLWithPath: "/usr/bin/shortcuts") - p.arguments = ["run", tool, "--input-path", payloadFile.path] - do { - try p.run() - } catch { - taskSinkLog.error("shortcuts run failed to launch: \(String(describing: error), privacy: .public)") - throw TaskError.sinkFailed("Could not run “\(tool)”.") - } - p.waitUntilExit() - if p.terminationStatus != 0 { - throw TaskError.sinkFailed("“\(tool)” reported an error (exit code \(p.terminationStatus)).") - } - } - #endif - } -} - -#if canImport(AppKit) -private extension WorkspaceToolOpener { - /// Open `payloadFile` with the app at `appURL`, awaiting the result so a failed open is surfaced - /// (the old fire-and-forget completion-handler form reported success unconditionally). - static func openFile(_ payloadFile: URL, withAppAt appURL: URL) async throws { - let config = NSWorkspace.OpenConfiguration() - do { - _ = try await NSWorkspace.shared.open([payloadFile], withApplicationAt: appURL, - configuration: config) - } catch { - taskSinkLog.error("NSWorkspace.open failed: \(String(describing: error), privacy: .public)") - throw TaskError.sinkFailed("Could not open “\(appURL.lastPathComponent)”.") - } - } -} -#endif - -// MARK: - Production: destination sender - -/// The production `DestinationSender`: routes content to a `Destination` adapter — a named Shortcut -/// (`shortcuts run`), a URL scheme (content substituted into `{content}` / appended), or a shell-out -/// (content on stdin). Mirrors `LaunchService`'s subprocess pattern. -@MainActor -final class AdapterDestinationSender: DestinationSender { - func send(_ destination: Destination, content: String) async throws { - switch destination { - case let .shortcut(name): - try runProcess("/usr/bin/shortcuts", ["run", name], stdin: content) - case let .urlScheme(template): - #if canImport(AppKit) - let urlString = Self.substitute(content, into: template) - guard let url = URL(string: urlString) else { - throw TaskError.sinkFailed("The destination URL was malformed.") - } - guard NSWorkspace.shared.open(url) else { - throw TaskError.sinkFailed("Nothing could open the destination URL.") - } - #endif - case let .shell(command): - try runProcess("/bin/zsh", ["-c", command], stdin: content) - } - } - - /// Substitute the content into a URL-scheme template: replace `{content}` if present, else append - /// it as a percent-encoded query-friendly suffix. Pure; unit-testable. - nonisolated static func substitute(_ content: String, into template: String) -> String { - let encoded = content.addingPercentEncoding(withAllowedCharacters: .urlQueryAllowed) ?? content - if template.contains("{content}") { - return template.replacingOccurrences(of: "{content}", with: encoded) - } - return template + encoded - } - - private func runProcess(_ executable: String, _ args: [String], stdin: String) throws { - let p = Process() - p.executableURL = URL(fileURLWithPath: executable) - p.arguments = args - let pipe = Pipe() - p.standardInput = pipe - do { - try p.run() - } catch { - taskSinkLog.error("destination adapter failed to launch: \(String(describing: error), privacy: .public)") - throw TaskError.sinkFailed("Could not run the destination.") - } - // The child may exit before we finish writing, so a blocking `write` can raise a broken-pipe - // failure — tolerate it (the adapter legitimately may not read stdin). The authoritative - // success signal is the exit STATUS below, not whether stdin was fully delivered. - if let data = stdin.data(using: .utf8) { - do { - try pipe.fileHandleForWriting.write(contentsOf: data) - } catch { - // Broken pipe / early exit: the input couldn't be delivered, but the process ran. - } - } - try? pipe.fileHandleForWriting.close() - p.waitUntilExit() - // A non-zero exit means the destination did NOT do its job — surface it, don't report "Done". - if p.terminationStatus != 0 { - throw TaskError.sinkFailed("The destination reported an error (exit code \(p.terminationStatus)).") - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Voice/PTTArmingModel.swift b/Sources/ThreeFingerSwitcher/AI/Voice/PTTArmingModel.swift deleted file mode 100644 index b085caf..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Voice/PTTArmingModel.swift +++ /dev/null @@ -1,134 +0,0 @@ -import Foundation - -/// The push-to-talk trigger state machine (`voice-double-tap-dwell-trigger`): **double-tap-then-hold**, -/// the macOS dictation idiom. Pure; the NSEvent monitor is a thin one-timer driver. -/// -/// Grammar: a first CLICK of the key (down and up, each within `tapMaxDuration`), a second press -/// within `secondTapGap`, HELD past `dwellTime` → push-to-talk begins (`firePTTDown`); release sends -/// (`firePTTUp`). Everything else is structurally a no-op: -/// - a LONG SINGLE hold is plain modifier use (special-character typing untouched), -/// - a bare double-tap without the dwell does nothing, -/// - any OTHER key at any pre-capture stage cancels to typing (chords can never trigger voice), -/// - the capture stack is untouched, and the mic closed, until the dwell elapses. -/// -/// Exactly ONE timer is pending in any phase (tap-max in `firstDown`, gap in `awaitingSecond`, -/// dwell in `dwelling`), so the driver needs a single one-shot slot and an undifferentiated -/// `.timerFired` event is unambiguous. -public struct PTTArmingModel { - - /// Max down-time (and up-time symmetry) for a press to read as a "click". - public static let tapMaxDuration: TimeInterval = 0.30 - /// Max wait between the first click's release and the second press. - public static let secondTapGap: TimeInterval = 0.30 - /// How long the second press must be held before capture begins. - public static let dwellTime: TimeInterval = 0.15 - - public enum Phase: Equatable, Sendable { - case idle - /// First press is down; the tap-max timer decides click vs plain hold. - case firstDown - /// First click completed; waiting (gap timer) for the second press. - case awaitingSecond - /// Second press is down; the dwell timer decides talk vs bare double-tap. - case dwelling - /// Push-to-talk is live (capture running). - case held - /// The key is down but the gesture is disqualified (long hold / chord) — wait for release. - case inert - } - - public enum Event: Equatable, Sendable { - case pttFlagDown - case pttFlagUp - /// Any OTHER key went down (the chord/typing signal, observed passively). - case otherKeyDown - /// The phase's one pending timer fired. - case timerFired - } - - public enum TimerKind: Equatable, Sendable { - case tapMax - case gap - case dwell - - public var duration: TimeInterval { - switch self { - case .tapMax: return PTTArmingModel.tapMaxDuration - case .gap: return PTTArmingModel.secondTapGap - case .dwell: return PTTArmingModel.dwellTime - } - } - } - - public enum Action: Equatable, Sendable { - case schedule(TimerKind) - case cancelTimer - /// Begin push-to-talk (capture starts HERE, never earlier). - case firePTTDown - /// End push-to-talk. - case firePTTUp - } - - public private(set) var phase: Phase = .idle - - public init() {} - - public mutating func handle(_ event: Event) -> [Action] { - switch (phase, event) { - - // MARK: idle - case (.idle, .pttFlagDown): - phase = .firstDown - return [.schedule(.tapMax)] - - // MARK: firstDown (click vs plain hold) - case (.firstDown, .pttFlagUp): - phase = .awaitingSecond - return [.cancelTimer, .schedule(.gap)] - case (.firstDown, .timerFired): - // Held too long for a click: plain modifier use — stand down until release. - phase = .inert - return [] - case (.firstDown, .otherKeyDown): - phase = .inert - return [.cancelTimer] - - // MARK: awaitingSecond (the gap) - case (.awaitingSecond, .pttFlagDown): - phase = .dwelling - return [.cancelTimer, .schedule(.dwell)] - case (.awaitingSecond, .timerFired): - phase = .idle // no second tap came - return [] - case (.awaitingSecond, .otherKeyDown): - phase = .idle // typing resumed between taps - return [.cancelTimer] - - // MARK: dwelling (talk vs bare double-tap) - case (.dwelling, .timerFired): - phase = .held - return [.firePTTDown] - case (.dwelling, .pttFlagUp): - phase = .idle // bare double-tap: a complete no-op - return [.cancelTimer] - case (.dwelling, .otherKeyDown): - phase = .inert - return [.cancelTimer] - - // MARK: held (talking) - case (.held, .pttFlagUp): - phase = .idle - return [.firePTTUp] - case (.held, .otherKeyDown): - return [] // keys while genuinely talking don't cancel - - // MARK: inert (disqualified; wait for the key to come back up) - case (.inert, .pttFlagUp): - phase = .idle - return [] - - default: - return [] - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Voice/PTTKeyMonitor.swift b/Sources/ThreeFingerSwitcher/AI/Voice/PTTKeyMonitor.swift deleted file mode 100644 index 18523e5..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Voice/PTTKeyMonitor.swift +++ /dev/null @@ -1,120 +0,0 @@ -import Foundation -import AppKit - -/// The push-to-talk trigger monitor (`add-voice-computer-use-agent` D4; **double-tap-then-hold** -/// grammar by `voice-double-tap-dwell-trigger`): passive `flagsChanged` + `keyDown` monitors -/// tracking one bare modifier (default **Right Option**, keyCode 61). Click, then press-and-HOLD -/// within the gap — capture begins at dwell-elapsed (`onDown`), release sends (`onUp`). Long single -/// holds, bare double-taps, and any typing chord are structural no-ops that never touch the capture -/// stack. Nothing is consumed or delayed; no new permission. The decision logic is the pure, -/// unit-tested `PTTArmingModel` — this class is the NSEvent/one-timer driver. -@MainActor -public final class PTTKeyMonitor { - - /// Right Option. Configurable via `AppSettings.voicePTTKeyCode`. - public static let defaultKeyCode: UInt16 = 61 - - public var onDown: (@MainActor () -> Void)? - public var onUp: (@MainActor () -> Void)? - /// Timer-duration scale (tests shrink it; production 1.0). - public var timerScale: Double = 1.0 - - private var keyCode: UInt16 - private var monitors: [Any] = [] - private var flagIsDown = false - private var model = PTTArmingModel() - private var pendingTimer: DispatchWorkItem? - - public init(keyCode: UInt16 = PTTKeyMonitor.defaultKeyCode) { - self.keyCode = keyCode - } - - public func setKeyCode(_ code: UInt16) { - keyCode = code - reset() - } - - public func start() { - guard monitors.isEmpty else { return } - let mask: NSEvent.EventTypeMask = [.flagsChanged, .keyDown] - if let global = NSEvent.addGlobalMonitorForEvents(matching: mask, handler: { [weak self] event in - MainActor.assumeIsolated { self?.handle(event) } - }) { - monitors.append(global) - } - if let local = NSEvent.addLocalMonitorForEvents(matching: mask, handler: { [weak self] event in - MainActor.assumeIsolated { self?.handle(event) } - return event // ALWAYS passed through — observation only - }) { - monitors.append(local) - } - } - - public func stop() { - for monitor in monitors { NSEvent.removeMonitor(monitor) } - monitors.removeAll() - reset() - } - - private func reset() { - pendingTimer?.cancel() - pendingTimer = nil - flagIsDown = false - model = PTTArmingModel() - } - - // MARK: - Event driving - - private func handle(_ event: NSEvent) { - switch event.type { - case .keyDown: - // Passive chord signal: any real key while arming = typing, never talk. - perform(model.handle(.otherKeyDown)) - case .flagsChanged: - guard event.keyCode == keyCode else { return } - let downNow = event.modifierFlags.contains(flagFamily(for: keyCode)) - guard downNow != flagIsDown else { return } - flagIsDown = downNow - perform(model.handle(downNow ? .pttFlagDown : .pttFlagUp)) - default: - break - } - } - - private func perform(_ actions: [PTTArmingModel.Action]) { - for action in actions { - switch action { - case let .schedule(kind): - pendingTimer?.cancel() - let work = DispatchWorkItem { [weak self] in - MainActor.assumeIsolated { - guard let self else { return } - self.pendingTimer = nil - self.perform(self.model.handle(.timerFired)) - } - } - pendingTimer = work - DispatchQueue.main.asyncAfter(deadline: .now() + kind.duration * timerScale, - execute: work) - case .cancelTimer: - pendingTimer?.cancel() - pendingTimer = nil - case .firePTTDown: - onDown?() - case .firePTTUp: - onUp?() - } - } - } - - private func flagFamily(for code: UInt16) -> NSEvent.ModifierFlags { - switch code { - case 61, 58: return .option - case 62, 59: return .control - case 60, 56: return .shift - case 54, 55: return .command - case 63: return .function - default: return .option - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Voice/SentenceChunker.swift b/Sources/ThreeFingerSwitcher/AI/Voice/SentenceChunker.swift deleted file mode 100644 index fc4abbb..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Voice/SentenceChunker.swift +++ /dev/null @@ -1,135 +0,0 @@ -import Foundation - -/// Pure sentence chunker (`add-voice-computer-use-agent`, design D3): consumes `.response`-channel -/// token text incrementally and emits speakable chunks at sentence boundaries, so the FIRST sentence -/// can start speaking while the rest of the reply is still generating. Rules: -/// - A chunk closes at a sentence terminator (`.` `!` `?` `…`) followed by whitespace/end, or at a -/// paragraph break (`\n\n`). -/// - A chunk that grows past `maxChunkLength` without a terminator flushes at the last whitespace -/// (never mid-word) so a long unpunctuated stream still speaks. -/// - Fenced code blocks (``` … ```) are NEVER read symbol-by-symbol: the whole fence collapses to a -/// spoken summary ("Code block, N lines."). The VISIBLE transcript keeps the full text — this -/// chunker only feeds the synthesizer. -/// - `flush()` emits whatever remains at stream end. -/// -/// Value type, no clocks, no I/O — the full behavior is unit-tested. -public struct SentenceChunker { - - public var maxChunkLength: Int - - private var buffer: String = "" - private var inFence = false - private var fenceLineCount = 0 - - public init(maxChunkLength: Int = 280) { - self.maxChunkLength = maxChunkLength - } - - /// Feed streamed text; returns every chunk that CLOSED as a result (usually zero or one; a large - /// paste can close several). - public mutating func consume(_ text: String) -> [String] { - var closed: [String] = [] - for character in text { - append(character, into: &closed) - } - return closed - } - - /// Stream end: emit the remainder (and close an unterminated fence honestly). - public mutating func flush() -> String? { - if inFence { - // An unterminated fence still summarizes (the reply was cut off mid-code). - let summary = fenceSummary() - inFence = false - fenceLineCount = 0 - buffer = "" - return summary - } - let rest = buffer.trimmingCharacters(in: .whitespacesAndNewlines) - buffer = "" - return rest.isEmpty ? nil : rest - } - - // MARK: - Internals - - private mutating func append(_ character: Character, into closed: inout [String]) { - buffer.append(character) - - // Fence tracking is line-oriented: a line that is exactly ``` (with optional language tag on - // open) toggles the fence. Detect on newline so partial tokens can't half-toggle. - if character == "\n" { - let lastLine = lastCompletedLine() - if lastLine.hasPrefix("```") { - if inFence { - // CLOSING fence: drop the buffered code, emit the summary chunk. - inFence = false - let summary = fenceSummary() - fenceLineCount = 0 - buffer = "" - closed.append(summary) - } else { - // OPENING fence: whatever preceded it closes as its own chunk first. - inFence = true - fenceLineCount = 0 - let before = String(buffer.dropLast(lastLine.count + 1)) - .trimmingCharacters(in: .whitespacesAndNewlines) - buffer = "" - if !before.isEmpty { closed.append(before) } - } - return - } - if inFence { - fenceLineCount += 1 - buffer = "" // code lines are never spoken; the count is the summary's payload - return - } - } - - guard !inFence else { return } - - // Paragraph break closes a chunk. - if buffer.hasSuffix("\n\n") { - let chunk = buffer.trimmingCharacters(in: .whitespacesAndNewlines) - buffer = "" - if !chunk.isEmpty { closed.append(chunk) } - return - } - - // Sentence terminator + trailing whitespace closes a chunk. - if character.isWhitespace { - let trimmedTail = buffer.dropLast() // the whitespace itself - if let last = trimmedTail.last, ".!?…".contains(last) { - let chunk = trimmedTail.trimmingCharacters(in: .whitespacesAndNewlines) - buffer = "" - if !chunk.isEmpty { closed.append(chunk) } - return - } - } - - // Length guard: flush at the last whitespace so speech never stalls on an unpunctuated run. - if buffer.count >= maxChunkLength { - if let cut = buffer.lastIndex(where: { $0.isWhitespace }), cut != buffer.startIndex { - let chunk = String(buffer[.. String { - // `buffer` ends with "\n"; the last completed line is between the previous newline and it. - let withoutTrailing = buffer.dropLast() - if let previous = withoutTrailing.lastIndex(of: "\n") { - return String(withoutTrailing[withoutTrailing.index(after: previous)...]) - } - return String(withoutTrailing) - } - - private func fenceSummary() -> String { - fenceLineCount == 1 ? "Code block, 1 line." : "Code block, \(fenceLineCount) lines." - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Voice/SpeechAnalyzerTranscriber.swift b/Sources/ThreeFingerSwitcher/AI/Voice/SpeechAnalyzerTranscriber.swift deleted file mode 100644 index 0c6e362..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Voice/SpeechAnalyzerTranscriber.swift +++ /dev/null @@ -1,177 +0,0 @@ -import Foundation -import AVFoundation -import Speech - -/// The real `SpeechTranscribing` conformer (`add-voice-computer-use-agent`, design D2): Apple's -/// on-device `SpeechAnalyzer`/`SpeechTranscriber` (macOS 26) fed by an `AVAudioEngine` input tap -/// with voice-processing I/O enabled (system echo cancellation — the physical precondition for -/// barge-in). Entirely `@available(macOS 26)`: on older systems `VoiceRuntimeInjection` resolves nil -/// and the controller surfaces `VoiceError.osTooOld` (the platform floor does not rise). -/// -/// Push-to-talk shape: `start()` opens the engine + analyzer; `stop()` (PTT release) finalizes the -/// analysis (`finalizeAndFinishThroughEndOfInput`) and the accumulated transcript arrives as the one -/// `isFinal` chunk; `cancel()` tears down without a final. The non-progressive `.transcription` -/// preset is deliberate: v1 consumes only the final utterance (no live captions), which sidesteps -/// volatile-result bookkeeping entirely. -/// -/// Boundary rule: Speech/AVFoundation errors are mapped to `VoiceError` HERE (raw text only in -/// `details`) — Core never sees vendor error types. -@available(macOS 26.0, *) -@MainActor -public final class SpeechAnalyzerTranscriber: SpeechTranscribing { - - private var engine: AVAudioEngine? - private var analyzer: SpeechAnalyzer? - private var module: SpeechTranscriber? - private var inputContinuation: AsyncStream.Continuation? - private var analysisTask: Task? - private var stopped = false - /// Process-wide "speech assets verified present" flag (`fix-ptt-chord-collision`). - private static var assetsEnsured = false - - public init() {} - - public func start() throws -> AsyncThrowingStream { - stopped = false - let engine = AVAudioEngine() - self.engine = engine - - // System AEC so the mic doesn't hear our own TTS. Best-effort: an engine that can't do - // voice processing still transcribes (barge-in-by-press works regardless). - try? engine.inputNode.setVoiceProcessingEnabled(true) - - let module = SpeechTranscriber(locale: .current, preset: .transcription) - let analyzer = SpeechAnalyzer(modules: [module]) - self.module = module - self.analyzer = analyzer - - let (inputSequence, inputBuilder) = AsyncStream.makeStream(of: AnalyzerInput.self) - inputContinuation = inputBuilder - - return AsyncThrowingStream { continuation in - let task = Task { @MainActor [weak self] in - guard let self else { return } - do { - // Language assets are system-managed: install on demand (a one-time, small, - // OS-owned download — NOT one of our model downloads). Checked ONCE per process - // (`fix-ptt-chord-collision`): a legit PTT press shouldn't pay an XPC round-trip - // to re-verify assets that were present seconds ago. - if !Self.assetsEnsured { - if let request = try await AssetInventory.assetInstallationRequest(supporting: [module]) { - try await request.downloadAndInstall() - } - Self.assetsEnsured = true - } - guard let analyzerFormat = await SpeechAnalyzer.bestAvailableAudioFormat( - compatibleWith: [module]) else { - throw VoiceError.speechUnavailable(detail: "no compatible audio format") - } - - // Tap the mic and convert each buffer to the analyzer's preferred format on the - // audio thread (the converter + continuation are the only captured state). - let inputFormat = engine.inputNode.outputFormat(forBus: 0) - guard let converter = AVAudioConverter(from: inputFormat, to: analyzerFormat) else { - throw VoiceError.captureFailed(detail: "no converter \(inputFormat)→\(analyzerFormat)") - } - let builder = inputBuilder - engine.inputNode.installTap(onBus: 0, bufferSize: 4096, format: inputFormat) { buffer, _ in - let ratio = analyzerFormat.sampleRate / inputFormat.sampleRate - let capacity = AVAudioFrameCount((Double(buffer.frameLength) * ratio).rounded(.up) + 16) - guard let converted = AVAudioPCMBuffer(pcmFormat: analyzerFormat, - frameCapacity: max(capacity, 1)) else { return } - var conversionError: NSError? - var served = false - converter.convert(to: converted, error: &conversionError) { _, status in - if served { - status.pointee = .noDataNow - return nil - } - served = true - status.pointee = .haveData - return buffer - } - guard conversionError == nil, converted.frameLength > 0 else { return } - builder.yield(AnalyzerInput(buffer: converted)) - } - - try engine.start() - try await analyzer.start(inputSequence: inputSequence) - - // Non-progressive preset: results arrive finalized; the sequence ends after - // `finalizeAndFinishThroughEndOfInput()` (driven by `stop()`). Concatenate every - // result's text into the one final transcript. - var transcript = "" - for try await result in module.results { - let piece = String(result.text.characters) - guard !piece.isEmpty else { continue } - transcript = transcript.isEmpty ? piece : transcript + " " + piece - } - continuation.yield(TranscriptChunk(transcript, isFinal: true)) - continuation.finish() - } catch let error as VoiceError { - continuation.finish(throwing: error) - } catch is CancellationError { - continuation.finish() - } catch { - // The boundary map: any Speech/AVFoundation failure crosses as taxonomy + details. - continuation.finish(throwing: VoiceError.speechUnavailable( - detail: String(describing: error))) - } - self.teardownEngine() - } - self.analysisTask = task - continuation.onTermination = { _ in task.cancel() } - } - } - - public func stop() { - guard !stopped else { return } - stopped = true - teardownEngine() - inputContinuation?.finish() - inputContinuation = nil - let analyzer = self.analyzer - Task { @MainActor in - // Finalize through end of input → the results sequence drains its final result and ends. - try? await analyzer?.finalizeAndFinishThroughEndOfInput() - } - } - - public func cancel() { - stopped = true - teardownEngine() - inputContinuation?.finish() - inputContinuation = nil - analysisTask?.cancel() - analysisTask = nil - let analyzer = self.analyzer - Task { @MainActor in - await analyzer?.cancelAndFinishNow() - } - } - - private func teardownEngine() { - guard let engine else { return } - engine.inputNode.removeTap(onBus: 0) - engine.stop() - self.engine = nil - } -} - -/// The lazy mic authorizer the controller injects (`add-voice-computer-use-agent` D11): requested on -/// the FIRST actual push-to-talk press, never at enable time. macOS 15-safe. -@MainActor -public enum MicrophoneAuthorizer { - public static func requestAccess() async -> Bool { - switch AVCaptureDevice.authorizationStatus(for: .audio) { - case .authorized: - return true - case .notDetermined: - return await AVCaptureDevice.requestAccess(for: .audio) - case .denied, .restricted: - return false - @unknown default: - return false - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Voice/SpeechSeams.swift b/Sources/ThreeFingerSwitcher/AI/Voice/SpeechSeams.swift deleted file mode 100644 index 504fd26..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Voice/SpeechSeams.swift +++ /dev/null @@ -1,117 +0,0 @@ -import Foundation - -/// The two voice seams (`add-voice-computer-use-agent`, design D2): Core declares the protocols and -/// ships scripted stubs so the ENTIRE voice lifecycle verifies under `swift test`; the real conformers -/// (`SpeechAnalyzerTranscriber` @available(macOS 26), `SystemSpeechSynthesizer` over -/// `AVSpeechSynthesizer`) are injected at composition via `VoiceRuntimeInjection`. Mirrors the -/// `LLMRuntime`/`StubLLMRuntime` and `MemoryPressureObserving`/fake idiom. - -/// One transcription update: partials stream while the user speaks; exactly one `isFinal` chunk -/// arrives after `stop()` (the finalized utterance the turn is built from). -public struct TranscriptChunk: Equatable, Sendable { - public var text: String - public var isFinal: Bool - public init(_ text: String, isFinal: Bool = false) { - self.text = text - self.isFinal = isFinal - } -} - -/// Speech-to-text over a push-to-talk window: `start()` opens capture and streams chunks; `stop()` -/// closes capture and finalizes (the stream emits its final chunk, then finishes). `cancel()` tears -/// down without a final (barge-in of one's own dictation / error paths). Failures throw/finish with -/// vendor errors mapped to `VoiceError` AT THE CONFORMER BOUNDARY — Core sees only the taxonomy. -@MainActor -public protocol SpeechTranscribing: AnyObject { - func start() throws -> AsyncThrowingStream - func stop() - func cancel() -} - -/// Text-to-speech with utterance queueing: `speak` enqueues a chunk (sentence-chunked upstream); -/// `stop()` halts and clears the queue immediately (barge-in). `onAllUtterancesFinished` fires on the -/// main actor when the queue drains (the voice turn returns to idle only after BOTH the generation -/// settles and the speech drains). -@MainActor -public protocol SpeechSynthesizing: AnyObject { - func speak(_ text: String) - func stop() - var isSpeaking: Bool { get } - var onAllUtterancesFinished: (@MainActor () -> Void)? { get set } -} - -// MARK: - Stubs (Core tests + the MLX-free dev build) - -/// Scripted transcriber: `start()` streams the scripted partials, `stop()` emits the scripted final. -/// Cancellation-aware so barge-in paths are testable. -@MainActor -public final class StubTranscriber: SpeechTranscribing { - public private(set) var isCapturing = false - /// The partial chunks streamed while "listening". - public var scriptedPartials: [String] - /// The final transcript emitted on `stop()`. - public var scriptedFinal: String - /// When set, `start()` throws it (permission/engine failure paths). - public var startError: VoiceError? - - private var continuation: AsyncThrowingStream.Continuation? - - public init(partials: [String] = [], final: String = "") { - self.scriptedPartials = partials - self.scriptedFinal = final - } - - public func start() throws -> AsyncThrowingStream { - if let startError { throw startError } - isCapturing = true - return AsyncThrowingStream { continuation in - self.continuation = continuation - for partial in self.scriptedPartials { - continuation.yield(TranscriptChunk(partial)) - } - } - } - - public func stop() { - guard isCapturing else { return } - isCapturing = false - continuation?.yield(TranscriptChunk(scriptedFinal, isFinal: true)) - continuation?.finish() - continuation = nil - } - - public func cancel() { - isCapturing = false - continuation?.finish() - continuation = nil - } -} - -/// Recording synthesizer: captures every spoken chunk in order; tests drive completion explicitly -/// via `finishAll()` so drain timing is deterministic. -@MainActor -public final class StubSynthesizer: SpeechSynthesizing { - public private(set) var spoken: [String] = [] - public private(set) var stopCount = 0 - public private(set) var isSpeaking = false - public var onAllUtterancesFinished: (@MainActor () -> Void)? - - public init() {} - - public func speak(_ text: String) { - spoken.append(text) - isSpeaking = true - } - - public func stop() { - stopCount += 1 - isSpeaking = false - } - - /// Simulate the queue draining (all enqueued utterances finished playing). - public func finishAll() { - guard isSpeaking else { return } - isSpeaking = false - onAllUtterancesFinished?() - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Voice/SystemSpeechSynthesizer.swift b/Sources/ThreeFingerSwitcher/AI/Voice/SystemSpeechSynthesizer.swift deleted file mode 100644 index 588f229..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Voice/SystemSpeechSynthesizer.swift +++ /dev/null @@ -1,63 +0,0 @@ -import Foundation -import AVFoundation - -/// The real `SpeechSynthesizing` conformer over `AVSpeechSynthesizer` (`add-voice-computer-use-agent` -/// design D2 — macOS 15-safe, no availability gate). Utterances arrive pre-chunked by -/// `SentenceChunker`, are enqueued in order, and `stop()` halts + clears immediately (the barge-in -/// contract). `onAllUtterancesFinished` fires on the main actor when the queue drains. AVFoundation -/// errors never escape raw: this boundary maps them (there is little to map — AVSpeech reports via -/// delegate callbacks, not thrown errors; a dead engine simply never speaks, which the turn model -/// tolerates by design). -@MainActor -public final class SystemSpeechSynthesizer: NSObject, SpeechSynthesizing { - - public private(set) var isSpeaking = false - public var onAllUtterancesFinished: (@MainActor () -> Void)? - - private let synthesizer = AVSpeechSynthesizer() - /// Outstanding utterances (enqueued, not yet finished/cancelled) — drives `isSpeaking` and the - /// drained callback without trusting AVSpeech's own `isSpeaking` timing. - private var outstanding = 0 - - public override init() { - super.init() - synthesizer.delegate = self - } - - public func speak(_ text: String) { - let utterance = AVSpeechUtterance(string: text) - // The system default voice; a voice picker is a named follow-up (design: Open Questions). - outstanding += 1 - isSpeaking = true - synthesizer.speak(utterance) - } - - public func stop() { - outstanding = 0 - isSpeaking = false - synthesizer.stopSpeaking(at: .immediate) - } - - private func utteranceSettled() { - guard outstanding > 0 else { return } - outstanding -= 1 - if outstanding == 0 { - isSpeaking = false - onAllUtterancesFinished?() - } - } -} - -extension SystemSpeechSynthesizer: AVSpeechSynthesizerDelegate { - public nonisolated func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, - didFinish utterance: AVSpeechUtterance) { - Task { @MainActor in self.utteranceSettled() } - } - - public nonisolated func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, - didCancel utterance: AVSpeechUtterance) { - // Cancellations arrive from `stop()`, which already zeroed the queue — but a defensive settle - // keeps the count honest if AVSpeech cancels for its own reasons. - Task { @MainActor in self.utteranceSettled() } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Voice/VoiceError.swift b/Sources/ThreeFingerSwitcher/AI/Voice/VoiceError.swift deleted file mode 100644 index 03e9da4..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Voice/VoiceError.swift +++ /dev/null @@ -1,49 +0,0 @@ -import Foundation - -/// The voice feature's error taxonomy (`add-voice-computer-use-agent`, design D11 / spec "Voice errors -/// join the single error taxonomy"). Parallel to `FileActionError`/`DockPreviewError`: a Core -/// `LocalizedError` with clean per-case headlines, populated at the AVFoundation/Speech boundary — -/// raw vendor error text rides ONLY in `copyableDetails`, never a headline. Surfaced bounded + -/// non-blocking (a card, never an `NSAlert`), translated by `AIError.message(for:)`. -public enum VoiceError: Error, Equatable { - /// Microphone authorization was denied (or restricted). The card offers a System Settings link. - case micDenied - /// The on-device speech engine is unavailable (asset missing, analyzer failed to start). Carries - /// the raw underlying text for the opt-in disclosure. - case speechUnavailable(detail: String? = nil) - /// Voice conversation requires macOS 26 (the `SpeechAnalyzer` floor). The feature reads as - /// unavailable on older systems — never a crash, never a degraded fallback engine. - case osTooOld - /// The audio capture session failed to start or died mid-press. - case captureFailed(detail: String? = nil) - /// Text-to-speech failed to start or errored mid-utterance. - case synthesisFailed(detail: String? = nil) - - /// The raw technical text for the opt-in "Show details / Copy" affordance; nil when the headline - /// already says everything. - public var copyableDetails: String? { - switch self { - case .micDenied, .osTooOld: - return nil - case let .speechUnavailable(detail), let .captureFailed(detail), let .synthesisFailed(detail): - return detail - } - } -} - -extension VoiceError: LocalizedError { - public var errorDescription: String? { - switch self { - case .micDenied: - return "Microphone access is off. Allow it in System Settings to talk to the assistant." - case .speechUnavailable: - return "Speech recognition isn't available right now." - case .osTooOld: - return "Voice conversation requires macOS 26." - case .captureFailed: - return "The microphone couldn't start." - case .synthesisFailed: - return "Speech playback failed." - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Voice/VoiceSessionController.swift b/Sources/ThreeFingerSwitcher/AI/Voice/VoiceSessionController.swift deleted file mode 100644 index 9c239ec..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Voice/VoiceSessionController.swift +++ /dev/null @@ -1,166 +0,0 @@ -import Foundation - -/// Executes the pure `VoiceTurnModel`'s effects through the seams (`add-voice-computer-use-agent`, -/// design D1/D2): mic authorization → transcriber lifecycle → the agent turn (a token stream) → -/// `SentenceChunker` → synthesizer, with barge-in and human-touch abort. `@MainActor` like every -/// observable controller in the app. The turn itself is a closure seam (`turnStarter`) so the -/// coordinator binds it to the real engine and tests bind a scripted stream — the controller knows -/// nothing about engines or runtimes. -@MainActor -public final class VoiceSessionController: ObservableObject { - - /// The UI-visible lifecycle phase (mic pill, thinking shimmer, speaking indicator). - @Published public private(set) var phase: VoiceTurnModel.Phase = .idle - /// The last voice failure, translated — a bounded, non-blocking card; nil when dismissed. - @Published public private(set) var lastFailure: AIPresentedError? - - /// Resolves the transcriber, nil when unavailable on this OS (`VoiceError.osTooOld` path). - private let transcriberFactory: @MainActor () -> SpeechTranscribing? - private let synthesizer: SpeechSynthesizing - /// Requests (or returns cached) mic authorization. Injected: real = AVCaptureDevice, tests = pure. - private let micAuthorizer: @MainActor () async -> Bool - /// Starts the agent turn for a finalized transcript and streams its tokens. - private let turnStarter: @MainActor (String) -> AsyncThrowingStream - /// Injected clock (the model takes time as an input). - private let now: @MainActor () -> Date - - private var model = VoiceTurnModel() - private var transcriber: SpeechTranscribing? - private var captureTask: Task? - private var turnTask: Task? - private var chunker = SentenceChunker() - - /// Whether a voice conversation is live RIGHT NOW — the voice feature's contribution to - /// `QuiescenceSnapshot.foregroundSessionActive` (spec: "A voice session is a foreground - /// conversational surface"). - public var isConversationActive: Bool { phase != .idle } - - public init(transcriberFactory: @escaping @MainActor () -> SpeechTranscribing?, - synthesizer: SpeechSynthesizing, - micAuthorizer: @escaping @MainActor () async -> Bool, - turnStarter: @escaping @MainActor (String) -> AsyncThrowingStream, - now: @escaping @MainActor () -> Date = { Date() }) { - self.transcriberFactory = transcriberFactory - self.synthesizer = synthesizer - self.micAuthorizer = micAuthorizer - self.turnStarter = turnStarter - self.now = now - synthesizer.onAllUtterancesFinished = { [weak self] in - self?.feed(.speechDrained) - } - } - - // MARK: - Inputs (the PTT trigger + the abort signal) - - public func pttDown() { feed(.pttDown) } - public func pttUp() { feed(.pttUp) } - /// A HUMAN trackpad touch while the agent is thinking/speaking — the kill switch. - public func humanTouch() { feed(.humanTouch) } - public func dismissFailure() { lastFailure = nil } - - // MARK: - The model loop - - private func feed(_ event: VoiceTurnModel.Event) { - let effects = model.handle(event, at: now()) - phase = model.phase - for effect in effects { execute(effect) } - } - - private func execute(_ effect: VoiceTurnModel.Effect) { - switch effect { - case .startCapture: - startCapture() - case .stopCapture: - transcriber?.stop() - case .cancelCapture: - captureTask?.cancel() - captureTask = nil - transcriber?.cancel() - transcriber = nil - case let .sendTurn(text, epoch): - startTurn(text, epoch: epoch) - case let .speak(chunk): - synthesizer.speak(chunk) - case .stopSpeaking: - synthesizer.stop() - case .cancelTurn: - turnTask?.cancel() - turnTask = nil - case let .presentFailure(error): - lastFailure = AIError.message(for: error) - } - } - - // MARK: - Capture - - private func startCapture() { - guard let transcriber = transcriberFactory() else { - feed(.voiceFailed(.osTooOld)) - return - } - self.transcriber = transcriber - captureTask?.cancel() - captureTask = Task { @MainActor [weak self] in - guard let self else { return } - // Lazy mic authorization on the FIRST actual press (spec: never at enable time). - guard await self.micAuthorizer() else { - self.feed(.voiceFailed(.micDenied)) - return - } - do { - let stream = try transcriber.start() - var finalText = "" - for try await chunk in stream { - if chunk.isFinal { finalText = chunk.text } - // Partials could drive a live caption; v1 keeps only the final. - } - guard !Task.isCancelled else { return } - self.feed(.transcriptFinal(finalText)) - } catch let error as VoiceError { - guard !Task.isCancelled else { return } - self.feed(.voiceFailed(error)) - } catch is CancellationError { - // Aborted capture: the model already left listening; nothing to report. - } catch { - guard !Task.isCancelled else { return } - // Boundary rule: an unmapped vendor error crosses as a taxonomy case with raw text - // only in details. - self.feed(.voiceFailed(.captureFailed(detail: String(describing: error)))) - } - } - } - - // MARK: - The turn - - private func startTurn(_ text: String, epoch: Int) { - chunker = SentenceChunker() - turnTask?.cancel() - turnTask = Task { @MainActor [weak self] in - guard let self else { return } - do { - let stream = self.turnStarter(text) - for try await token in stream { - try Task.checkCancellation() - // Thinking is NEVER spoken (spec); only the response channel feeds the chunker. - guard token.channel == .response else { continue } - for chunk in self.chunker.consume(token.text) { - self.feed(.chunkReady(chunk, epoch: epoch)) - } - } - try Task.checkCancellation() - if let rest = self.chunker.flush() { - self.feed(.chunkReady(rest, epoch: epoch)) - } - self.feed(.turnSettled(epoch: epoch)) - } catch is CancellationError { - // Barge-in / abort: a discard — the model already transitioned; late chunks from this - // epoch are dropped by the epoch guard even if any were in flight. - } catch { - guard !Task.isCancelled else { return } - // The canvas surface owns showing the translated failure; voice returns to idle - // without speaking an error at the user. - self.feed(.turnFailed(epoch: epoch)) - } - } - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Voice/VoiceToolContributor.swift b/Sources/ThreeFingerSwitcher/AI/Voice/VoiceToolContributor.swift deleted file mode 100644 index 8dc6117..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Voice/VoiceToolContributor.swift +++ /dev/null @@ -1,113 +0,0 @@ -import Foundation - -/// The voice-side tools (`add-voice-computer-use-agent`, design D6/D7): `speak` (.auto — the agent -/// narrates on demand and the loop uses it for progress) and the auto-mode pair — -/// `enable_auto_mode` (.confirm: the ONE approval that can't be skipped) / `disable_auto_mode` -/// (.auto: revocation is instant). Two descriptors instead of one boolean tool so the tier is -/// honest per direction. -struct VoiceToolContributor: ToolContributor { - - /// Live voice opt-in (speak is absent when voice is off; auto-mode tools ride the computer-use flag). - let voiceEnabled: @Sendable () -> Bool - let computerUseEnabled: @Sendable () -> Bool - /// Speak through the synthesizer seam (main-actor). - let speak: @MainActor (String) -> Void - /// Flip the CURRENT conversation's auto-approve grant (main-actor; wired per-engine). - let setAutoMode: @MainActor (Bool) -> Void - - static let speakName = "speak" - static let enableAutoName = "enable_auto_mode" - static let disableAutoName = "disable_auto_mode" - - func descriptors() -> [ToolDescriptor] { - var out: [ToolDescriptor] = [] - if voiceEnabled() { - out.append(ToolDescriptor( - name: Self.speakName, - summary: "Say a short message aloud to the user.", - argsSchema: StructuredSchema(name: Self.speakName, - json: #"{"type":"object","required":["text"],"properties":{"text":{"type":"string"}}}"#), - writePolicy: .auto, - keywords: ["speak", "say", "read", "aloud", "voice", "tell"])) - } - if computerUseEnabled() { - out.append(ToolDescriptor( - name: Self.enableAutoName, - summary: "Turn ON auto mode for this conversation: acts run without per-step approval (narrated).", - argsSchema: StructuredSchema(name: Self.enableAutoName, json: #"{"type":"object"}"#), - writePolicy: .confirm, - keywords: ["auto", "mode", "without asking", "automatically", "hands-free", "don't ask"])) - out.append(ToolDescriptor( - name: Self.disableAutoName, - summary: "Turn OFF auto mode for this conversation (acts need approval again).", - argsSchema: StructuredSchema(name: Self.disableAutoName, json: #"{"type":"object"}"#), - writePolicy: .auto, - keywords: ["auto", "mode", "off", "stop", "ask", "approval"])) - } - return out - } - - func canHandle(_ tool: String) -> Bool { - [Self.speakName, Self.enableAutoName, Self.disableAutoName].contains(tool) - } - - func run(_ call: RoutedCall, gate: ApprovalGate) async -> ToolStepResult { - switch call.descriptor.name { - case Self.speakName: - guard voiceEnabled() else { - return ToolStepResult(tool: call.descriptor.name, - status: .failed(headline: "Voice is turned off."), - summary: "Voice is turned off.") - } - let text = Self.textArgument(from: call.route.argumentsJSON) ?? call.userText - guard !text.isEmpty else { - return ToolStepResult(tool: Self.speakName, - status: .failed(headline: "There's nothing to say."), - summary: "speak needs text.") - } - await MainActor.run { speak(text) } - return ToolStepResult(tool: Self.speakName, status: .done, - summary: "Spoke: \(String(text.prefix(120)))") - - case Self.enableAutoName: - // The one grant that ALWAYS gates (spec: "Granting auto mode is itself gated"). - let review = TaskReview.action( - title: "Enable auto mode", - fields: [ReviewField("Scope", "this conversation"), - ReviewField("Effect", "clicks/typing run without per-step approval (narrated)")], - payload: .openTool(tool: Self.enableAutoName, - action: ParsedOpenTool(applicable: true, reason: nil, payload: "on"))) - switch await gate.awaitDecision(for: review) { - case .approve: - await MainActor.run { setAutoMode(true) } - return ToolStepResult(tool: Self.enableAutoName, status: .done, - summary: "Auto mode is ON for this conversation.") - case .skip: - return ToolStepResult(tool: Self.enableAutoName, status: .declined(reason: "skipped"), - summary: "Auto mode stays off.") - case .cancel: - return ToolStepResult(tool: Self.enableAutoName, - status: .declined(reason: TaskKindToolContributor.cancelledReason), - summary: "Cancelled.") - } - - case Self.disableAutoName: - await MainActor.run { setAutoMode(false) } // instant revoke, no gate - return ToolStepResult(tool: Self.disableAutoName, status: .done, - summary: "Auto mode is OFF.") - - default: - return ToolStepResult(tool: call.descriptor.name, - status: .failed(headline: "That tool isn't available."), - summary: "Unknown tool: \(call.descriptor.name).") - } - } - - static func textArgument(from json: String) -> String? { - guard let data = json.data(using: .utf8), - let object = try? JSONSerialization.jsonObject(with: data) as? [String: Any], - let text = object["text"] as? String else { return nil } - let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines) - return trimmed.isEmpty ? nil : trimmed - } -} diff --git a/Sources/ThreeFingerSwitcher/AI/Voice/VoiceTurnModel.swift b/Sources/ThreeFingerSwitcher/AI/Voice/VoiceTurnModel.swift deleted file mode 100644 index 3ca333f..0000000 --- a/Sources/ThreeFingerSwitcher/AI/Voice/VoiceTurnModel.swift +++ /dev/null @@ -1,178 +0,0 @@ -import Foundation - -/// The PURE voice-turn state machine (`add-voice-computer-use-agent`, design D1): owns the -/// push-to-talk conversation lifecycle — who may speak, when the mic is open, what happens on -/// barge-in — with time as an input and effects as outputs (the `DockHoverModel` idiom). It never -/// touches AVFoundation/Speech/the runtime; `VoiceSessionController` executes its effects through -/// the seams. Every transition below is unit-tested with fake timestamps. -/// -/// Lifecycle: `idle → listening (PTT held) → transcribing (released, STT finalizing) → thinking -/// (turn streaming) → speaking (TTS draining) → idle`. Barge-in: a NEW PTT press during -/// thinking/speaking stops speech, cancels the turn (a DISCARD), and opens the mic — a fluent -/// correction. A human trackpad touch during thinking/speaking ABORTS to idle (the trackpad is the -/// kill switch, never a talk trigger). Chunks arriving after a barge-in/abort are dropped on the -/// floor (`turnEpoch` guards staleness). -public struct VoiceTurnModel { - - public enum Phase: Equatable, Sendable { - case idle - case listening - case transcribing - case thinking - case speaking - } - - /// Inputs. Every event carries `at:` (unused by v1 rules beyond ordering, kept so cadence rules - /// can land without an API break — the time-as-input idiom). - public enum Event: Equatable, Sendable { - /// The push-to-talk trigger went down / up. - case pttDown - case pttUp - /// The transcriber finalized the utterance (after `stopCapture`). - case transcriptFinal(String) - /// The transcriber/capture failed. - case voiceFailed(VoiceError) - /// A speakable chunk closed (the controller runs `SentenceChunker` over the `.response` - /// stream). `epoch` stamps which turn produced it — stale chunks are dropped. - case chunkReady(String, epoch: Int) - /// The generation stream settled (all tokens delivered). `epoch` as above. - case turnSettled(epoch: Int) - /// The generation failed (already translated upstream; the canvas shows it — voice just - /// returns to idle without speaking a raw error). - case turnFailed(epoch: Int) - /// The synthesizer's queue drained. - case speechDrained - /// A HUMAN trackpad touch (the abort signal, never a talk trigger). - case humanTouch - } - - /// Outputs, executed by the controller in order. - public enum Effect: Equatable, Sendable { - case startCapture - case stopCapture - /// Cancel capture WITHOUT finalizing (abort paths — no turn is sent). - case cancelCapture - /// Send the finalized transcript as the agent turn, tagged with the new epoch. - case sendTurn(String, epoch: Int) - case speak(String) - case stopSpeaking - /// Cancel the in-flight generation (a discard, never a failure). - case cancelTurn - /// Surface a voice failure (bounded, non-blocking card). - case presentFailure(VoiceError) - } - - public private(set) var phase: Phase = .idle - /// The current turn's epoch. Incremented by every `sendTurn`; chunk/settle events from an older - /// epoch are stale (their turn was barged in / aborted) and are dropped. - public private(set) var turnEpoch = 0 - /// Whether the current turn's generation has settled (the phase leaves `speaking` only when BOTH - /// the turn settled AND the speech queue drained). - private var turnDone = false - /// Whether any speech has been enqueued for the current turn (a turn with no speakable output - /// returns to idle on settle without waiting for a drain that will never come). - private var spokeAnything = false - - public init() {} - - /// Advance the machine. Returns the effects to execute, in order. - public mutating func handle(_ event: Event, at _: Date) -> [Effect] { - switch (phase, event) { - - // MARK: idle - case (.idle, .pttDown): - phase = .listening - return [.startCapture] - - // MARK: listening (PTT held, mic open) - case (.listening, .pttUp): - phase = .transcribing - return [.stopCapture] - case (.listening, .voiceFailed(let error)): - phase = .idle - return [.cancelCapture, .presentFailure(error)] - case (.listening, .humanTouch): - // Touch while dictating aborts the dictation (no turn is sent). - phase = .idle - return [.cancelCapture] - - // MARK: transcribing (finalizing) - case (.transcribing, .transcriptFinal(let text)): - let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines) - guard !trimmed.isEmpty else { - phase = .idle // an empty press-and-release is a no-op, not an error - return [] - } - turnEpoch += 1 - turnDone = false - spokeAnything = false - phase = .thinking - return [.sendTurn(trimmed, epoch: turnEpoch)] - case (.transcribing, .voiceFailed(let error)): - phase = .idle - return [.presentFailure(error)] - case (.transcribing, .pttDown): - // Re-press while finalizing: abandon the finalize and listen again (fluent correction). - phase = .listening - return [.cancelCapture, .startCapture] - - // MARK: thinking (generation streaming, nothing spoken yet) - case (.thinking, .chunkReady(let chunk, let epoch)) where epoch == turnEpoch: - phase = .speaking - spokeAnything = true - return [.speak(chunk)] - case (.thinking, .turnSettled(let epoch)) where epoch == turnEpoch: - // Settled with no speakable output (e.g. a pure tool turn) → idle, nothing to drain. - turnDone = true - phase = .idle - return [] - case (.thinking, .turnFailed(let epoch)) where epoch == turnEpoch: - phase = .idle - return [] - case (.thinking, .pttDown): - return bargeIn() - case (.thinking, .humanTouch): - return abortTurn() - - // MARK: speaking (TTS draining while/after generation) - case (.speaking, .chunkReady(let chunk, let epoch)) where epoch == turnEpoch: - spokeAnything = true - return [.speak(chunk)] - case (.speaking, .turnSettled(let epoch)) where epoch == turnEpoch: - turnDone = true - return [] // wait for the drain - case (.speaking, .turnFailed(let epoch)) where epoch == turnEpoch: - turnDone = true - return [] // whatever was already enqueued finishes speaking - case (.speaking, .speechDrained): - if turnDone { - phase = .idle - } - return [] // drained but the stream is still going → stay speaking - case (.speaking, .pttDown): - return bargeIn() - case (.speaking, .humanTouch): - return abortTurn() - - // MARK: stale-epoch chunks/settles (a barged-in turn's leftovers) and everything else - case (_, .chunkReady), (_, .turnSettled), (_, .turnFailed): - return [] // dropped on the floor — never spoken (spec: barge-in) - default: - return [] - } - } - - /// Barge-in (design D4): stop output, discard the turn, and LISTEN. - private mutating func bargeIn() -> [Effect] { - phase = .listening - turnDone = false - return [.stopSpeaking, .cancelTurn, .startCapture] - } - - /// Human-touch abort: stop output, discard the turn, back to idle (no mic). - private mutating func abortTurn() -> [Effect] { - phase = .idle - turnDone = false - return [.stopSpeaking, .cancelTurn] - } -} diff --git a/Sources/ThreeFingerSwitcher/App/AppCoordinator.swift b/Sources/ThreeFingerSwitcher/App/AppCoordinator.swift index 20588c5..f4fe214 100644 --- a/Sources/ThreeFingerSwitcher/App/AppCoordinator.swift +++ b/Sources/ThreeFingerSwitcher/App/AppCoordinator.swift @@ -60,34 +60,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate private let windowGroups = WindowGroupStore() private lazy var snapMonitor = WindowSnapMonitor(store: windowGroups) - /// The notch home zone + notch-native sessions (`ai-parked-sessions` + `notch-native-conversations`): - /// the durable store, the one-active-now scheduler (K-ready for the batched runtime), the lifecycle - /// coordinator, the cursor-reveal rail, and the per-session conversation engines (born at the notch, - /// expanded in place). Gated by the agent feature (here: `aiCommandsEnabled`). - private lazy var parkController = ParkController( - maxParked: settings.agentMaxParkedSessions, - autoDismissCountdown: settings.agentParkAutoDismissCountdown, - revealDwell: settings.agentNotchRevealDwell, - // `unowned`: the coordinator owns the controller and outlives it (app-lifetime singleton), so the - // factory can't be called after self is gone; `weak` would force a nonsensical fallback engine. - engineFactory: { [unowned self] in self.makeNotchSessionEngine() }, - // The shared ledger, for the purge-delete gesture only (`notch-conversation-gestures`). - auditLog: auditLog) - .configuredForTuning( - // The in-notch settings zone (`notch-timeline-and-tuning`): the slider reads/writes the - // notch's own dial; the model max caps the "Max" stop's token caption + snapshot. - provider: { [weak self] in self?.settings.notchTuning ?? .balanced }, - modelMax: { [weak self] in self?.selectedAIModelDescriptor()?.maxContextTokens ?? 8_192 }, - onChange: { [weak self] tuning in self?.settings.notchTuning = tuning }) - - /// Coarse repeating timer for park MAINTENANCE: the (opt-in) auto-dismiss pass — an idle, fully-seen - /// session past the configured countdown is dismissed forever — plus the background-driver advance - /// pass (recovered turns, scheduled retries). Installed/torn down alongside `parkController`'s - /// enable (mirrors `previewRefreshTimer`). - private var parkAutoDismissTimer: Timer? - /// How often the maintenance passes run (a minute is a fine grain for expiry and retries). - private static let parkAutoDismissInterval: TimeInterval = 60 - /// The Keep Awake automation's stateful owner (`automations`). Idle until a `.automation(.keepAwake)` /// item is fired (which toggles it via `LaunchService.onAutomation`). Fed the touch stream for its /// first-touch-to-stop arming, and force-stopped on quit / will-sleep so a dimmed screen is never @@ -97,10 +69,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate // Four-finger launcher. private let favoritesStore = FavoritesStore.shared private let launcherOverlay = LauncherOverlayController() - /// The interactive screen-region picker (vision capture). Shown after a `screenRegion` AI command - /// dismisses the launcher; on a drag it captures the region and re-opens the canvas, on a - /// click-without-drag it cancels (`screen-region-picker`). - private let regionPicker = RegionPickerOverlay() private lazy var launchService = LaunchService( favoritesProvider: { [weak self] in self?.favoritesStore.favorites ?? Favorites() }, mover: SpaceWindowMover(), @@ -117,11 +85,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate // After a Next/Previous Space shortcut, focus the destination Space's front window once the // switch settles (the OS leaves it visually front but not key — same as the native shortcut). onSpaceSwitch: { [weak self] in self?.focusFrontWindowAfterSpaceSwitch() }, - // An AI command hands off to the executor, which opens the CONVERSATIONAL canvas (design D1/D5, - // task 6.1 FIRE path): a generic "Ask…" opens showing the seed and WAITING; a preset (Fix Grammar, - // Translate, a task) pre-fills + auto-sends turn 1. Firing does NOT dismiss the overlay (the overlay - // handles that exception). - onAICommand: { [weak self] command in self?.aiCommandExecutor.fire(command) }, // Persist the folder picked at fire time for a choose-folder-at-launch item, so its chooser // re-opens there next time (the item is the single source of truth for its last-used folder). onPromptedFolderChosen: { [weak self] itemID, bandID, folder in @@ -139,9 +102,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate lockOnStop: settings.lockOnStop)) } }, - // Speak-last-response from a launcher band (`add-speak-last-response-launcher-action`) — - // same verb as the menu-bar item. - onSpeakLastResponse: { [weak self] in self?.speakLastResponse() }, // The band carries only bounded/light clipboard previews (see `bandWindow`); resolve the FULL // entry by id at fire time so paste restores the complete payload, not the truncated preview. clipboardResolver: { [weak self] id in self?.clipboardStore.materializedEntry(id: id) } @@ -150,42 +110,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate /// Frontmost app captured at launcher-open time (target for `.action(.closeFrontWindow)`). private var capturedFrontApp: NSRunningApplication? - // Files band (opt-in; the synthetic Files band + the on-demand directory navigator). Like Clipboard, - // the band is synthetic and ephemeral — projected fresh on every launcher open from the current - // directory column, never persisted into Favorites. - /// Opens a chosen Files entry (a file in its default app / via Open-With, a folder as a Finder window) - /// as a defusable held open (design D7). Reuses `capturedFrontApp` exactly like `LaunchService` / - /// `SelectionService`, so the open targets the app the user was looking at before the non-activating - /// overlay appeared. `SystemFileWorkspace` maps every OS error to a typed `FileActionError` at the - /// boundary, surfaced only through the service's bounded `.failed` state. - private lazy var fileOpenService = FileOpenService( - workspace: SystemFileWorkspace(), - activateFrontAppContext: { [weak self] in - // Re-assert the captured front app before the open fires (mirrors `SelectionService`), so the - // opened document lands in the context the user was looking at, not the frontmost app at fire - // time. The `activate` result is best-effort and intentionally discarded. - _ = self?.capturedFrontApp?.activate(options: []) - } - ) - - /// The last Files open that was fired (default open / Open-With), captured so the failure row's **Retry** - /// can re-fire the identical open through `FileOpenService`. Set in `filesOpen`/`filesOpenWith`, replaced - /// by each new open; nil until the first Files open. A closure (not the entry) so the same defusable - /// prepare→commit path runs verbatim on a retry. - private var lastFilesOpen: (() -> Void)? - - /// When the Open-With app grid was reached via the action menu's "Open in ▸", the entry it was opened - /// for — so a discard backs out to the **action menu** (one level), not straight to the folder list. - /// Nil when the picker isn't open or was opened directly. - private var filesPickerOriginEntry: FileEntry? - - /// A pending **Cut** (move-on-Paste, Finder ⌘X): the file(s) cut and the pasteboard `changeCount` at cut - /// time. The next `pasteInto` MOVES them only while `NSPasteboard.general.changeCount` still equals this - /// (the pasteboard is still that cut); a Copy or any other write since bumps the count → the cut is - /// superseded and Paste copies. Cleared after the move (or when superseded). Coordinator state, so a cut - /// persists across launcher sessions until consumed — matching Finder. - private var pendingCut: (sources: [URL], changeCount: Int)? - // Clipboard history (opt-in; the synthetic Clipboard band + the background recorder). private let clipboardStore = ClipboardStore.shared private lazy var clipboardMonitor = ClipboardMonitor(store: clipboardStore) @@ -212,332 +136,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate return DeviceIdentity(id: id, name: Host.current().localizedName ?? "Mac") } - // AI commands (opt-in; the on-device model + the streaming canvas). AI commands now live as - // persisted items inside the favorites bands (configuration-hub fold-in), so there is no separate - // command store — a fired `.aiCommand` item carries its `AICommand` to the executor directly. - /// Reads/writes the captured front app's selection (and the clipboard / screen region) for an AI - /// command. Reuses `capturedFrontApp` exactly like `LaunchService`, so output lands in the app the - /// user was looking at (the overlay is non-activating). - private lazy var selectionService = SelectionService( - frontAppProvider: { [weak self] in self?.capturedFrontApp ?? NSWorkspace.shared.frontmostApplication } - ) - /// Manages the on-device model lifecycle. Until the real MLX/Gemma runtime (phase 10) is wired, - /// this runs against a **dev stub**: a `StubLLMRuntime` + a registry whose integrity SHA matches a - /// fabricated dev payload, so download/verify/load succeed WITHOUT a real multi-gigabyte fetch and - /// the streaming canvas is fully usable in a signed build today. Swapping in the real runtime is a - /// one-line `runtimeFactory` change (design D1/D7) — feature code never sees a concrete model. - private lazy var modelManager: ModelManager = { - // Resolve the two heavy GPU/CPU gates through the SINGLE resolver (`FullPotentialGate.isUnlocked`): - // a master-OFF or ai-commands-OFF locks BOTH at once (the calm panic-off), and each sub-flag gates - // only its own capability. The factory then constructs the CPU ternary lane / multi-stream batched - // runtime only when unlocked; OFF → today's single-GPU-lane, single-session build. - let gate = settings.fullPotentialGate - return AIRuntimeInjection.modelManagerFactory?(settings.aiCommandsEnabled, - gate.isUnlocked(.cpuLane), - gate.isUnlocked(.batchedRuntime)) - ?? DevAIRuntime.makeModelManager(optedIn: settings.aiCommandsEnabled) - }() - /// The agentic task layer (calendar / save-to-project / open-tool / send-to), driven by the model's - /// structured output. Calendar permission is requested lazily at first calendar-task use. - private lazy var taskDispatcher = TaskDispatcher( - modelManager: modelManager, - permissions: permissions - ) - /// Orchestrates one AI command fire end-to-end (acquire → stream → commit), exposing the observable - /// state the launcher's preview canvas binds to. The context provider supplies the captured app - /// name so `{app}` resolves; input is filled by acquisition. - /// The append-only audit ledger ("what did my agents do while I was away", `ai-background-autonomy`, - /// design Decision 5). A single durable `DiskAuditLog` instance: the background route-loop host (when - /// wired) records every tool step here, and the Hub AI page's audit viewer reads `recent(limit:)` - /// synchronously + surfaces `lastPersistError` as a bounded, non-blocking banner. MLX-free Core. - private lazy var auditLog = DiskAuditLog() - /// The live tool registry the AI route loop advertises + dispatches through (`wire-tool-routing`). - /// v1 contributes the parameterless side-effecting `TaskKind`s (EventKit calendar/reminder, Contacts) - /// as routable tools, bridged back into the UNCHANGED `taskDispatcher`. This is the EXTENSION POINT: - /// later waves ADD contributors to this array (`MemoryToolContributor`, `SkillToolProvider`, - /// `MediaToolContributor`, `ClaudeHandoffContributor`) — no loop change, just more descriptors. - // Generative media (`wire-media-tool`): the `generate_image` tool routes to the app's image - // `MediaRuntime` and FAILS HONESTLY until Wave 2 builds the diffusion pipeline (a clean bounded - // `MediaError`, never the model's raw tool-call text, never a blank/false-Done). The runtime is - // injected at the same seam as Gemma (`AIRuntimeInjection.imageRuntimeFactory`, set in `main.swift`); - // Core/test builds leave it nil so the tool simply isn't advertised (MLX-free). - /// The persisted image-model selection (Q4 default / FP16 opt-in). No Hub picker is wired yet, so this - /// resolves to `nil` → the Q4 default (`ImageModelCatalog.selected`); when an `imageModelID` setting - /// lands it threads through here unchanged. - private var aiImageModelID: String? { nil } - /// The image backend for the selected `imageModelID`, or nil in a Core/test build. Resolved once from - /// the injected factory. - private lazy var aiImageRuntime: MediaRuntime? = - AIRuntimeInjection.imageRuntimeFactory?(aiImageModelID) - /// Output #1 — the generated-media gallery (a Files-band `.fileEntry` source). Local-only. - private let aiMediaGallery = MediaGallery() - /// The thread-safe live mirror of the AI route loop's `AppSettings`-derived gating inputs (Full - /// Potential master / media / cloud gates + the cloud-video budget). The route loop's contributors run - /// OFF the main actor (`AgentLoop` is a non-isolated `Sendable` struct), so their `@Sendable` gating - /// closures MUST read these values from any thread WITHOUT `MainActor.assumeIsolated` (which traps off - /// main — the first-routed-tool crash). The main actor refreshes it on every relevant settings change - /// (`refreshAIGatingSnapshot()`), so gating stays LIVE, not a stale build-time snapshot. - private let aiGatingSnapshot = AIGatingSnapshot() - /// The cloud-video per-day budget cap (consumed by the contributor + sink). Video has no provider - /// wired yet, so this is effectively dormant; the cap reads the live snapshot (refreshed on every - /// settings change) — thread-safe off the main actor, where the route loop's sink runs. - private lazy var aiMediaVideoBudget = PerDayVideoBudget( - cap: { [aiGatingSnapshot] in aiGatingSnapshot.mediaVideoBudgetPerDay }) - /// The route-loop executor for a routed media call — drives the runtime, threads progress, writes the - /// gallery asset, and returns a clean `.done`/`.declined`/`.failed` step. Video runtime is nil (its - /// own wave); image is the injected `aiImageRuntime`. - private lazy var aiMediaGenSink = MediaGenSink( - imageRuntime: aiImageRuntime, - videoRuntime: nil, - gallery: aiMediaGallery, - budget: aiMediaVideoBudget, - audit: auditLog, - imageModelID: aiImageModelID) - /// The media-tool availability gate (master ∧ media floor; cloud-video extras). Reads the live - /// `fullPotentialEnabled`/`mediaGenEnabled` flags; no video provider is wired, so `generate_video` - /// stays dark until that wave. - private lazy var aiMediaAvailability = MediaToolAvailability( - // These `@Sendable` predicates are invoked by `MediaToolContributor.descriptors()`/`videoAvailable`, - // which the route loop reaches OFF the main actor (`AgentLoop` is a non-isolated `Sendable` struct). - // So they MUST read the gating flags thread-safely, NOT via `MainActor.assumeIsolated` (which TRAPS - // off main — the first-routed-tool crash). They read the live `aiGatingSnapshot`, which the main - // actor refreshes on every relevant settings change, so gating stays LIVE (never a stale snapshot). - // Each gate already routed through the SINGLE resolver (`FullPotentialGate.isUnlocked`) at refresh - // time — so `mediaGen` / `fleetCloud` are honestly locked whenever the master or ai-commands opt-in - // is off (the calm panic-off), never just their own sub-flag. `isFullPotentialEnabled` stays the raw - // master read (the contributor ANDs it with `isMediaGenEnabled`, which is already the full gate). - isFullPotentialEnabled: { [aiGatingSnapshot] in aiGatingSnapshot.isFullPotentialEnabled }, - isMediaGenEnabled: { [aiGatingSnapshot] in aiGatingSnapshot.isMediaGenUnlocked }, - isCloudEscalationEnabled: { [aiGatingSnapshot] in aiGatingSnapshot.isFleetCloudUnlocked }, - hasVideoProvider: { false }) - - // MARK: - Memory + Skills (`wire-memory-skills`) - // - // The agent's long-term memory store: `core.md` facts + `subfiles/` notes under Application Support - // (`…/ThreeFingerSwitcher/memory`, created on first write). MLX-free Core; the `MemoryToolProvider` - // projects it into the `memory.read` (.auto) / `memory.write`/`update`/`forget`/`promote` (.confirm) - // routable tools the registry advertises. The default-directory init points at the live folder. - private let aiMemoryStore = MemoryStore() - private lazy var aiMemoryToolProvider = MemoryToolProvider(store: aiMemoryStore) - - // The skills store: built-in skills are projected in-memory from `AICommandCatalog` (the catalog stays - // the source of truth) ∪ user `.skill.md` files dropped into `…/ThreeFingerSwitcher/Skills/`. The - // folder watcher coalesces edits into an off-main reload that republishes the snapshot (no rebuild). - // The migration is a no-op by design (built-ins are projected, never written to disk) and idempotent. - private let aiSkillStore = SkillStore(userFolder: AppCoordinator.skillsUserFolder()) - // A lazily-resolving runtime: skills generate their text result through the resident model, loaded - // lazily by `ModelManager`. The forwarder resolves it at call time (cheap — kept resident) so the - // registry can be built before any model is loaded; a no-model state surfaces as a clean `.failed`. - private lazy var aiSkillRuntime = ForwardingLLMRuntime( - resolve: { [modelManager] caps in try await modelManager.runtime(requiring: caps) }) - private lazy var aiSkillToolProvider = SkillToolProvider( - manifests: SkillStore.builtInManifests(), - runtime: aiSkillRuntime, - dispatcher: taskDispatcher, - globalReasoning: settings.aiReasoningEnabled) - - /// Watches the user `Skills/` folder and coalesces edits into an off-main `store.loadAll()` that - /// republishes the snapshot — a dropped/edited `.skill.md` appears with NO rebuild. Built-ins are - /// projected in-memory (loaded once, never watched). Started in `start()`; nil until then. - private var aiSkillWatcher: SkillFolderWatcher? - - /// The live Skills user folder (`…/Application Support/ThreeFingerSwitcher/Skills`, created on first - /// run by the store's load). Parallels `MemoryStore.defaultDirectory()`. - private static func skillsUserFolder() -> URL { - let base = (try? FileManager.default.url(for: .applicationSupportDirectory, in: .userDomainMask, - appropriateFor: nil, create: true)) - ?? URL(fileURLWithPath: NSTemporaryDirectory()) - return base.appendingPathComponent("ThreeFingerSwitcher/Skills", isDirectory: true) - } - - private lazy var aiToolRegistry = ToolRegistry([ - TaskKindToolContributor( - dispatcher: taskDispatcher, - kinds: [.addToCalendar, .addToReminder, .newContact]), - // Generative media (`wire-media-tool`): advertises `generate_image` (and later `generate_video`) - // when the master ∧ media flags are on AND a capable runtime is wired; dispatches to the sink. - MediaToolContributor( - availability: aiMediaAvailability, - imageRuntime: aiImageRuntime, - videoRuntime: nil, - budget: aiMediaVideoBudget, - sink: aiMediaGenSink), - // Long-term memory (`wire-memory-skills`): `memory.read`/`write`/`update`/`forget`/`promote`. - aiMemoryToolProvider, - // Skills as tools (`wire-memory-skills`): each built-in/user skill projected as a routable tool. - aiSkillToolProvider, - // Subagents (`refactor-park-and-background-agents`): fixed fresh-context templates as routable - // tools — context hygiene, not parallelism; only the summary re-enters the orchestrator thread. - SubagentToolContributor( - templates: Subagent.builtIns, - runtimeProvider: { [modelManager] in try await modelManager.runtime(requiring: [.text]) }), - // Computer use (`add-voice-computer-use-agent`): AX-first read/focus/click/type over the - // switcher's own enumeration + commit path. Flag-gated LIVE (off = absent from candidates); - // UserDefaults reads are thread-safe (the loop queries descriptors off-main). - ComputerUseToolContributor( - enabled: { UserDefaults.standard.bool(forKey: "computerUseEnabled") }, - resolveWindow: { [weak self] app, title in self?.resolveComputerUseWindow(app: app, title: title) }, - focusWindow: { [weak self] target in self?.focusComputerUseWindow(target) ?? false }, - performer: aiAXPerformer, - arbiter: aiActionArbiter, - narrate: { [weak self] text in self?.voiceNarrate(text) }), - // Voice tools: `speak` + the gated auto-mode pair. - VoiceToolContributor( - voiceEnabled: { UserDefaults.standard.bool(forKey: "voiceConversationEnabled") }, - computerUseEnabled: { UserDefaults.standard.bool(forKey: "computerUseEnabled") }, - speak: { [weak self] text in self?.voiceSpeak(text) }, - setAutoMode: { [weak self] on in self?.setConversationAutoMode(on) }) - // + future contributors here. - ]) - - // MARK: - Voice + computer-use composition (`add-voice-computer-use-agent`) - - /// Hot-path arming flags (`fix-evict-thrash-and-hot-path`): the touch stream is the - /// latency-critical gesture pipeline, so its abort hook reads ONLY these plain stored Bools. - /// They're maintained by state-change sinks installed inside the lazy initializers below — a - /// touch frame can never instantiate the agent/voice stack and never reads settings. - private var agentActingNow = false - private var voicePhaseLive = false - /// Whether the lazy voice stack has ever been built (guards the quiescence read the same way). - private var voiceStackLive = false - - /// The agent-vs-human input arbitration: tagged synthetic input + the any-touch kill switch. - private lazy var aiActionArbiter: AgentActionArbiter = { - let arbiter = AgentActionArbiter() - arbiter.onAbort = { [weak self] in self?.abortAgentAction() } - // Arm/disarm the hot-path flag on acting-state CHANGES (rare), never per frame. - arbiter.$isActing - .sink { [weak self] acting in - MainActor.assumeIsolated { self?.agentActingNow = acting } - } - .store(in: &cancellables) - return arbiter - }() - /// The AX read/act engine, posting through the arbiter's tagged event source. - private lazy var aiAXPerformer = AXActionPerformer(eventSource: aiActionArbiter.eventSource) - /// The process-wide synthesizer (sentence chunks + narration + speak-last-response). - private lazy var aiSpeechSynthesizer = SystemSpeechSynthesizer() - /// The voice conversation's transcript (in-memory v1 — voice sessions aren't parked rows yet). - private var voiceConversation = AgentConversation(title: "Voice", messages: []) - /// The VOICE conversation's auto-approve grant (per-engine grants live on the engines). - private let voiceAutoGrant = LockedBool() - /// The push-to-talk hold-key monitor (default Right Option), installed only while voice is on. - private lazy var pttMonitor = PTTKeyMonitor() - /// The voice session controller: pure turn model + seams; the turn runs the SAME agent loop and - /// tool registry as typed chat (voice is an input/output mode, not a second agent). - private lazy var voiceController: VoiceSessionController = { - let controller = VoiceSessionController( - transcriberFactory: { - if #available(macOS 26.0, *) { return SpeechAnalyzerTranscriber() } - return nil - }, - synthesizer: aiSpeechSynthesizer, - micAuthorizer: { await MicrophoneAuthorizer.requestAccess() }, - turnStarter: { [weak self] text in - self?.voiceTurnStream(text) ?? AsyncThrowingStream { $0.finish() } - }) - voiceStackLive = true - // Arm the hot-path abort flag only for the phases a touch should interrupt. - controller.$phase - .sink { [weak self] phase in - MainActor.assumeIsolated { - self?.voicePhaseLive = (phase == .thinking || phase == .speaking) - } - } - .store(in: &cancellables) - return controller - }() - /// The v1 candidate retriever: cheap keyword ranking over the registry's descriptors, offering ~5 - /// tools per turn (the loop also offers `widen_candidates` so the model can ask for more). - private lazy var aiToolCandidateSource = - KeywordToolCandidateSource(all: { [aiToolRegistry] in aiToolRegistry.allDescriptors() }) - - /// Background autonomy (`ai-background-autonomy`): the per-step auto-vs-escalate runner. Built ONLY - /// when `FullPotentialGate.isUnlocked(.backgroundAutonomy)` (master ∧ sub-flag ∧ ai-commands) — when - /// LOCKED this is `nil`, so the `AICommandExecutor`/`AgentLoop` take the plain foreground path: a - /// parked session's step simply doesn't auto-run/escalate (calmly inert, no error, never a false - /// "Done"). Reads the live whitelist (the trust boundary) and routes escalation + park-state through - /// the live `ParkController`; every step is audited to the shared `auditLog`. - private lazy var aiBackgroundRunner: BackgroundToolRunner? = { - guard settings.fullPotentialGate.isUnlocked(.backgroundAutonomy) else { return nil } - // The route loop drives `BackgroundToolRunner.run` (and thus `parkStateOf`) OFF the main actor, so - // this `@Sendable` closure MUST read park state thread-safely, NOT via `MainActor.assumeIsolated` - // (which TRAPS off main). `ParkScheduler.parkState(of:)` is the lock-guarded, any-thread read (the - // same live state `ParkController.parkState(of:)` exposes) — no main-actor hop, fully live (an - // unknown id → `.active`, the foreground path). - let scheduler = parkController.parkScheduler - return BackgroundToolRunner( - resolver: BackgroundPolicyResolver(whitelist: settings.agentWhitelist), - audit: auditLog, - // Escalation routes through the CONTROLLER (persist + repaint — - // `refactor-park-and-background-agents`), hopping to the main actor from the off-main loop: - // a background needs-you survives relaunch and lights the rail, never a silent in-memory - // scheduler flag. - onEscalate: { [weak self] id, reason in - Task { @MainActor in self?.parkController.escalate(id, reason: reason) } - }, - parkStateOf: { id in scheduler.parkState(of: id) }) - }() - - private lazy var aiCommandExecutor = AICommandExecutor( - modelManager: modelManager, - selection: selectionService, - dispatcher: taskDispatcher, - contextProvider: { [weak self] in - FireContext(capturedAppName: self?.capturedFrontApp?.localizedName) - }, - loadLanguage: { [weak self] id in self?.settings.rememberedLanguage(for: id) }, - saveLanguage: { [weak self] id, lang in self?.settings.rememberLanguage(lang, for: id) }, - reasoning: { [weak self] in self?.settings.aiReasoningEnabled ?? false } - ) - - /// The notch-session conversation engine factory (`notch-native-conversations` D1): each notch session - /// with a live foreground turn gets its own engine; `ParkController` owns the instances. The engine - /// carries the tool-routing/skills/background-autonomy seams the executor shed when the command band - /// reverted to one-shot presets. - private func makeNotchSessionEngine() -> NotchSessionEngine { - NotchSessionEngine( - modelManager: modelManager, - selection: selectionService, - contextProvider: { [weak self] in - FireContext(capturedAppName: self?.capturedFrontApp?.localizedName) - }, - reasoning: { [weak self] in self?.settings.aiReasoningEnabled ?? false }, - // BORN-WITH tuning (`notch-timeline-and-tuning`): a NEW notch conversation snapshots the - // notch dial (reasoning + context tokens, clamped to the model max) at birth and carries it - // for life; the legacy `reasoning:` closure above remains the fallback for pre-change - // conversations with no stored tuning. - tuningDefault: { [weak self] in - guard let self else { return nil } - let tuning = self.settings.notchTuning - let modelMax = self.selectedAIModelDescriptor()?.maxContextTokens ?? 8_192 - return (reasoning: tuning.reasoning, - contextTokens: tuning.contextTokens(modelMax: modelMax)) - }, - registry: aiToolRegistry, - candidateSource: aiToolCandidateSource, - // Active-skill allow-list (`wire-memory-skills`): the bound skill's `toolNames` are always - // offered as candidates while it drives the conversation. Resolves through the live store - // (user skills, post-reload) with a synchronous built-in projection fallback. - skillTools: { [aiSkillStore] id in - (aiSkillStore.manifest(id: id) - ?? SkillStore.builtInManifests().first { $0.id == id })?.toolNames ?? [] - }, - // Background autonomy: nil when `.backgroundAutonomy` is locked (the plain foreground path). - backgroundRunner: aiBackgroundRunner, - // The loop's wall-clock bounds (`add-voice-computer-use-agent` D8), settings-fed live. - loopBudget: { [weak self] in - guard let self else { return .default } - return LoopBudget(stepTimeout: TimeInterval(self.settings.agentStepTimeoutSeconds), - turnDeadline: TimeInterval(self.settings.agentTurnDeadlineSeconds)) - }, - // Auto-approved acts are narrated ALOUD when a voice conversation is live (always visible - // in the thinking stream regardless) — spec: silence never hides an act. - narrator: { [weak self] text in - Task { @MainActor in self?.voiceNarrate(text) } - }) - } - // Per-app keyboard language (opt-in; remembers and re-selects the input source per app/site). Gated // on its OWN toggle, independent of the switcher master enable. The store holds the learned // context-key → source map; the service ties it to the pure policy and the Carbon @@ -610,10 +208,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate /// rebuild time (`StatusItemController.menuNeedsUpdate`), so no live observer is needed. var showDiagnostics: Bool { settings.showDiagnostics } - /// Whether the AI opt-in is on — gates the menu-bar "Speak Last Response" line - /// (`add-voice-computer-use-agent` v0; TTS + AX only, no mic). - var aiCommandsEnabledForMenu: Bool { settings.aiCommandsEnabled } - var onStateChange: (() -> Void)? /// The wizard's menu-bar moment: pulses the real status-item mark (wired by @@ -631,13 +225,10 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate /// window underneath; with both overlays closed it reverts to the `≥3`-fingers rule, leaving /// normal two-finger scroll alone. /// - /// `3+` finger is always consumed (gesture territory — a 4-finger resolve swipe's incidental - /// scroll must not leak to the front app); the normal launcher / switcher still consume 1-2 - /// finger so stray scroll doesn't leak during nav; but while the AI **canvas** is active we - /// DON'T consume 1-2 finger scroll, so it reaches the canvas's SwiftUI ScrollView (the panel is - /// key + interactive and under the cursor) to scroll the thinking / response. - static func shouldConsumeScroll(fingerCount: Int, launcherOpen: Bool, switcherOpen: Bool, canvasActive: Bool) -> Bool { - fingerCount >= 3 || ((launcherOpen || switcherOpen) && !canvasActive) + /// `3+` finger is always consumed (gesture territory); the launcher / switcher consume 1-2 + /// finger so stray scroll doesn't leak during nav. + static func shouldConsumeScroll(fingerCount: Int, launcherOpen: Bool, switcherOpen: Bool) -> Bool { + fingerCount >= 3 || launcherOpen || switcherOpen } /// Read-only tap on the touch stream for the wizard's live-hand act (the recognizer path is @@ -657,23 +248,13 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate // Keep Awake's first-touch-to-stop arming (non-consuming — a no-op unless it's active, and it // never swallows the frame; the recognizer still sees it below). self.keepAwakeController.noteTouch(fingerCount: frame.fingerCount) - // The agent kill switch (`add-voice-computer-use-agent`): ANY human contact aborts an - // in-flight agent act / spoken reply. HOT-PATH RULE (`fix-evict-thrash-and-hot-path`): - // this is the latency-critical gesture pipeline — the check is two stored Bools, armed - // by state-change sinks; the agent/voice stack is NEVER touched (or instantiated) here - // unless one of them is genuinely live. - if frame.fingerCount > 0, self.agentActingNow || self.voicePhaseLive { - if self.agentActingNow { self.aiActionArbiter.humanTouchDetected() } - if self.voicePhaseLive { self.voiceController.humanTouch() } - } self.recognizer.feed(frame) } scrollTap.consumePredicate = { [weak self] in guard let self else { return false } return Self.shouldConsumeScroll(fingerCount: self.currentFingerCount, launcherOpen: self.launcherOverlay.isVisible, - switcherOpen: self.overlay.isVisible, - canvasActive: self.launcherOverlay.canvasActive) + switcherOpen: self.overlay.isVisible) } thumbnails.onThumbnail = { [weak self] id, image in self?.overlay.model.setThumbnail(image, for: id) @@ -685,84 +266,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate guard case let .clipboardEntry(entry) = item.kind else { return } self?.clipboardStore.togglePin(id: entry.id) } - // AI preview canvas: the executor it observes, and the two-stage commit / discard gestures. - launcherOverlay.executor = aiCommandExecutor - // The canvas panel becomes key-interactive only AFTER the executor has read its input — otherwise - // a `.nonactivatingPanel` taking key focus first steals the front app's key focus and the selection - // read comes back empty (falling through to the clipboard). The executor calls this back once the - // input is acquired (or it resolves `.unavailable`, whose Enable/Download controls need the mouse). - aiCommandExecutor.onReadyForInteraction = { [weak self] in self?.launcherOverlay.makeCanvasInteractive() } - // Enable/download wiring for the canvas's `.unavailable` state (fired an AI item while AI is - // off or the model isn't downloaded → the canvas offers Enable/Download + a model picker). - launcherOverlay.aiAvailability = AICanvasAvailability( - settings: settings, - models: modelManager, - onDownload: { [weak self] in self?.downloadAIModel() } - ) - launcherOverlay.onCommitCanvas = { [weak self] in - guard let self else { return } - // A fresh two-finger DOWN swipe commits: route the ready result / reviewed side effect via - // `commit()`. Errors surface in the executor's `.failed` state (the canvas is already - // dismissed by the controller, but the executor records them). - Task { @MainActor in - try? await self.aiCommandExecutor.commit() - } - } - launcherOverlay.onDiscardCanvas = { [weak self] in - // A one-shot fire is ephemeral: a discard cancels the in-flight generation and resets — it - // has no session, no parked row, nothing durable to clean up (notch-native-conversations). - self?.aiCommandExecutor.cancel() - } - // Notch conversation flick grammar (`notch-conversation-gestures`): the recognizer watches - // two-finger flick excursions only while a conversation is expanded. Driven from the controller's - // single choke point so the mode can never be left stuck on after a collapse from any path. - parkController.onExpandedChanged = { [weak self] expanded in - self?.recognizer.notchConversationActive = expanded - } - // Keep Awake flips active/inactive → rebuild the menu bar so its "Active / Stop" fallback tracks. - keepAwakeController.onActiveChanged = { [weak self] in self?.onStateChange?() } - // Screen-region (vision) command: the launcher already dismissed to reveal the desktop. Run the - // interactive region picker; on a drag, capture the designated region and re-open the canvas - // firing the executor with the captured image (the executor maps a permission gap → .failed and an - // unavailable capture → .noInput). A click-without-drag cancels — no canvas, nothing generated; the - // captured front app already retains focus (both the launcher and the picker are non-activating). - launcherOverlay.onScreenRegionCommand = { [weak self] command in - guard let self else { return } - self.regionPicker.show { [weak self] resolution in - guard let self else { return } - switch resolution { - case .cancel: - break // defused — the front app was never deactivated, so there is nothing to restore - case let .region(rect): - Task { @MainActor in - let outcome = await self.selectionService.captureScreenRegion(rect) - self.launcherOverlay.showCanvas(for: command) - self.aiCommandExecutor.fire(command, screenCapture: outcome) - } - } - } - } - // When the canvas opens, put the recognizer in canvas-resolution mode so a FRESH four-finger - // swipe resolves it (horizontal = discard, down = apply) instead of re-opening the launcher. - launcherOverlay.onCanvasStateChanged = { [weak self] active in - self?.recognizer.launcherCanvasResolutionActive = active - } - // When the Files band becomes current, put the recognizer in the sustained Files-drill mode so a - // FRESH contact drills the directory tree (horizontal = depth, vertical = highlight) and a - // resolving lift opens / Open-Withs the highlighted entry — instead of stepping the grid. Mirrors - // the canvas-resolution wiring above. - launcherOverlay.onFilesColumnStateChanged = { [weak self] active in - self?.recognizer.filesDrillActive = active - } - // Persist the per-root remembered deepest location as the Files navigator drills (so the next open - // restores where the user left each root). Keyed/valued by standardized path. - launcherOverlay.model.onFilesRememberLocation = { [weak self] path, rootPath in - self?.settings.rememberLocation(path, forRoot: rootPath) - } - // The Files-band failure row's Retry re-fires the last open through `FileOpenService` (which the - // state sink mirrors back into `model.filesOpenFailure`), so a transient open failure can be retried - // without re-navigating. No-op when nothing was opened yet. - launcherOverlay.model.onFilesRetryOpen = { [weak self] in self?.retryLastFilesOpen() } observeSleepWake() observeEnabledToggle() observeSpacesRearrangeToggle() @@ -771,19 +274,11 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate observeLauncherToggle() observeClipboardToggle() observeDeviceLinkToggle() - observeFileOpenState() - observeAICommandsToggle() observeKeyboardLanguageToggle() observeKeyboardLanguagePerSiteToggle() observeKeyboardLanguageBrowserControlToggle() observeDockPreviewsToggle() observeWindowGroupsToggle() - observeParkToggle() - observeAIGatingSnapshot() - reconcileAIModelAtLaunch() - installAutomaticModelEviction() - observeVoiceToggle() - syncVoicePTTMonitor() } deinit { @@ -849,13 +344,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate // Snap-to-bind window groups: same standalone opt-in shape — when off, the left-mouse // monitors aren't even installed. snapMonitor.setEnabled(settings.enableWindowGroups) - // The notch home zone rail follows the agent feature (here: AI commands enabled). When off, the - // cursor monitor isn't even installed. - parkController.setEnabled(settings.aiCommandsEnabled) - setParkMaintenanceEnabled(settings.aiCommandsEnabled) - // Recover interrupted turns NOW (a quit mid-response was normalized to parked+scheduled at the - // controller's init) instead of waiting for the first coarse maintenance tick. - if settings.aiCommandsEnabled { parkController.runAdvancePass(now: Date()) } refreshRowSwitchingGate() refreshClipboardMonitor() applySpacesRearrangeOnLaunchIfManaged() @@ -866,12 +354,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate // captures don't stall the reel slide mid-animation (SCK stays warm process-wide thereafter — // the reason only the first run stutters). No-op without Screen Recording permission. Task { await thumbnails.warmUp() } - // Skills-as-files (`wire-memory-skills`): load the built-in ∪ user skill corpus off-main once (this - // IS the idempotent catalog→skill-files migration — built-ins are projected in-memory, so there's - // nothing on disk to rewrite), then start the folder watcher so a dropped/edited user `.skill.md` - // re-indexes with no rebuild. The store's `loaded` snapshot backs `manifest(id:)` (the active-skill - // allow-list resolver) and `index()`. AI feature-gated, like the park rail. - if settings.aiCommandsEnabled { startSkillsLoadAndWatch() } // The First Touch wizard IS the first-run flow: it replaced the four one-shot consent // alerts (didPrompt* — set on the wizard's completion so they can never fire) and the // open-Hub-on-Setup fallback. Resume-aware: any interruption (relaunch, re-login, plain @@ -879,19 +361,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate maybeShowFirstTouchWizard() } - /// Load the skill corpus once + start the user-folder watcher (`wire-memory-skills`). Idempotent: a - /// re-call no-ops the watcher (it's started once) and re-loads the snapshot. The built-in projection is - /// already available synchronously to the provider/resolver; this populates the store's `loaded` so - /// user-folder skills (shadowing/extra) participate in `manifest(id:)`/`index()`. - private func startSkillsLoadAndWatch() { - guard aiSkillWatcher == nil else { return } - let store = aiSkillStore - Task { _ = await store.loadAll() } // initial off-main load (the idempotent migration is a no-op) - let watcher = SkillFolderWatcher(store: store, onReload: { _ in }) // store.loadAll() updates `loaded` - aiSkillWatcher = watcher - watcher.start() - } - // MARK: - Enable / disable func enable() { @@ -1467,16 +936,8 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate // (the recognizer only emits these when the relocation is effective, so this is exactly // the post-re-login / replay case). if wizardOwnsGestures { wizardModel?.launcherTourActivate(); return } - // Defensive: while the AI preview canvas is open the recognizer is in canvas-resolution mode and - // routes swipes to `launcherCanvasResolve` (down = commit, horizontal = discard), so it does NOT - // call this. Should it ever reach here, do NOT re-show — that would discard the canvas and reset - // to the grid; let the open canvas keep handling the gesture. - guard !launcherOverlay.canvasActive else { return } let fav = favoritesStore.favorites - // AI commands are persisted band items now (configuration-hub fold-in), so they project from - // `fav.bands` like any item — no synthetic AI band, no opt-in filtering (a fired AI item - // resolves its availability in the canvas). Only the Clipboard band remains synthetic. var bands = fav.bands var clipboardBandIndex: Int? if settings.keepClipboardHistory { @@ -1486,22 +947,9 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate bands.append(ClipboardBandBuilder.build(from: entries)) clipboardBandIndex = bands.count - 1 } - // The Files band is synthetic + ephemeral like Clipboard (never persisted into Favorites): build a - // fresh directory navigator over the configured roots, project its current column as the band, and - // thread both the band index and the controller through `show`. The controller owns the on-demand - // listing cache + the column state machine; the model routes the recognizer's drill into it. - var filesBandIndex: Int? - var filesColumn: FilesColumnController? - if settings.filesBandEnabled { - let controller = makeFilesColumnController() - bands.append(FilesBandBuilder.build(currentColumn: controller.visibleEntries)) - filesBandIndex = bands.count - 1 - filesColumn = controller - } guard !bands.isEmpty else { return } // Capture the app the user was looking at before the (non-activating) overlay appears, so a - // `.action(.closeFrontWindow)` item — a clipboard paste, and an AI command's selection I/O — - // targets that window. + // `.action(.closeFrontWindow)` item — and a clipboard paste — targets that window. let front = NSWorkspace.shared.frontmostApplication capturedFrontApp = (front?.processIdentifier == getpid()) ? capturedFrontApp : front // Edge-triggered auto-repeat acceleration. @@ -1514,37 +962,7 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate startBand: fav.homeBandIndex, startColumn: fav.resolvedHomeColumn, dwell: settings.dwellToArmDuration, - clipboardBandIndex: clipboardBandIndex, - filesBandIndex: filesBandIndex, - filesColumn: filesColumn) - } - - /// Build a fresh `FilesColumnController` for a launcher open: the configured roots, the per-root - /// remembered deepest locations restored from settings, and the sort key/direction mapped from - /// settings. The controller seeds its current column synchronously from the cache (empty on a cold - /// open) and warms the landing column asynchronously; the band's items reproject when the listing - /// lands (via the model's `onColumnChanged` binding). Orphaned remembered locations (a removed root) - /// are pruned opportunistically so the map doesn't grow unbounded. - private func makeFilesColumnController() -> FilesColumnController { - let roots = settings.filesRoots.map { URL(fileURLWithPath: $0).standardizedFileURL } - // Restore root → deepest-folder from settings, keyed by the SAME standardized path the model - // persists with (`root.standardizedFileURL.path`), so the lookup matches what was written. - var remembered: [URL: URL] = [:] - for root in roots { - if let path = settings.rememberedLocation(forRoot: root.path) { - remembered[root] = URL(fileURLWithPath: path).standardizedFileURL - } - } - // Drop orphaned remembered entries whose root is no longer configured (standardized key space). - settings.pruneRememberedLocations(keepingRoots: Set(roots.map(\.path))) - return FilesColumnController( - roots: roots, - remembered: remembered, - sortOrder: FilesColumnController.sortOrder(field: settings.filesSortField), - sortDirection: settings.filesSortDirection, - // Open displaying the last folder visited (so crossing in from the band icon lands there, - // no jump) when the user has the Hub toggle on; otherwise land on the roots list. - restoreLastLocation: settings.filesRememberLocation) + clipboardBandIndex: clipboardBandIndex) } func launcherDidStepItem(_ direction: Int) { @@ -1580,71 +998,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate launcherOverlay.cancel() } - /// The action a canvas resolve excursion resolves to, once the binding is consulted. `commit` is - /// further gated by `canvasAtTop` at the call site (binding-independent); `discard`/`ignore` are not. - enum CanvasResolveDecision: Equatable { case commit, discard, ignore } - - /// Pure decision: given the recognizer's axis-locked excursion (exactly one of `dx`/`dy` non-zero) and - /// the user's `canvas` binding, resolve which action it performs (`add-gesture-previews-and-bindings` - /// §9.3). The recognizer's sign convention is fixed: `dy<0 → swipeDown`, `dy>0 → swipeUp`, - /// `dx<0 → swipeLeft`, `dx>0 → swipeRight`. The rule (reproducing today's defaults and honoring any - /// remap): the excursion bound to commit → commit; to ignore → ignore; to dismiss → discard; the one - /// spare (unbound) excursion → discard if HORIZONTAL ("any horizontal = dismiss"), ignore if VERTICAL. - static func canvasResolveDecision( - dx: Int, dy: Int, binding: GestureBindings.CanvasBinding - ) -> CanvasResolveDecision { - let performed: GestureBindings.CanvasExcursion? - if dy < 0 { performed = .swipeDown } - else if dy > 0 { performed = .swipeUp } - else if dx < 0 { performed = .swipeLeft } - else if dx > 0 { performed = .swipeRight } - else { performed = nil } - guard let performed else { return .ignore } - - if performed == binding.commit { return .commit } - if performed == binding.ignore { return .ignore } - if performed == binding.dismiss { return .discard } - // The one spare (unbound) excursion: a HORIZONTAL spare discards, a VERTICAL spare is ignored. - return (performed == .swipeLeft || performed == .swipeRight) ? .discard : .ignore - } - - /// A fresh TWO-finger swipe while the AI preview canvas is open resolves it (change - /// `positional-navigation`, D5 — 4 fingers open/dismiss the platform, 2 fingers act within it). The - /// recognizer has already axis-locked; the performed excursion is mapped to an action through the - /// user's **configured canvas binding** (`add-gesture-previews-and-bindings` §9.3). Defaults reproduce - /// today's grammar exactly: down = commit-at-top, up = ignore, left = dismiss, spare (right) = discard. - /// A fresh two-finger FLICK while a notch conversation is expanded (`notch-conversation-gestures`): - /// fast UP minimizes it into the notch dock (the standard collapse — persisted, background-scheduled, - /// an in-flight turn untouched); fast RIGHT purge-deletes the session (authoritative discard + the - /// audit-ledger purge — no trace, no log line). Fast DOWN and fast LEFT are reserved no-ops. Soft - /// scrubs never reach here (the D4 classifier emits nothing for them), so thread scrolling is native. - func notchConversationResolve(dx: Int, dy: Int) { - guard let id = parkController.expandedID else { return } - if dy > 0 { - parkController.collapse(closingPanel: true) // fast up → close straight into the notch (no rail dwell) - } else if dx > 0 { - parkController.purge(id) // fast right → gone everywhere, no trace - } - // dy < 0 (fast down) and dx < 0 (fast left): reserved — deliberately nothing. - } - - func launcherCanvasResolve(dx: Int, dy: Int) { - guard launcherOverlay.canvasActive else { return } - switch AppCoordinator.canvasResolveDecision(dx: dx, dy: dy, binding: settings.gestureBindings.canvas) { - case .commit: - // The commit-bound excursion applies — but ONLY when the canvas is scrolled to the TOP. Off the - // top the same two-finger pan is the user SCROLLING the response/thinking back up (the native - // scroll already handled it), so it must not insert the result. This at-top guard is - // binding-independent: it holds for whatever excursion is bound to commit. - guard aiCommandExecutor.canvasAtTop else { return } - launcherOverlay.resolveCanvasCommit() // at top → apply (replace / paste / run task) - case .discard: - launcherOverlay.discardCanvas() - case .ignore: - break // no-op (e.g. default up scrolls toward the tail) - } - } - func launcherEdgeChanged(dx: Int, dy: Int) { guard !wizardOwnsGestures else { return } // no edge auto-repeat in the wizard's tour guard launcherOverlay.isVisible else { return } @@ -1654,447 +1007,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate launcherOverlay.setEdgeAutoScroll(dx: h, dy: v) } - // MARK: - GestureRecognizerDelegate (Files-band drill) - - /// Horizontal drill while the Files band is current: descend / ascend the directory tree (the - /// direction is already reverse-adjusted in the recognizer). Forwarded to the overlay, which routes it - /// into the navigator and reprojects the band, then **recharges the dwell-to-arm** on the new row - /// (`filesManageDwell` — add-files-band-dwell-arm): a Files lift fires only when the row has armed. - /// - /// While the Open-With picker is open it is **vertical-only** (the apps are a single scrubbable column), - /// so a depth step is ignored — horizontal must not drill folders out from under the open popup. - func filesDepth(_ direction: Int) { - guard launcherOverlay.isVisible else { return } - // A popup (the Open-With grid or the action menu) is vertical-only — depth doesn't drill the tree. - if launcherOverlay.model.filesPicker != nil || launcherOverlay.model.filesActionMenu != nil { return } - launcherOverlay.filesDepth(direction) - launcherOverlay.filesManageDwell() // descend/ascend lands on a new row → recharge the dwell-to-arm - } - - /// Vertical highlight move while the Files band is current (reverse-adjusted upstream): an up-step at - /// the top of the column overflows into a focus-search request inside the model. - /// - /// While the Open-With picker is open the same vertical scrub moves the **picker** highlight (the app - /// list), not the folder list — the popup is what the user is navigating. - func filesHighlight(_ direction: Int) { - guard launcherOverlay.isVisible else { return } - // Route the vertical scrub to whichever popup is open (action menu, then Open-With grid), else the - // folder list — a popup is what the user is navigating while it is up. - if launcherOverlay.model.filesActionMenu != nil { - launcherOverlay.model.filesActionMenuMove(direction) - } else if launcherOverlay.model.filesPicker != nil { - launcherOverlay.model.filesPickerMove(direction) - } else { - launcherOverlay.filesHighlight(direction) - } - launcherOverlay.filesManageDwell() // recharge the dwell-to-arm on the row/cell we landed on - } - - /// The resolving lift with no added finger. Two cases: - /// - /// - **Picker open:** a lift CHOOSES the highlighted app — Open-With the highlighted file using that - /// app's URL (the same defusable held open), then leave the picker and dismiss the navigator. - /// - **Picker closed:** open the highlighted entry in its default app (a folder as a Finder window), - /// then dismiss. - /// - /// The open is a **defusable held open** under the hood (design D7): prepared then committed after a - /// short fuse, so a discard within the fuse window still cancels it. A failure surfaces ONLY through - /// `FileOpenService`'s bounded `.failed` state (its clean `FileActionError` headline) — never an alert - /// from here. A no-op (dismiss only) when nothing is highlighted (empty column / empty picker). - func filesOpen() { - guard launcherOverlay.isVisible else { return } - // DWELL GATE (mirrors the launcher's `end()`): a committing Files lift fires only when the highlighted - // row has armed (rested past the dwell); an unarmed scrub-and-lift just DISMISSES, acting on nothing. - // One gate covers every committing branch below — picker app, menu row, and the default deliver/open. - guard launcherOverlay.model.armed else { launcherOverlay.hide(); return } - // Picker open: the lift chooses the highlighted app and Open-Withs the file with it. - if launcherOverlay.model.filesPicker != nil { - defer { filesPickerOriginEntry = nil; launcherOverlay.model.exitFilesPicker(); launcherOverlay.hide() } - guard let entry = launcherOverlay.filesHighlightedEntry, - case let .external(candidate)? = launcherOverlay.model.filesPickerSelected() else { return } - fireFilesOpen { [weak self] in - self?.fileOpenService.prepareOpenWith(entry, appURL: candidate.app.url) - .commit(afterFuse: Self.filesOpenFuse) - } - return - } - // Action menu open: a lift COMMITS the highlighted row ("Open in ▸" descends into the app grid; a - // tool row opens the folder in that terminal/editor; any other action runs its effect). - if let menu = launcherOverlay.model.filesActionMenu { - filesCommitMenuRow(menu) - return - } - // Picker closed: perform the configured lift action on the highlighted entry — DELIVER it to the - // captured front app (the default — `files-contextual-delivery`) or OPEN it (file → default app, - // folder → Finder window). Open dismisses immediately (the defusable held open fires after a fuse); - // deliver keeps the navigator up until the async paste lands, so a no-front-app failure surfaces as - // a bounded row (mirroring `surfaceNoApplication` — `hide()` destroys the panel synchronously, so a - // failure can only show while the navigator is still open), and hides on success. - guard let entry = launcherOverlay.filesHighlightedEntry else { launcherOverlay.hide(); return } - switch settings.filesLiftAction { - case .open: - launcherOverlay.hide() - fireFilesOpen { [weak self] in - self?.fileOpenService.prepareOpen(entry).commit(afterFuse: Self.filesOpenFuse) - } - case .deliver: - filesDeliver(entry) - } - } - - /// Deliver the highlighted entry to the captured front app (the default lift — `files-contextual-delivery`): - /// write the dual-representation payload (path string + file reference) and synthesize a paste, so a text - /// target receives the **path** and a Finder window receives the **file** — no context detection on our - /// side. On success the navigator dismisses; when there is **no captured front app** to deliver into, the - /// delivery surfaces a **bounded, non-blocking** failure row (never a false "Done", never an alert) and the - /// navigator stays open with the drill re-armed so the user can retry or discard. The keystroke landing - /// itself is unobservable, so a successful attempt is "delivered," not a confirmed paste. - private func filesDeliver(_ entry: FileEntry) { - let payload = FilesDelivery.payload(for: entry) - let deliver: () -> Void = { [weak self] in - guard let self else { return } - Task { @MainActor in - if await self.selectionService.deliverFile(url: payload.url, path: payload.path) { - self.launcherOverlay.hide() - } else { - self.launcherOverlay.model.filesOpenFailure = LauncherModel.FilesOpenFailure( - headline: "Couldn't deliver — no app was frontmost to receive it.", details: nil) - self.recognizer.rearmDrill() // keep navigation alive for another try / discard - self.launcherOverlay.filesRearmDwell() // a re-lift must re-dwell first (Retry bypasses) - } - } - } - lastFilesOpen = deliver // the failure row's Retry re-runs the identical delivery - deliver() - } - - /// Capture `open` as the retryable last Files open (so the failure row's Retry re-runs the identical - /// prepare→commit), then fire it. The single fire path for both default opens and Open-With, so a retry - /// reproduces exactly what the lift did. - private func fireFilesOpen(_ open: @escaping () -> Void) { - lastFilesOpen = open - open() - } - - /// The resolving lift after a relative +1 finger: open the **action menu** for the highlighted file or - /// folder (`files-action-menu`). Builds the per-type rows (the configured menu + the live pasteboard / - /// installed-tools context) and enters the navigable menu (the user scrubs vertically and lifts to commit - /// — see `filesOpen`/`filesCommitMenuRow`), then **re-arms** the drill so a fresh gesture scrubs the popup - /// (the firing lift already raised the fingers). A no-op when a popup is already open (a stray +1 inside - /// it must not reset it). The method name is retained because it is the recognizer's `+1`-finger delegate - /// hook; its action is now "open the menu" (Open-With folds in as the menu's "Open in ▸"). - func filesOpenWith() { - guard launcherOverlay.isVisible else { return } - // Already in a popup: a stray +1 must not reset it — re-arm so it stays scrubbable (the lift latched - // the drill as resolved). The dwell is NOT recharged here (the same popup row stays highlighted). - guard launcherOverlay.model.filesPicker == nil, launcherOverlay.model.filesActionMenu == nil else { - recognizer.rearmDrill(); return - } - // DWELL GATE: the menu opens only over an armed row — a quick scrub-and-`+1`-lift dismisses, never - // popping a menu you didn't dwell on (the `+1`-finger morph itself moved no highlight, so the arm the - // user charged on this row is the same arm gating it here). - guard launcherOverlay.model.armed else { launcherOverlay.hide(); return } - defer { recognizer.rearmDrill() } - guard let entry = launcherOverlay.filesHighlightedEntry else { return } - let rows = buildActionMenuRows(for: entry) - guard !rows.isEmpty else { return } // defensive — the default menus always have at least one row - launcherOverlay.model.enterFilesActionMenu(entry: entry, rows: rows) - launcherOverlay.filesManageDwell() // entering the menu lands on row 0 → a fresh dwell there - } - - /// Resolve the configured action menu into the concrete rows for `entry`, applying live context: whether - /// the pasteboard holds a file (gates Paste-into) and the installed, user-curated terminals/editors. - private func buildActionMenuRows(for entry: FileEntry) -> [FilesMenuRow] { - let hasFile = NSPasteboard.general.canReadObject(forClasses: [NSURL.self], - options: [.urlReadingFileURLsOnly: true]) - let tools = detectFilesTools() - return settings.filesActionMenu.visibleRows(for: entry, pasteboardHasFile: hasFile, - terminals: tools.terminals, editors: tools.editors) - } - - /// Probe which catalog terminals/editors are installed (`NSWorkspace.urlForApplication(withBundleIdentifier:)`) - /// and apply the user's curation (`filesToolsDisabled`). No new permission — a bundle-id lookup. - private func detectFilesTools() -> (terminals: [FilesTool], editors: [FilesTool]) { - let disabled = Set(settings.filesToolsDisabled) - func detect(_ seeds: [(bundleID: String, name: String)], role: FilesTool.Role) -> [FilesTool] { - seeds.compactMap { seed in - guard NSWorkspace.shared.urlForApplication(withBundleIdentifier: seed.bundleID) != nil else { return nil } - return FilesTool(bundleID: seed.bundleID, name: seed.name, role: role, - enabled: !disabled.contains(seed.bundleID)) - } - } - return (detect(FilesToolCatalog.terminals, role: .terminal), - detect(FilesToolCatalog.editors, role: .editor)) - } - - /// Commit the highlighted action-menu row. "Open in ▸" descends into the Open-With app grid (remembering - /// it came from the menu, so a discard backs out to the menu); a tool row opens the folder in that - /// terminal/editor; any other action runs its effect (which dismisses, or surfaces a bounded failure). - private func filesCommitMenuRow(_ menu: LauncherModel.FilesActionMenuState) { - guard let row = menu.highlighted else { - launcherOverlay.model.exitFilesActionMenu(); recognizer.rearmDrill(); return - } - let entry = menu.entry - switch row { - case .action(.openIn): - launcherOverlay.model.exitFilesActionMenu() - filesPickerOriginEntry = entry - presentOpenWithPicker(for: entry) - case let .tool(_, tool): - openEntry(entry, inToolBundleID: tool.bundleID) - launcherOverlay.hide() - case let .action(action): - performMenuAction(action, on: entry) - } - } - - /// Present the Open-With **app grid** for `entry`: for a **file**, the apps that can open it (default - /// indicated); for a **folder**, the folder-openers (Finder + the curated editors/terminals). When the - /// candidate list is empty, surface the bounded `noApplicationForFile` notice and keep the navigator - /// open. Always re-arms the drill so a fresh gesture scrubs the grid. - private func presentOpenWithPicker(for entry: FileEntry) { - // Entering the grid lands on the default app (or, on empty candidates, drops back to the folder row) — - // recharge the dwell so the landing cell must itself be dwelled before a lift opens it. - defer { recognizer.rearmDrill(); launcherOverlay.filesManageDwell() } - let candidates = entry.isDirectory - ? folderOpenerCandidates() - : fileOpenService.openWithCandidates(for: entry) - let entries = OpenWithEntries.build(externalApps: candidates) - guard !entries.isEmpty else { - fileOpenService.surfaceNoApplication(for: entry) - return - } - launcherOverlay.model.enterFilesPicker(entries) - } - - /// The "Open in ▸" candidates for a **folder**: Finder (the default), then the installed, enabled - /// editors and terminals — a folder has no LaunchServices opener list of its own (which is exactly why - /// the menu was empty on folders before). Each opens the folder with that app via `prepareOpenWith`. - private func folderOpenerCandidates() -> [OpenWithCandidate] { - var candidates: [OpenWithCandidate] = [] - if let finder = NSWorkspace.shared.urlForApplication(withBundleIdentifier: "com.apple.finder") { - candidates.append(OpenWithCandidate(app: AppCandidate(url: finder), isDefault: true)) - } - let tools = detectFilesTools() - for tool in (tools.editors + tools.terminals) where tool.enabled { - if let url = NSWorkspace.shared.urlForApplication(withBundleIdentifier: tool.bundleID) { - candidates.append(OpenWithCandidate(app: AppCandidate(url: url), isDefault: false)) - } - } - return candidates - } - - /// Open the entry's folder (a folder itself, or a file's containing folder) as `bundleID`'s working - /// directory — the ‹terminals› / Open-in-‹editor› rows. Reuses the no-new-permission `NSWorkspace.open` - /// handoff; most terminals/editors set a folder argument as their CWD. - private func openEntry(_ entry: FileEntry, inToolBundleID bundleID: String) { - let folder = entry.isDirectory ? entry.url : entry.url.deletingLastPathComponent() - guard let appURL = NSWorkspace.shared.urlForApplication(withBundleIdentifier: bundleID) else { - surfaceFilesFailure(.openFailed(name: entry.name, details: "That app isn’t installed.")) - return - } - let config = NSWorkspace.OpenConfiguration() - config.activates = true // the user chose this tool — bring it forward - NSWorkspace.shared.open([folder], withApplicationAt: appURL, configuration: config) { [weak self] _, error in - guard let error else { return } - Task { @MainActor in - self?.surfaceFilesFailure(.openFailed(name: entry.name, details: String(describing: error))) - } - } - } - - /// Run a non-navigation menu action and resolve the navigator. The copy/reveal/name/favorite actions - /// complete the interaction (dismiss); Paste-into keeps the navigator open on failure (a bounded row + - /// retry), and the copies land in clipboard history via the live monitor (no manual insert). - private func performMenuAction(_ action: FilesMenuAction, on entry: FileEntry) { - let pb = NSPasteboard.general - switch action { - case .copyAsPath: - pendingCut = nil // an explicit copy supersedes any pending cut - pb.clearContents() - pb.setString(entry.url.standardizedFileURL.path, forType: .string) - launcherOverlay.hide() - case .copy: - pendingCut = nil - pb.clearContents() - pb.writeObjects([entry.url as NSURL]) // the file/folder OBJECT (paste-in-Finder copies it) - launcherOverlay.hide() - case .cut: - // Mark for move: write the object like Copy, then record the cut keyed on the pasteboard's NEW - // change-count, so the next Paste moves it only while the pasteboard is still this cut. - pb.clearContents() - pb.writeObjects([entry.url as NSURL]) - pendingCut = (sources: [entry.url], changeCount: pb.changeCount) - launcherOverlay.hide() - case .copyName: - pendingCut = nil - pb.clearContents() - pb.setString(entry.name, forType: .string) - launcherOverlay.hide() - case .revealInFinder: - NSWorkspace.shared.activateFileViewerSelecting([entry.url]) - launcherOverlay.hide() - case .addToFavorites: - addEntryToFavorites(entry) - launcherOverlay.hide() - case .pasteInto: - pasteIntoFolder(for: entry) - case .delete: - deleteEntry(entry) - case .openIn, .openInTerminals, .openInEditor: - // openIn descends (handled in `filesCommitMenuRow`); the tool groups expand to `.tool` rows, so - // a bare `.action` here is an unreachable fallthrough — dismiss to be safe. - launcherOverlay.hide() - } - } - - /// Paste-into: put the pasteboard's file(s) INTO the target folder (a highlighted folder, or a file's - /// containing folder) — **dual-mode**: a **move** when fulfilling a pending Cut (the pasteboard is still - /// that cut), else a **copy**; **keep-both** on conflict (auto-rename) either way, never overwriting. - /// Dismisses on success; on failure surfaces a bounded `pasteFailed` row and keeps the navigator open - /// (re-armed). File URLs only in v1. - private func pasteIntoFolder(for entry: FileEntry) { - // Capture the paste as the retryable last action, so the failure row's Retry re-pastes (not a stale - // open). Keep-both makes a re-paste safe (it never overwrites). - let paste: () -> Void = { [weak self] in self?.performPasteInto(for: entry) } - lastFilesOpen = paste - paste() - } - - private func performPasteInto(for entry: FileEntry) { - let destination = entry.isDirectory ? entry.url : entry.url.deletingLastPathComponent() - let sources = (NSPasteboard.general.readObjects(forClasses: [NSURL.self], - options: [.urlReadingFileURLsOnly: true]) as? [URL]) ?? [] - guard !sources.isEmpty else { - surfaceFilesFailure(.pasteFailed(name: destination.lastPathComponent, - details: "The clipboard holds no file to paste.")) - recognizer.rearmDrill(); return - } - // MOVE iff the live pasteboard is still the pending cut (its change-count is unchanged); otherwise - // COPY. Any Copy / external write since the Cut bumps the count → the cut is superseded → we copy. - let isMove = pendingCut.map { $0.changeCount == NSPasteboard.general.changeCount } ?? false - let fm = FileManager.default - do { - var taken = Set((try? fm.contentsOfDirectory(atPath: destination.path)) ?? []) - for source in sources { - let unique = FilesPasteName.uniqueName(for: source.lastPathComponent, existing: taken) - let target = destination.appendingPathComponent(unique) - if isMove { try fm.moveItem(at: source, to: target) } - else { try fm.copyItem(at: source, to: target) } - taken.insert(unique) - } - if isMove { pendingCut = nil } // the cut is consumed by the move - launcherOverlay.hide() - } catch { - surfaceFilesFailure(.pasteFailed(name: destination.lastPathComponent, details: String(describing: error))) - recognizer.rearmDrill() - } - } - - /// Delete: move the entry to the **Trash** (recoverable from Finder) — never a permanent `removeItem`. - /// Dismisses on success (the entry is simply gone on the next listing); on failure surfaces a bounded - /// `trashFailed` row and keeps the navigator open (re-armed). Committed by the dwell-armed lift, so a - /// stray scrub-and-lift never deletes (`add-files-band-dwell-arm`). - private func deleteEntry(_ entry: FileEntry) { - let delete: () -> Void = { [weak self] in self?.performDelete(for: entry) } - lastFilesOpen = delete // the failure row's Retry re-tries the trash - delete() - } - - private func performDelete(for entry: FileEntry) { - do { - try FileManager.default.trashItem(at: entry.url, resultingItemURL: nil) - launcherOverlay.hide() - } catch { - surfaceFilesFailure(.trashFailed(name: entry.name, details: String(describing: error))) - recognizer.rearmDrill() - } - } - - /// Add `entry` to the launcher as a persistent favorite (`.path` opens it in its default handler) — the - /// opt-in "Add to Favorites" item, bridging the Files band back into the launcher. Lands in the home - /// band, else the first band, else a fresh "Files" band. - private func addEntryToFavorites(_ entry: FileEntry) { - let item = LaunchItem(title: entry.name, icon: .fileIcon, kind: .path(entry.url)) - let fav = favoritesStore.favorites - let bandID = fav.homeBandID ?? fav.bands.first?.id ?? favoritesStore.addBand(name: "Files") - favoritesStore.addItem(item, toBand: bandID) - } - - /// Surface a Files-band action failure as the existing bounded, non-blocking row (clean headline + opt-in - /// copyable details) — never an alert, never raw text in a headline. - private func surfaceFilesFailure(_ error: FileActionError) { - launcherOverlay.model.filesOpenFailure = LauncherModel.FilesOpenFailure( - headline: error.errorDescription ?? "Something went wrong.", details: error.copyableDetails) - } - - /// A fresh deliberate four-finger horizontal swipe-away while drilled: discard. - /// - /// - **Picker open:** the swipe backs OUT of the picker to the folder list — the navigator stays open - /// (it is not a full dismiss), and any pending open is defused. - /// - **Picker closed:** dismiss the navigator. Defuses any held open (it never terminates an - /// already-running app — `cancelPending` only cancels a not-yet-fired open). - func filesDiscard() { - guard launcherOverlay.isVisible else { return } - fileOpenService.cancelPending() - // Action menu open: back out to the folder list (navigator stays). - if launcherOverlay.model.filesActionMenu != nil { - launcherOverlay.model.exitFilesActionMenu() - recognizer.rearmDrill() - launcherOverlay.filesManageDwell() // landed back on the folder row → recharge its dwell - return - } - if launcherOverlay.model.filesPicker != nil { - launcherOverlay.model.exitFilesPicker() - // If the grid was reached via the action menu's "Open in ▸", back out ONE level — re-open the - // menu — rather than dropping straight to the folder list. - if let origin = filesPickerOriginEntry { - filesPickerOriginEntry = nil - let rows = buildActionMenuRows(for: origin) - if !rows.isEmpty { launcherOverlay.model.enterFilesActionMenu(entry: origin, rows: rows) } - } - recognizer.rearmDrill() // a fresh gesture resumes navigation - launcherOverlay.filesManageDwell() // landed back on the menu / folder row → recharge - return - } - launcherOverlay.hide() - } - - /// Short pre-launch fuse on a committed Files open, so a discard issued within the window still defuses - /// it (design D7). Tiny — the held → swipe-to-resolve UI (a longer visible hold) is the view stage; here - /// the fuse just preserves the defuse seam. - private static let filesOpenFuse: Duration = .milliseconds(120) - - /// Mirror `FileOpenService`'s observable state into `model.filesOpenFailure`, so a failed open surfaces - /// as a **bounded, non-blocking** row in the navigator (never an app-modal alert — spec: bounded + - /// non-blocking, never silent). On `.failed` it sets the clean headline + opt-in details; on - /// `.idle`/`.opening`/`.opened` it clears the row to nil (a fresh / in-flight / succeeded open has no - /// failure to show). Mirrors `observeClipboardToggle`'s emitted-value sink pattern (the `@Published` - /// `willSet` reports the new value); no `dropFirst()` — the service starts `.idle`, which clears anyway. - private func observeFileOpenState() { - fileOpenService.$state - .sink { [weak self] state in - MainActor.assumeIsolated { - guard let self else { return } - switch state { - case let .failed(headline, details): - self.launcherOverlay.model.filesOpenFailure = - LauncherModel.FilesOpenFailure(headline: headline, details: details) - case .idle, .opening, .opened: - self.launcherOverlay.model.filesOpenFailure = nil - } - } - } - .store(in: &cancellables) - } - - /// Re-fire the last Files open (the failure row's Retry). Re-running it transitions `FileOpenService` - /// back through `.opening` (which the state sink clears the failure row on) and then to `.opened` or a - /// fresh `.failed` — so a transient failure can be retried in place. A no-op when nothing has been opened. - private func retryLastFilesOpen() { - lastFilesOpen?() - } - // MARK: - Clipboard history monitor lifecycle /// React to the "Keep clipboard history" toggle: start/stop the recorder. The pause toggle just @@ -2136,71 +1048,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate .store(in: &cancellables) } - /// The notch home zone rail follows the agent feature (here: `aiCommandsEnabled`) — install/remove the - /// cursor-reveal monitor off the toggle (mirrors `observeDockPreviewsToggle`). - private func observeParkToggle() { - settings.$aiCommandsEnabled - .dropFirst() - .sink { [weak self] on in - MainActor.assumeIsolated { - self?.parkController.setEnabled(on) - self?.setParkMaintenanceEnabled(on) - } - } - .store(in: &cancellables) - // Live-update the notch reveal dwell as the Hub slider moves. - settings.$agentNotchRevealDwell - .dropFirst() - .sink { [weak self] dwell in - MainActor.assumeIsolated { self?.parkController.setRevealDwell(dwell) } - } - .store(in: &cancellables) - } - - /// Keep the off-main `aiGatingSnapshot` in step with the live Full Potential gating flags + cloud-video - /// budget, so the route loop's `@Sendable` gating closures read CURRENT values (not a stale build-time - /// snapshot) WITHOUT trapping off the main actor. Seeds once now, then refreshes on every relevant - /// `@Published` change (`@Published` fires in `willSet`, so we re-read the assembled gate AFTER the - /// change lands by hopping onto the main run loop's next tick via `receive(on:)`). The flags feeding the - /// gate: the ai-commands opt-in + the master + the media/cloud sub-flags + the cloud-video budget. - private func observeAIGatingSnapshot() { - aiGatingSnapshot.refresh(from: settings) // seed before the first AI command can route a tool - // Each `@Published` emits its current value on subscribe; those initial emissions are harmless - // (they re-seed the same values just set above — `refresh` is idempotent). Every later emission is a - // real flag edit; `receive(on:)` defers the re-read to the next main-run-loop tick so the new value - // (set in `willSet`) has landed before we re-assemble the gate. - Publishers.MergeMany( - settings.$aiCommandsEnabled.map { _ in () }.eraseToAnyPublisher(), - settings.$fullPotentialEnabled.map { _ in () }.eraseToAnyPublisher(), - settings.$mediaGenEnabled.map { _ in () }.eraseToAnyPublisher(), - settings.$fleetCloudEscalationEnabled.map { _ in () }.eraseToAnyPublisher(), - settings.$mediaVideoBudgetPerDay.map { _ in () }.eraseToAnyPublisher() - ) - .receive(on: RunLoop.main) - .sink { [weak self] in - guard let self else { return } - self.aiGatingSnapshot.refresh(from: self.settings) - } - .store(in: &cancellables) - } - - /// Install / tear down the coarse park MAINTENANCE timer alongside the park rail. Each tick runs the - /// (opt-in) auto-dismiss pass AND the background-driver advance pass (serving runnable sessions — - /// recovered turns, scheduled retries — one at a time); idempotent (invalidates any prior timer - /// first) and safe to call with `false` repeatedly. - private func setParkMaintenanceEnabled(_ on: Bool) { - parkAutoDismissTimer?.invalidate() - parkAutoDismissTimer = nil - guard on else { return } - parkAutoDismissTimer = Timer.scheduledTimer(withTimeInterval: Self.parkAutoDismissInterval, - repeats: true) { [weak self] _ in - MainActor.assumeIsolated { - _ = self?.parkController.runAutoDismissPass(now: Date()) - _ = self?.parkController.runAdvancePass(now: Date()) - } - } - } - // MARK: - Device link lifecycle /// React to the `enableDeviceLink` toggle (mirrors `observeClipboardToggle` — uses the emitted value, @@ -2298,300 +1145,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate setClipboardRecording(settings.keepClipboardHistory) } - // MARK: - AI commands opt-in lifecycle - - /// React to the "Enable AI commands" toggle: drive the model manager's opt-in. Turning it OFF - /// evicts any resident model and forgets download progress (privacy + frees weights — handled in - /// `ModelManager`); turning it ON only allows a download (it never auto-fetches — the user starts - /// it from Settings). Uses the EMITTED value, not a re-read (the `@Published` willSet would still - /// report the OLD value here — same reason the gesture toggles pass `enabled` through). - private func observeAICommandsToggle() { - settings.$aiCommandsEnabled - .dropFirst() - .sink { [weak self] on in - MainActor.assumeIsolated { - guard let self else { return } - self.modelManager.setOptedIn(on) - // Re-enabling rediscovers an already-downloaded model (→ .ready) so the user isn't - // asked to "Download" again; the heavy load stays lazy (first command). Then settle - // the displayed status to the SELECTED model (reconcile probes only the default). - if on { - self.modelManager.reconcileWithDisk() - if let d = self.selectedAIModelDescriptor() { self.modelManager.showStatus(for: d) } - } - } - } - .store(in: &cancellables) - } - - /// Wire the automatic model-weight eviction triggers (`model-idle-ttl-and-memory-pressure`): the - /// REAL memory-pressure observer (the previously-aspirational "evict on memory pressure" comment, - /// now true) + the quiescence-keyed idle TTL. The quiescence snapshot pulls from the park - /// controller on the main actor at decision time; the TTL closure reads the live setting so a Hub - /// change applies without re-wiring. Installed unconditionally — `evaluateAutomaticEviction` - /// itself no-ops while not opted in / nothing resident. An open VOICE conversation joins through - /// `foregroundSessionActive` (the OR-shaped flag) so a live dialogue never pays a mid-chat evict. - private func installAutomaticModelEviction() { - modelManager.installAutomaticEviction( - pressure: SystemMemoryPressureSource(), - quiescence: { [weak self] in - guard let self else { return QuiescenceSnapshot() } - // EVERY conversational surface joins the snapshot (`fix-evict-thrash-and-hot-path` - // — the original only saw notch sessions, so canvas conversations were invisible - // and eviction fired between canvas turns → the reload storm): - var snapshot = self.parkController.quiescenceSnapshot() - // The launcher-canvas executor: a loading/streaming turn blocks ALL eviction; an - // open canvas (incl. a paused action review) is a foreground conversation. - if self.aiCommandExecutor.state.isTurnInFlight { - snapshot.turnInFlight = true - } - if self.launcherOverlay.canvasActive { - snapshot.foregroundSessionActive = true - } - // A live voice conversation (checked WITHOUT instantiating the lazy voice stack). - if self.voicePhaseLive || (self.settings.voiceConversationEnabled - && self.voiceStackLive - && self.voiceController.isConversationActive) { - snapshot.foregroundSessionActive = true - } - return snapshot - }, - idleTTL: { [weak self] in TimeInterval((self?.settings.aiIdleEvictMinutes ?? 0) * 60) }) - } - - // MARK: - Voice + computer-use behaviors (`add-voice-computer-use-agent`) - - /// Resolve a computer-use window target against the switcher's OWN enumeration (fuzzy app-name - /// contains + optional title hint; nil app = the frontmost app's window). - private func resolveComputerUseWindow(app: String?, title: String?) -> ComputerUseWindowTarget? { - let windows = windowService.snapshot() - let appQuery = app?.lowercased().trimmingCharacters(in: .whitespaces) ?? "" - let titleQuery = title?.lowercased() ?? "" - var candidates = windows - if !appQuery.isEmpty { - candidates = candidates.filter { $0.appName.lowercased().contains(appQuery) } - } else if let front = NSWorkspace.shared.frontmostApplication { - let own = candidates.filter { $0.pid == front.processIdentifier } - if !own.isEmpty { candidates = own } - } - if !titleQuery.isEmpty { - let titled = candidates.filter { $0.title.lowercased().contains(titleQuery) } - if !titled.isEmpty { candidates = titled } - } - // Prefer the current Space (the cheap raise path; reading works regardless). - let pick = candidates.first(where: { $0.isOnCurrentSpace }) ?? candidates.first - return pick.map { ComputerUseWindowTarget(pid: $0.pid, title: $0.title, appName: $0.appName) } - } - - /// Focus a resolved target through the EXISTING hardened commit path (`raiseCommitted` — the - /// agent is the third caller after trackpad and ⌘-Tab; switcher-as-API). - private func focusComputerUseWindow(_ target: ComputerUseWindowTarget) -> Bool { - guard let window = windowService.snapshot().first(where: { - $0.pid == target.pid && ($0.title == target.title || target.title.isEmpty) - }) ?? windowService.snapshot().first(where: { $0.pid == target.pid }) else { return false } - raiseCommitted(window) - return true - } - - /// The one voice turn: run the SAME bounded agent loop over the voice conversation, streaming - /// `.response` tokens back for sentence-chunked speech. Failures speak a clean headline and the - /// stream throws so the turn model returns to idle. - private func voiceTurnStream(_ text: String) -> AsyncThrowingStream { - AsyncThrowingStream { continuation in - let task = Task { @MainActor [weak self] in - guard let self else { continuation.finish(); return } - self.applySpokenAutoIntent(text) - self.voiceConversation.messages.append(AgentMessage(role: .user, text: text)) - do { - let runtime = try await self.modelManager.runtime(requiring: [.text]) - let speakLine: @Sendable (String) -> Void = { [weak self] line in - Task { @MainActor in self?.voiceSpeak(line) } - } - // Voice approvals (design D7): no canvas → guidance-and-skip base gate; the - // auto-mode grant (spoken explicitly) lifts `.confirm` acts, narrated. - let gate = AutoApprovingGate(base: SpokenGuidanceGate(speak: speakLine), - isGranted: { [grant = self.voiceAutoGrant] in grant.value }, - narrate: speakLine) - let budget = LoopBudget( - stepTimeout: TimeInterval(self.settings.agentStepTimeoutSeconds), - turnDeadline: TimeInterval(self.settings.agentTurnDeadlineSeconds)) - let loop = AgentLoop(runtime: runtime, registry: self.aiToolRegistry, - candidateSource: self.aiToolCandidateSource, - gate: gate, - reasoning: false, // voice favors latency; reasoning is a chat affair - source: TaskSource(), - budget: budget, - isAutoGranted: { [grant = self.voiceAutoGrant] in grant.value }, - onResponseToken: { token in - continuation.yield(Token(token, isFinal: false)) - }) - let result = await loop.run( - context: RouteContext(messages: self.voiceConversation.messages)) - switch result.outcome { - case let .answered(answer), let .capReached(answer): - if !answer.isEmpty { - self.voiceConversation.messages.append(AgentMessage(role: .assistant, text: answer)) - } - continuation.finish() - case let .stopped(_, answer): - if !answer.isEmpty { - self.voiceConversation.messages.append(AgentMessage(role: .assistant, text: answer)) - } - continuation.finish() - case .pausedAwaitingUser: - continuation.finish() - case let .failed(headline): - self.voiceSpeak(headline) - continuation.finish(throwing: RuntimeError.unavailable(reason: headline)) - } - } catch { - let presented = AIError.message(for: error) - self.voiceSpeak(presented.headline) - continuation.finish(throwing: error) - } - } - continuation.onTermination = { _ in task.cancel() } - } - } - - /// The USER's explicit spoken auto-mode grant/revoke: the gate protects against MODEL-initiated - /// acts; a deliberate, push-to-talk-held spoken grant IS user consent (design D7 — the - /// initial-command intent path). - private func applySpokenAutoIntent(_ transcript: String) { - let lowered = transcript.lowercased() - let grants = ["enable auto mode", "auto mode on", "without asking", "don't ask", "dont ask"] - let revokes = ["disable auto mode", "auto mode off", "stop auto mode", "ask me again"] - if revokes.contains(where: lowered.contains) { - voiceAutoGrant.value = false - voiceSpeak("Auto mode off.") - } else if grants.contains(where: lowered.contains) { - voiceAutoGrant.value = true - voiceSpeak("Auto mode on for this conversation.") - } - } - - /// Speak a line through the shared synthesizer (the `speak` tool + failure lines + narration). - private func voiceSpeak(_ text: String) { - guard settings.voiceConversationEnabled else { return } - aiSpeechSynthesizer.speak(text) - } - - /// Narrate an auto-executed act: SPOKEN when a voice conversation is live; the visible step list - /// carries it regardless (the loop's thinking stream — spec: silence never hides an act). - private func voiceNarrate(_ text: String) { - guard settings.voiceConversationEnabled, voiceController.isConversationActive else { return } - aiSpeechSynthesizer.speak(text) - } - - /// The `set_auto_mode` tools land here: an expanded/live engine's conversation takes the grant; - /// otherwise the voice conversation does. - private func setConversationAutoMode(_ on: Bool) { - if let engine = parkController.expandedEngine() { - engine.setAutoApprove(on) - } else { - voiceAutoGrant.value = on - } - } - - /// The any-human-touch kill switch (spec: "The human always wins the input"): cancel every - /// in-flight turn as a DISCARD and acknowledge. - private func abortAgentAction() { - parkController.discardInFlightTurns() - voiceController.humanTouch() - } - - /// Speak-last-response (the v0 wedge — no mic, no conversation): resolve the frontmost (or - /// terminal-looking) window, read it via AX, extract the tail, speak it. Uses the model for the - /// extraction when it's resident; falls back to the last visible lines so the command ALWAYS - /// works. Failures surface as one spoken line + a log — bounded, never a modal. - func speakLastResponse() { - Task { @MainActor in - do { - guard let target = resolveComputerUseWindow(app: nil, title: nil) else { - aiSpeechSynthesizer.speak("I can't find a window to read.") - return - } - let snapshot = try await aiAXPerformer.snapshot(pid: target.pid, titleHint: target.title) - let text = snapshot.joinedText - guard !text.isEmpty else { - aiSpeechSynthesizer.speak("That window has no readable text.") - return - } - let tail = String(text.suffix(6_000)) - var toSpeak: String - if modelManager.isResident, let runtime = try? await modelManager.runtime(requiring: [.text]) { - let prompt = """ - Below is the tail of a window's visible text (likely a terminal transcript). \ - Extract the FINAL assistant/agent response verbatim, or if none exists, briefly \ - summarize what the window shows. Reply with only that text. - - \(tail) - """ - toSpeak = (try? await runtime.generateText(LLMRequest(prompt: prompt))) ?? "" - } else { - toSpeak = "" - } - if toSpeak.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty { - // Model not resident / extraction failed → the last visible lines, honestly. - toSpeak = snapshot.textBlocks.suffix(12).joined(separator: "\n") - } - var chunker = SentenceChunker() - for chunk in chunker.consume(toSpeak) { aiSpeechSynthesizer.speak(chunk) } - if let rest = chunker.flush() { aiSpeechSynthesizer.speak(rest) } - } catch { - let presented = AIError.message(for: error) - aiSpeechSynthesizer.speak(presented.headline) - NSLog("[ThreeFingerSwitcher] speak-last-response failed: \(presented.details ?? presented.headline)") - } - } - } - - /// Install/remove the PTT trigger with the voice opt-in (and at launch). - private func syncVoicePTTMonitor() { - if settings.voiceConversationEnabled { - pttMonitor.setKeyCode(UInt16(clamping: settings.voicePTTKeyCode)) - pttMonitor.onDown = { [weak self] in self?.voiceController.pttDown() } - pttMonitor.onUp = { [weak self] in self?.voiceController.pttUp() } - pttMonitor.start() - } else { - pttMonitor.stop() - } - } - - private func observeVoiceToggle() { - settings.$voiceConversationEnabled - .dropFirst() - .sink { [weak self] _ in - MainActor.assumeIsolated { self?.syncVoicePTTMonitor() } - } - .store(in: &cancellables) - settings.$voicePTTKeyCode - .dropFirst() - .sink { [weak self] code in - MainActor.assumeIsolated { self?.pttMonitor.setKeyCode(UInt16(clamping: code)) } - } - .store(in: &cancellables) - } - - /// At launch, if AI commands are already opted in, rediscover a previously-downloaded model so its - /// status shows "Downloaded" (and a command can lazy-load it) instead of resetting to "Not - /// downloaded" and forcing the user to click Download again every relaunch. Pure disk probe — no - /// network, no heavy load (that happens on first command). No-op if nothing is on disk. - private func reconcileAIModelAtLaunch() { - guard settings.aiCommandsEnabled else { return } - modelManager.reconcileWithDisk() - if let d = selectedAIModelDescriptor() { modelManager.showStatus(for: d) } - } - - /// The model the AI surfaces act on: the user's pinned selection if it resolves, else the registry - /// default. Single source of truth for the download action and the displayed-status settle - /// (`showStatus`) across launch and the opt-in observer. - private func selectedAIModelDescriptor() -> ModelDescriptor? { - let registry = modelManager.registry - return settings.aiSelectedModelID.flatMap { registry.descriptor(id: $0) } - ?? registry.defaultDescriptor ?? registry.models.first - } - // MARK: - Keyboard language opt-in lifecycle /// React to the "Remember the keyboard language per app" toggle: start/stop the service so its @@ -2664,54 +1217,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate } } - /// Begin (or retry) the on-device model download from Settings. Gated on the opt-in by the manager - /// itself. The PRIMARY (and only) error surface is the in-window `.failed` status row + its Retry - /// button — the manager's observable state already carries the clean headline (and copyable - /// details) for it. No app-modal `NSAlert.runModal()` here: its nested run loop would freeze the - /// Settings window, and it would just duplicate the row's message (spec: "Error surfaces are - /// non-blocking and bounded"; design D3). - private func downloadAIModel() { - modelManager.setOptedIn(settings.aiCommandsEnabled) - // Honor the user's pinned model selection (the AI page / unavailable canvas picker), falling - // back to the registry default. - guard let descriptor = selectedAIModelDescriptor() else { return } - Task { @MainActor in - do { - try await modelManager.downloadAndVerify(descriptor) - } catch is CancellationError { - // User cancelled — not a failure; the manager already reset its state. - } catch RuntimeError.cancelled { - // Same: a cancelled provision is not a failure surface. - } catch { - // The manager already reflects `.failed` (clean headline + details) in its observable - // state, which the Settings row renders with a Retry action. Just log for diagnostics. - NSLog("[ThreeFingerSwitcher] AI model download failed: \(AIError.message(for: error).details ?? AIError.message(for: error).headline)") - } - } - } - - /// Download a SPECIFIC capability fleet model (image / ternary / video) the user just enabled in the - /// roster. Reuses the EXISTING `ModelManager.downloadAndVerify` path (the same provisioner / byte-SHA - /// pipeline `downloadAIModel` drives) — no new provisioning seam. The PRIMARY error surface is the - /// roster row's per-model `.failed` status + Retry; the manager's observable state already carries the - /// clean headline + copyable details. No app-modal `NSAlert` here (it would freeze the Settings window). - private func downloadCapabilityModel(_ descriptor: ModelDescriptor) { - modelManager.setOptedIn(settings.aiCommandsEnabled) - Task { @MainActor in - do { - try await modelManager.downloadAndVerify(descriptor) - } catch is CancellationError { - // User cancelled — not a failure; the manager already reset its state. - } catch RuntimeError.cancelled { - // Same: a cancelled provision is not a failure surface. - } catch { - // The manager already reflects `.failed` (clean headline + details) in its observable - // state, which the roster row renders. Just log for diagnostics. - NSLog("[ThreeFingerSwitcher] AI capability model download failed: \(AIError.message(for: error).details ?? AIError.message(for: error).headline)") - } - } - } - // MARK: - Post-Space-switch cleanup & focus /// A Next/Previous Space action only synthesizes the OS shortcut; once the switch lands the @@ -3261,13 +1766,12 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate var lines: [String] = [] if selection.contains(.appData) { - lines.append("• App data & settings — preferences, bands, AI commands, keyboard-language memory, clipboard history, project outputs, first-run state.") + lines.append("• App data & settings — preferences, bands, keyboard-language memory, clipboard history, project outputs, first-run state.") if anyGestureBackupExists { lines.append(" Gesture relocations will be restored FIRST so their backups aren't lost.") } } if selection.contains(.caches) { lines.append("• Caches.") } - if selection.contains(.aiModels) { lines.append("• AI models — the downloaded weights are deleted (re-downloadable); the AI opt-in turns off.") } if selection.contains(.permissions) { lines.append("• Permissions — every granted permission is reset; macOS will prompt again.") } let relaunches = selection.contains(.appData) || selection.contains(.permissions) if relaunches { lines.append("\nThe app will relaunch afterwards.") } @@ -3287,16 +1791,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate } // 2. Quiesce writers so nothing re-creates what's being removed. clipboardMonitor.stop() - if selection.contains(.aiModels) || selection.contains(.appData) { - settings.aiCommandsEnabled = false // evicts residency + forgets download progress - } - if selection.contains(.aiModels) { - // Delete the weights from the dir the runtime actually loads from (the HF cache, via the - // manager's injected provisioner-delete). `appDataReset` below only removes the app-support - // `models/` dir — the WRONG location on the real path — so without this the weights survive - // and the model re-discovers as "Downloaded" on the next opt-in. - modelManager.deleteAllFromDisk() - } // 3. Delete (preferences last, inside `clear`). let outcome = appDataReset.clear(selection) // 4. A cleared identity needs a fresh process (and a data wipe replays the wizard). @@ -3495,16 +1989,9 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate ctx.relocationsPending = { [weak self] in self?.relocationsStillPending ?? false } ctx.logOutNow = { [weak self] in self?.sendLogOutKeystroke() } // The tour's fixed composition (WizardTourBands): the flame band of every app across the - // user's bands, the display band of the twelve window actions, plus the AI band when AI is - // on and the Clipboard band when history is on (sample entries while the store is empty). - // - // The Files band is INTENTIONALLY SKIPPED for the v1 onboarding tour. Unlike Clipboard (a static - // list of sample entries that reads identically in the tour and the real launcher), the Files band - // is a LIVE, controller-backed drill surface: its navigation is meaningless without a - // `FilesColumnController` + the recognizer's `filesDrillActive` routing, neither of which the - // wizard's static `launcherTour*` path wires. A non-drillable sample Files row would misrepresent - // the band, so it's discovered via the Hub's Files page + its own opt-in, not the first-touch tour. - ctx.launcherBands = { [weak self] clipboardOn, aiOn in + // user's bands, the display band of the twelve window actions, plus the Clipboard band when + // history is on (sample entries while the store is empty). + ctx.launcherBands = { [weak self] clipboardOn in guard let self else { return [] } let clipboard: ContextBand? = clipboardOn ? { let entries = self.clipboardStore.bandWindow(limit: self.settings.clipboardRecentWindow) @@ -3512,8 +1999,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate from: entries.isEmpty ? WizardSampleContent.clipboardEntries() : entries) }() : nil return WizardTourBands.compose(userBands: self.favoritesStore.favorites.bands, - aiOn: aiOn, - seededAIBand: AIBand.seededBand, clipboardBand: clipboard) } ctx.launcherLive = { [weak self] in self?.isLauncherEffective ?? false } @@ -3646,7 +2131,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate let ctx = HubContext(settings: settings, favorites: favoritesStore, clipboard: clipboardStore, - models: modelManager, permissions: permissions) // §11.2 Real demo content for the gesture previews — the SAME providers `makeWizardContext` // wires, so the Hub renders the real switcher/launcher seeded with the user's content. No new @@ -3683,7 +2167,7 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate } } } - ctx.launcherBands = { [weak self] clipboardOn, aiOn in + ctx.launcherBands = { [weak self] clipboardOn in guard let self else { return [] } let clipboard: ContextBand? = clipboardOn ? { let entries = self.clipboardStore.bandWindow(limit: self.settings.clipboardRecentWindow) @@ -3691,23 +2175,10 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate from: entries.isEmpty ? WizardSampleContent.clipboardEntries() : entries) }() : nil return WizardTourBands.compose(userBands: self.favoritesStore.favorites.bands, - aiOn: aiOn, - seededAIBand: AIBand.seededBand, clipboardBand: clipboard) } // Clipboard. ctx.onClearClipboard = { [weak self] includingPinned in self?.clipboardStore.clear(includingPinned: includingPinned) } - // AI. - ctx.onDownloadModel = { [weak self] in self?.downloadAIModel() } - ctx.onDownloadCapabilityModel = { [weak self] descriptor in self?.downloadCapabilityModel(descriptor) } - // AI — Background autonomy audit viewer (`ai-background-autonomy`, §7). The viewer reads the - // durable ledger synchronously; a store-persist failure is surfaced as a bounded, non-blocking - // banner (headline routed through the single `AIError.message(for:)` translator — never raw OS text). - ctx.recentAuditRecords = { [weak self] limit in self?.auditLog.recent(limit: limit) ?? [] } - ctx.auditStorePersistError = { [weak self] in - guard let error = self?.auditLog.lastPersistError else { return nil } - return AIError.message(for: error) - } // Keyboard Language — the picker's source list (forwarded from the service's controller seam so // the page never imports Carbon). ctx.enabledInputSources = { [weak self] in self?.keyboardLanguageService.controllerEnabledSources() ?? [] } diff --git a/Sources/ThreeFingerSwitcher/App/ParkController.swift b/Sources/ThreeFingerSwitcher/App/ParkController.swift deleted file mode 100644 index 48408ea..0000000 --- a/Sources/ThreeFingerSwitcher/App/ParkController.swift +++ /dev/null @@ -1,536 +0,0 @@ -import Foundation -import AppKit - -/// App-side coordinator for the NOTCH-NATIVE sessions (`ai-parked-sessions` + `notch-native-conversations`): -/// wires the pure machinery (`SerialParkScheduler` + `ParkedSessionStore` + `ParkLifecycleCoordinator`) to -/// the `NotchHomeZoneController` rail/expanded panel and owns the per-session `NotchSessionEngine`s. This is -/// the lifecycle/scheduler GLUE: it owns the one place the slot count is interpreted (the scheduler) and the -/// durable store, and it republishes the rail rows whenever the parked set changes. -/// -/// Sessions are BORN here (durable at birth — `newSession()` upserts before the first turn), expanded in -/// place (`expand(id:)` binds an engine to the stored conversation), collapsed back to background -/// (`collapse()` persists WITHOUT cancelling an in-flight turn), and removed only by expiry (the -/// auto-dismiss pass) or deletion (`discard(_:)`, the single authoritative removal path). The launcher's -/// quick actions never reach this controller — they have no sessions at all. -/// -/// Verification: the pure pieces are `swift test`-ed (scheduler one-active-now + K-ready, store round-trip, -/// lifecycle evict/expire/discard, and the session verbs via an injected engine factory); this controller is -/// `xcodebuild` compile-verified and user-run-verified for the live rail/expand/glow behavior. -@MainActor -final class ParkController { - private let store: ParkedSessionStore - private let scheduler: SerialParkScheduler - private let lifecycle: ParkLifecycleCoordinator - /// Builds a fresh conversation engine per session (injected so tests drive a stub runtime). - private let engineFactory: () -> NotchSessionEngine - /// The shared audit ledger, used ONLY by `purge(_:)` (`notch-conversation-gestures`): the - /// purge-delete gesture erases the session's audit records everywhere; the plain `discard(_:)` - /// never touches the ledger. Optional so existing call sites/tests stay source-compatible. - private let auditLog: AuditLog? - let notch: NotchHomeZoneController - - /// Fires whenever whether-a-conversation-is-expanded CHANGES (nil↔value on `expandedID`), from every - /// path (new session, expand, expand-another, collapse, discard-of-expanded, feature-off). The - /// coordinator drives `GestureRecognizer.notchConversationActive` off it — the single choke point, so - /// the recognizer can never be left stuck in notch mode. An expand-another (a→b) publishes NO blip - /// (the state stays "expanded"); the publish compares against the last published value. - var onExpandedChanged: ((Bool) -> Void)? - private var lastPublishedExpanded = false - - /// The per-session engines that currently exist: the expanded (foreground) session's engine, plus any - /// collapsed session whose foreground turn is still streaming (collapse does NOT cancel — the engine - /// stays alive detached until the turn settles, then is dropped). Exactly ONE session is expanded at a - /// time; an idle collapsed session has no engine (recreated on the next expand). - private var engines: [AgentSessionID: NotchSessionEngine] = [:] - /// The session currently expanded in the notch panel, if any. - private(set) var expandedID: AgentSessionID? - - init(store: ParkedSessionStore = DiskParkedSessionStore(), - maxParked: Int, - autoDismissCountdown: TimeInterval, - revealDwell: TimeInterval = 0, - engineFactory: @escaping () -> NotchSessionEngine, - auditLog: AuditLog? = nil) { - self.store = store - self.auditLog = auditLog - self.scheduler = SerialParkScheduler(sessions: store.all()) - self.lifecycle = ParkLifecycleCoordinator( - store: store, - lifecycle: ParkLifecycle(maxParked: maxParked, autoDismissCountdown: autoDismissCountdown)) - self.engineFactory = engineFactory - self.notch = NotchHomeZoneController() - normalizeRowsAtLaunch() - self.notch.setRevealDwell(revealDwell) - // The rail lists sessions most-recently-used FIRST (right after the "+ New chat" card). - self.notch.sessionsProvider = { [weak self] in self?.scheduler.railSnapshot() ?? [] } - self.notch.onExpand = { [weak self] id in self?.expand(id) } - self.notch.onNewSession = { [weak self] in self?.newSession() } - self.notch.onCollapse = { [weak self] in self?.collapse() } - self.notch.onDiscard = { [weak self] id in self?.discard(id) } - self.notch.engineProvider = { [weak self] id in self?.engines[id] } - self.notch.refresh() - } - - /// Install/remove the cursor-reveal rail (gated by the agent-feature opt-in). Turning the feature off - /// collapses any expanded session first (persisting it) — the synchronous-teardown path. - func setEnabled(_ on: Bool) { - if !on, expandedID != nil { collapse() } - notch.setEnabled(on) - } - - /// Live-update the notch reveal dwell (Hub slider). Forwards to the cursor-reveal controller. - func setRevealDwell(_ interval: TimeInterval) { - notch.setRevealDwell(interval) - } - - /// Wire the in-notch settings zone's tuning dial (`notch-timeline-and-tuning`): the slider's - /// read/write seams and the model max that caps the "Max" stop. Returns self so the owner's lazy - /// construction can chain it. - @discardableResult - func configuredForTuning(provider: @escaping () -> NotchTuning, - modelMax: @escaping () -> Int, - onChange: @escaping (NotchTuning) -> Void) -> ParkController { - notch.tuningProvider = provider - notch.modelMaxProvider = modelMax - notch.onTuningChanged = onChange - return self - } - - /// The current park state for `id`, delegating to the scheduler's thread-safe (lock-guarded) read — the - /// same any-thread seam the background-autonomy runner's `parkStateOf` uses. The expanded session's row - /// is kept `.active` so the runner takes the foreground path — its engine's approval gate owns it. - func parkState(of id: AgentSessionID) -> ParkState { - scheduler.parkState(of: id) - } - - // MARK: - Birth (the "+ New chat" card — durable at FIRST MESSAGE) - - /// Create a NEW session at the notch: a fresh engine + conversation, expanded in place for typing. It is - /// **NOT** written to the store yet — an empty new chat is never docked. The session becomes durable + - /// joins the rail on its **first message** (the engine's `onTurnStarted` → `persistNewlySent`); if it is - /// closed still empty, `collapseCurrentIfNeeded` discards it (no store row, no rail card). - @discardableResult - func newSession() -> AgentSessionID { - collapseCurrentIfNeeded() - let engine = obtainEngine(for: nil) - let conversation = engine.startNew() - let id = conversation.id - engines[id] = engine - expandedID = id - wireEngine(engine, id: id) - notch.expandSession(id) - publishExpandedState() - return id - } - - /// The first-message durability hook (`engine.onTurnStarted`): the user just sent a turn, so the - /// (until-now unsaved) session becomes durable + joins the dock. The very first save runs the eviction - /// pass (the chat is now real and counts toward the cap); later sends just keep the store fresh. - private func persistNewlySent(_ id: AgentSessionID, _ conversation: AgentConversation) { - guard expandedID == id else { return } - let firstSave = store.conversation(id) == nil - persist(conversation, rowState: .active, badge: .clear) - if firstSave { runEvictionAndRepublish() } else { republish() } - } - - // MARK: - Expand (a card opens in place) - - /// Expand a session's card into the in-place conversation panel: bind an engine to the stored - /// conversation, mark its row `.active` (foreground — excluded from the background runnable set), - /// clear its unseen badge, and republish (which clears the glow if this was the last needs-you). - func expand(_ id: AgentSessionID) { - guard let conversation = store.conversation(id) else { return } - collapseCurrentIfNeeded() - let engine = obtainEngine(for: id) - if engine.conversation?.id != id || !engine.isTurnInFlight { - engine.bind(conversation) - } - engines[id] = engine - expandedID = id - wireEngine(engine, id: id) - updateRow(id) { row in - row.state = .active - row.badgeCount = 0 - row.updatedAt = Date() - } - republish() - notch.expandSession(id) - publishExpandedState() - } - - /// Collapse the expanded session back to the rail: persist its snapshot, return its row to the - /// background set, and — the load-bearing contract — do NOT cancel an in-flight turn: the engine stays - /// alive detached; its `onTurnSettled` persists the completed turn and bumps the badge, and only then - /// is the engine dropped. An idle engine is dropped immediately. - /// `closingPanel` (the swipe-up "minimize" gesture) straight-CLOSES the panel — it shrinks the expanded - /// conversation directly into the notch rather than returning to the visible rail and then grace- - /// dismissing (the two-stage "dwell on the dock"). The session state work is identical either way; only - /// the panel animation differs. The chevron / Esc collapse (default) returns to the visible rail. - func collapse(closingPanel: Bool = false) { - guard expandedID != nil else { return } - if closingPanel { - // Straight-close: shrink the panel with the conversation STILL BOUND (so it shrinks the REAL - // conversation, never flashing the empty/new-chat state or the rail), and do the session - // state-collapse (unbind + persist + republish) only AFTER the panel is hidden. - notch.collapseAndClose { [weak self] in - guard let self else { return } - self.collapseCurrentIfNeeded() - self.republish() - self.publishExpandedState() - } - return - } - collapseCurrentIfNeeded() - republish() - notch.collapseToRail() - publishExpandedState() - } - - /// The shared collapse core (used by `collapse()`, and by `newSession`/`expand` when another session - /// is already expanded — one foreground at a time). - private func collapseCurrentIfNeeded() { - guard let id = expandedID else { return } - guard let engine = engines[id] else { expandedID = nil; return } - let snapshot = engine.unbind() - // An unsaved EMPTY new chat (opened but never sent a message) is DISCARDED, not docked: drop the - // engine, remove any stray store trace, and create no rail row. Nothing is saved to the dock until - // the first message (which persists the session before it can ever reach here empty). - if snapshot?.messages.isEmpty ?? true { - engines[id] = nil - if store.conversation(id) != nil { try? lifecycle.discard(id) } - expandedID = nil - return - } - if let snapshot { try? store.upsert(row(for: id, default: snapshot), conversation: snapshot) } - if engine.isPausedAtApproval { - // Docked while the route loop is SUSPENDED on this engine's approval gate: keep the engine - // (dropping it would orphan the suspended continuation — the response stops forever) and - // surface `.needsYou` (the step is blocked on the user regardless of tier). Re-expanding - // re-presents the same card; Approve/Skip resumes the original paused step. - updateRow(id) { row in - row.state = .needsYou - row.badgeCount = max(row.badgeCount, 1) - row.updatedAt = Date() - } - } else if engine.isTurnInFlight { - // Detached but still streaming: keep the engine + the `.active` row (structurally excluded - // from the background runnable set, so nothing double-advances it). `onTurnSettled` finishes - // the story: persist, badge, drop. - } else { - engines[id] = nil - updateRow(id) { row in - row.state = .idle - row.updatedAt = Date() - } - } - expandedID = nil - } - - // MARK: - Discard (deletion — cancel pending, remove; completed side effects are NOT rolled back) - - /// The SINGLE authoritative removal path for ALL dismissals (manual deletion AND the expiry pass): - /// cancel any pending generation (the engine is the single cancellation owner — `cancelAll()` - /// cancels the turn task and resolves a suspended approval gate), remove the durable conversation + - /// rail row, re-seed the scheduler, republish. Idempotent — discarding an already-removed id is a - /// harmless no-op. - func discard(_ id: AgentSessionID) { - engines[id]?.cancelAll() - engines[id] = nil - if expandedID == id { - expandedID = nil - notch.collapseToRail() - } - try? lifecycle.discard(id) - scheduler.setSessions(store.all()) - notch.refresh() - publishExpandedState() - } - - /// The PURGE-DELETE (`notch-conversation-gestures`, the fast right-flick on the expanded - /// conversation): the authoritative discard PLUS erasing every audit record attributed to the - /// session — from the in-memory ring and the durable audit file. User-initiated only; this path - /// deliberately writes no log line referencing the session. The plain delete affordances stay - /// `discard(_:)`, which leaves the ledger intact. - func purge(_ id: AgentSessionID) { - discard(id) - auditLog?.purge(sessionID: id) - } - - // MARK: - Scheduler feedback (background advance / escalation) - - /// Report a background-advance result back to the scheduler (the driver's failure path, the media - /// feed) and republish the rail. There is NO terminal flow (`refactor-park-and-background-agents`): - /// a settled `.done` is an unseen result (idle + badge), a `.failed` re-parks with a scheduled - /// retry — nothing here ever removes a session. - func didAdvance(_ id: AgentSessionID, result: ToolStepResult) { - scheduler.didAdvance(id, result: result) - persistRowSnapshot() - scheduler.setSessions(store.all()) - notch.refresh() - } - - /// Accept an escalation (`ai-background-autonomy`): set `.needsYou` + badge, persist, and light the - /// ambient glow via the republish — never a modal. - func escalate(_ id: AgentSessionID, reason: String) { - scheduler.escalate(id, reason: reason) - persistRowSnapshot() - notch.refresh() - } - - /// The scheduler seam the batched runtime fills (`runnableSessions(now:maxSlots:)`) — exposed so the - /// batched-runtime slice can drive K slots with NO protocol change (one active now, K-ready). - var parkScheduler: ParkScheduler { scheduler } - - /// The AI system's business RIGHT NOW, for automatic model eviction - /// (`model-idle-ttl-and-memory-pressure` D2): pulled by `ModelManager` at decision time on the - /// main actor, so evict-vs-just-scheduled ordering is deterministic. `foregroundSessionActive` is - /// an OR over conversational surfaces — the expanded notch session, any `.active` row, and (when - /// the voice change lands) an open voice conversation joins through the same flag. - func quiescenceSnapshot() -> QuiescenceSnapshot { - let rows = scheduler.snapshot() - return QuiescenceSnapshot( - turnInFlight: engines.values.contains { $0.isTurnInFlight }, - foregroundSessionActive: expandedID != nil || rows.contains { $0.state == .active }, - nextScheduledWork: rows.compactMap(\.nextRunAt).min()) - } - - /// The engine bound to the expanded session, if any (`add-voice-computer-use-agent`: the - /// `set_auto_mode` tools land the grant on the conversation the user is looking at). - func expandedEngine() -> NotchSessionEngine? { - expandedID.flatMap { engines[$0] } - } - - /// The any-human-touch kill switch's fan-out (`add-voice-computer-use-agent`): discard every - /// in-flight turn — a DISCARD (no partial message appended, not a failure), exactly the verb the - /// canvas's own discard uses. - func discardInFlightTurns() { - for engine in engines.values where engine.isTurnInFlight { - engine.discardTurn() - } - } - - // MARK: - Maintenance - - /// Relaunch normalization (`refactor-park-and-background-agents`): no engine can exist at launch, so - /// a row persisted `.active` (the app quit while the session was expanded or its turn was detached - /// in flight) is stale. When its conversation's last message still awaits an assistant reply, the - /// row becomes `.parked` + scheduled NOW — the background driver re-runs the interrupted turn (a - /// quit mid-response resumes instead of stranding a forever-protected zombie row); otherwise it - /// simply idles. `.needsYou` rows stay (blocked on the user, not runnable); rows persisted under the - /// retired terminal state decode as `.idle` at the store boundary. - private func normalizeRowsAtLaunch() { - for var row in scheduler.snapshot() where row.state == .active { - let awaitsReply = store.conversation(row.id) - .flatMap(\.messages.last) - .map { $0.role != .assistant } ?? false - row.state = awaitsReply ? .parked : .idle - row.nextRunAt = awaitsReply ? Date() : nil - row.updatedAt = Date() - try? store.upsertRow(row) - } - scheduler.setSessions(store.all()) - } - - /// The background driver (`refactor-park-and-background-agents`): serve the scheduler's runnable set - /// (ONE slot in this capability — the K-ready seam is unchanged) by rebuilding a detached engine - /// from the durable conversation and running its pending turn through the same machinery, settle - /// path, and badge classification as a collapsed foreground turn. Skips entirely while ANY turn is - /// in flight (the foreground or a detached turn owns the single slot) — which is also the - /// double-serve guard: an advancing session keeps its engine in flight, so the next tick serves - /// nothing until the turn settles and rewrites the row. Called on the coarse maintenance timer and - /// once after launch normalization. Returns the served ids (for tests). - @discardableResult - func runAdvancePass(now: Date = Date()) -> [AgentSessionID] { - // Only a PRODUCING turn owns the slot; an engine suspended at its approval gate consumes no - // generation and must never block other sessions from advancing. - guard !engines.values.contains(where: { $0.isGenerating }) else { return [] } - let runnable = scheduler.runnableSessions(now: now, maxSlots: 1) - var served: [AgentSessionID] = [] - for id in runnable { - guard let conversation = store.conversation(id) else { continue } - let engine = obtainEngine(for: id) - engine.bind(conversation) - engines[id] = engine - wireEngine(engine, id: id) - engine.advance() - if !engine.isTurnInFlight { - // Nothing pending after all, or the model is unavailable: go DORMANT (clear the - // schedule) until the user returns — never a busy retry loop against a missing model. - engines[id] = nil - updateRow(id) { row in - row.nextRunAt = nil - row.updatedAt = now - } - } - served.append(id) - } - if !served.isEmpty { notch.refresh() } - return served - } - - /// Run the auto-dismiss pass (the OPT-IN expiry): an `.idle`, fully-seen (zero-badge) session past - /// the configured countdown is DISMISSED FOREVER through the authoritative `discard(_:)` path — a - /// countdown of 0 (the default) dismisses nothing. Unseen results, active/needs-you sessions, and - /// parked pending work are never touched. Called on a coarse repeating timer by the owner - /// (`AppCoordinator`). `now:` injected so the pass is deterministic in tests. - @discardableResult - func runAutoDismissPass(now: Date = Date()) -> [AgentSessionID] { - let dismissed = lifecycle.runAutoDismissPass(now: now) - for id in dismissed { - engines[id]?.cancelAll() - engines[id] = nil - } - scheduler.setSessions(store.all()) - notch.refresh() - publishExpandedState() // defensive — expanded rows are protected, but the choke point stays single - return dismissed - } - - /// Publish the expanded-or-not state IFF it changed since the last publish (the no-blip guarantee: - /// an expand-another transition keeps it `true` throughout, so the recognizer's notch mode never - /// flickers off mid-gesture). - private func publishExpandedState() { - let expanded = (expandedID != nil) - guard expanded != lastPublishedExpanded else { return } - lastPublishedExpanded = expanded - onExpandedChanged?(expanded) - } - - // MARK: - Test seams - - /// The engine currently bound for `id`, if one exists (expanded, or detached finishing a turn). - func engineForTest(_ id: AgentSessionID) -> NotchSessionEngine? { engines[id] } - - // MARK: - Helpers - - /// Reuse the live engine for `id` (a detached one still finishing a turn) or build a fresh one. - private func obtainEngine(for id: AgentSessionID?) -> NotchSessionEngine { - if let id, let existing = engines[id] { return existing } - return engineFactory() - } - - /// Wire the engine's settle callback: persist the snapshot, then update the row — the expanded - /// session stays `.active` with no badge (the user is watching); a detached (collapsed) session - /// classifies by HOW the turn ended. A settled answer is NEVER terminal - /// (`refactor-park-and-background-agents` — the old `taskComplete → .completed → instant - /// auto-dismiss` classification deleted every chat docked mid-response the moment its answer - /// landed): detached-answered goes `.idle` + unseen badge. A detached PAUSE keeps the session - /// waiting on the user (`.needsYou` when the step escalated — never downgraded — else dormant - /// `.parked`); a detached FAILURE re-parks with a scheduled retry via the advance-feedback seam. - private func wireEngine(_ engine: NotchSessionEngine, id: AgentSessionID) { - // A confirm/dangerous step that pauses at the gate AFTER the user docked (the turn kept - // streaming detached) surfaces `.needsYou` — never an invisible suspension behind "Working…". - engine.onApprovalPending = { [weak self] in - guard let self, self.expandedID != id else { return } - self.updateRow(id) { row in - row.state = .needsYou - row.badgeCount = max(row.badgeCount, 1) - row.updatedAt = Date() - } - self.republish() - } - engine.onTurnStarted = { [weak self] conversation in - self?.persistNewlySent(id, conversation) - } - engine.onTurnSettled = { [weak self] conversation, settlement in - guard let self else { return } - let isExpanded = (self.expandedID == id) - if isExpanded { - self.persist(conversation, rowState: .active, badge: .clear) - self.republish() - return - } - switch settlement { - case .answered: - self.persist(conversation, rowState: .idle, badge: .increment) - self.engines[id] = nil - case .pausedAwaitingUser: - // The escalation callback (dangerous) already set + persisted `.needsYou`; a - // confirm-tier pause waits DORMANT (`.parked` with the schedule cleared, so the driver - // can never re-serve — and re-run — the paused turn until the user returns). - let rowState: ParkState = self.scheduler.parkState(of: id) == .needsYou ? .needsYou : .parked - self.persist(conversation, rowState: rowState, badge: .keep) - self.updateRow(id) { row in row.nextRunAt = nil } - self.engines[id] = nil - case let .failed(headline): - self.persist(conversation, rowState: .parked, badge: .keep) - self.engines[id] = nil - // Re-park with a scheduled retry so the failure is observable and never a dead end. - self.didAdvance(id, result: ToolStepResult(tool: "turn", - status: .failed(headline: headline), - summary: headline)) - } - self.republish() - } - } - - /// How a persist touches the row's unseen-result badge. - private enum BadgeUpdate { - /// Seen: the user is watching (expanded) — reset to zero. - case clear - /// A new unseen result landed detached — bump the stored count. - case increment - /// No new result (a pause / failure) — leave the stored count alone. - case keep - } - - /// Upsert `conversation` + its row. - private func persist(_ conversation: AgentConversation, rowState: ParkState, badge: BadgeUpdate) { - var row = row(for: conversation.id, default: conversation) - row.title = conversation.title - row.state = rowState - switch badge { - case .clear: row.badgeCount = 0 - case .increment: row.badgeCount += 1 - case .keep: break - } - row.updatedAt = Date() - do { - try store.upsert(row, conversation: conversation) - } catch { - // The store already mapped + logged the failure; record it as a bounded failed row so the - // user sees it on the rail (never an NSAlert, never raw text in the headline). - recordFailedRow(row, error: error) - } - scheduler.setSessions(store.all()) - } - - /// The stored row for `id`, or a fresh one derived from `conversation` when none exists yet. - private func row(for id: AgentSessionID, default conversation: AgentConversation) -> ParkedSession { - scheduler.snapshot().first(where: { $0.id == id }) - ?? ParkedSession(id: id, title: conversation.title, state: .idle, - badgeCount: 0, nextRunAt: nil, updatedAt: Date()) - } - - /// Mutate the stored row for `id` in place (no-op when the row doesn't exist) and re-seed the scheduler. - private func updateRow(_ id: AgentSessionID, _ mutate: (inout ParkedSession) -> Void) { - guard var row = scheduler.snapshot().first(where: { $0.id == id }) else { return } - mutate(&row) - try? store.upsertRow(row) - scheduler.setSessions(store.all()) - } - - private func runEvictionAndRepublish() { - lifecycle.runEvictionPass(now: Date()) - scheduler.setSessions(store.all()) - notch.refresh() - } - - private func republish() { - scheduler.setSessions(store.all()) - notch.refresh() - } - - /// Persist the scheduler's current rows back to the store (after a `didAdvance`/`escalate` mutation) so - /// the rail rebuilds correctly on relaunch. - private func persistRowSnapshot() { - for row in scheduler.snapshot() { try? store.upsertRow(row) } - } - - private func recordFailedRow(_ row: ParkedSession, error: Error) { - var failed = row - failed.state = .idle - failed.updatedAt = Date() - try? store.upsertRow(failed) - } -} diff --git a/Sources/ThreeFingerSwitcher/App/StatusItemController.swift b/Sources/ThreeFingerSwitcher/App/StatusItemController.swift index fd0572d..da60ae6 100644 --- a/Sources/ThreeFingerSwitcher/App/StatusItemController.swift +++ b/Sources/ThreeFingerSwitcher/App/StatusItemController.swift @@ -85,13 +85,6 @@ final class StatusItemController: NSObject, NSMenuDelegate { if !coordinator.isTrackpadAvailable { toggle.isEnabled = false } groups.append([toggle, quickAddMenuItem()]) - // Speak-last-response (`add-voice-computer-use-agent`, the v0 wedge): read the front window's - // last assistant reply aloud. No mic, no conversation — TTS + AX only, so it's offered - // whenever AI commands are on (voice opt-in not required). - if coordinator.aiCommandsEnabledForMenu { - groups.append([item("Speak Last Response", #selector(speakLastResponse))]) - } - // Keep Awake fallback: while the automation is active (screen dimmed near-black), surface a // checked "Active — Stop" line as a fire escape so brightness can always be recovered even if // the trackpad first-touch stop misbehaves. Its own group, present only while active. @@ -154,7 +147,6 @@ final class StatusItemController: NSObject, NSMenuDelegate { @objc private func toggleEnabled() { coordinator.toggleEnabled() } @objc private func stopKeepAwake() { coordinator.stopKeepAwake() } - @objc private func speakLastResponse() { coordinator.speakLastResponse() } @objc private func addFrontApp(_ sender: NSMenuItem) { guard let id = sender.representedObject as? UUID else { return } coordinator.addFrontAppToBand(id) diff --git a/Sources/ThreeFingerSwitcher/Dock/DockPreviewError.swift b/Sources/ThreeFingerSwitcher/Dock/DockPreviewError.swift index 84c4baf..165dd38 100644 --- a/Sources/ThreeFingerSwitcher/Dock/DockPreviewError.swift +++ b/Sources/ThreeFingerSwitcher/Dock/DockPreviewError.swift @@ -2,16 +2,16 @@ import Foundation /// Failures the Dock-preview commit (raise / un-minimize-then-raise) can report. /// -/// The Dock-domain parallel to the Files feature's `FileActionError`: a small Core taxonomy conforming to +/// The Dock-domain error taxonomy: a small Core taxonomy conforming to /// `LocalizedError` with a clean, per-case, user-facing sentence for every case — so a failed commit /// surfaces as a bounded headline that reads the same everywhere, never a reflected enum dump or raw OS -/// text. (The AI `RuntimeError` and `FileActionError` are deliberately NOT reused: their cases are +/// text. (Other domains' taxonomies are deliberately NOT reused: their cases are /// model-/filesystem-layer concepts with no meaning for a window-raise action.) /// /// **Map at the boundary:** any underlying AX/OS failure is converted into one of these cases where it /// crosses into feature/UI code, and the raw text is stringified into the opt-in `details` payload there — /// kept ONLY for a "Show details / Copy" disclosure and logs, NEVER used as the headline. Carrying -/// `details` as a `String?` (not a raw `Error`) keeps the enum `Equatable`, like `FileActionError`. +/// `details` as a `String?` (not a raw `Error`) keeps the enum `Equatable`. enum DockPreviewError: Error, Equatable { /// The chosen window could no longer be found at commit time (closed, or moved off this Space). /// `name` is the window's display title. diff --git a/Sources/ThreeFingerSwitcher/Dock/DockTile.swift b/Sources/ThreeFingerSwitcher/Dock/DockTile.swift index b43f898..b310a02 100644 --- a/Sources/ThreeFingerSwitcher/Dock/DockTile.swift +++ b/Sources/ThreeFingerSwitcher/Dock/DockTile.swift @@ -35,7 +35,7 @@ struct DockSnapshot: Equatable { let screenFrame: CGRect } -/// The seam onto the Dock's Accessibility tree — mirrors the `FileWorkspace` / `LLMRuntime` idea so the +/// The seam onto the Dock's Accessibility tree — a protocol seam so the /// hover logic depends only on this protocol and can be driven by a static fake in tests. The real /// conformer (`AXDockReader`) reads `Dock.app`'s AX tree; it degrades to `nil` (no crash, no raw error) /// when the tree or an attribute can't be read. diff --git a/Sources/ThreeFingerSwitcher/Files/DirectoryLister.swift b/Sources/ThreeFingerSwitcher/Files/DirectoryLister.swift deleted file mode 100644 index 80973a3..0000000 --- a/Sources/ThreeFingerSwitcher/Files/DirectoryLister.swift +++ /dev/null @@ -1,168 +0,0 @@ -import Foundation -import UniformTypeIdentifiers - -/// The Files band's directory-listing boundary: it reads ONE folder's **local** contents off the live -/// filesystem and turns each `URL` into a pure `FileEntry`. This is the only place that touches -/// `FileManager` (the navigation model stays pure and is fed the resulting entries), so it is also the -/// place that maps a `FileManager`/OS read failure into the shared `FileActionError` taxonomy — Core -/// above it never sees a raw `NSError`. -/// -/// **Off the main thread (spec):** the read runs on a detached `userInitiated` task so opening a large -/// folder never blocks the UI. The call is `async throws` — a non-throwing return yields the listed -/// entries; a throwing return is a clean `FileActionError.folderUnreadable` whose raw OS text rides only -/// on the opt-in `details` payload (and logs), never the headline. -/// -/// **Local only (spec / design D9):** the listing requests `URLResourceValues` and **skips** any entry -/// the filesystem reports as a non-local / iCloud-placeholder item, so navigation never blocks on (or -/// descends into) a network or yet-to-download location. Hidden files are skipped via -/// `.skipsHiddenFiles`, mirroring how `loadInstalledApps` keeps the shallow scan to user-facing items. -/// -/// **Stable order (spec):** entries come back sorted by the supplied `FilesSortOrder` so a re-list of the -/// same folder yields the same order (the path-derived `FileEntry.id` keeps the highlight stable across -/// re-lists). The folder may itself be re-listed at any time; nothing here is cached. -struct DirectoryLister { - /// The resource values the listing pulls per entry — exactly the spec's "is-directory, modification - /// date, regular-file", plus `.contentType` (to pick a `FileKind` row glyph) and the locality keys - /// used to drop non-local / iCloud-placeholder items at the boundary. Requesting them up front (via - /// `includingPropertiesForKeys:`) lets the filesystem prefetch them, so `resourceValues(forKeys:)` - /// per entry is cheap. - private static let prefetchKeys: [URLResourceKey] = [ - .isDirectoryKey, .contentModificationDateKey, .isRegularFileKey, - .contentTypeKey, .isUbiquitousItemKey, .nameKey, - ] - - /// The `Set` form used for the per-entry `resourceValues(forKeys:)` read. - private static let readKeys: Set = Set(prefetchKeys) - - init() {} - - /// List `folder`'s **local** contents, sorted by `order`, off the main thread. - /// - /// Maps a read failure into `FileActionError.folderUnreadable(name:details:)` at this boundary (the - /// raw `FileManager`/OS error becomes the opt-in `details`, never the headline). The detached task is - /// `userInitiated`; the body is pure-Foundation, so it is safe to run off the actor. - func contents(of folder: URL, sortedBy order: FilesSortOrder) async throws -> [FileEntry] { - try await Task.detached(priority: .userInitiated) { - try Self.read(folder, order: order) - }.value - } - - /// The synchronous read body (extracted so it is exercisable directly and runs on the detached task). - /// `nonisolated`/`static` and free of any actor state — it only reads `FileManager`. - private static func read(_ folder: URL, order: FilesSortOrder) throws -> [FileEntry] { - let fm = FileManager.default - let urls: [URL] - do { - urls = try fm.contentsOfDirectory( - at: folder, - includingPropertiesForKeys: prefetchKeys, - options: [.skipsHiddenFiles, .skipsPackageDescendants] - ) - } catch { - // Map at the boundary: the raw FileManager/OS error never escapes into feature/UI code — it - // is stringified into the opt-in `details` here (the only place it survives, off the headline). - throw FileActionError.folderUnreadable(name: displayName(of: folder), - details: String(describing: error)) - } - - let entries = urls.compactMap(entry(for:)) - return order.sorted(entries) - } - - /// Turn one listed `URL` into a `FileEntry`, or `nil` when it should be skipped (a non-local / - /// iCloud-placeholder item, per the local-only scope). Reads the per-entry resource values; an entry - /// whose values can't be read is treated as a plain file (it still lists, just without rich metadata) - /// rather than dropped, so a transient stat hiccup doesn't make a row vanish. - private static func entry(for url: URL) -> FileEntry? { - let values = try? url.resourceValues(forKeys: readKeys) - - // Local-only: drop iCloud / non-local placeholder items so navigation never blocks on a - // not-yet-downloaded file (these would also blow the latency budget — design D9 / non-goals). - if values?.isUbiquitousItem == true { return nil } - - let isDirectory = values?.isDirectory ?? false - let name = values?.name ?? url.lastPathComponent - let modificationDate = values?.contentModificationDate - let kind = fileKind(isDirectory: isDirectory, contentType: values?.contentType) - return FileEntry(url: url, name: name, isDirectory: isDirectory, - modificationDate: modificationDate, kind: kind) - } - - /// The folder's display name for an error headline — the localized filesystem name when available, - /// else the last path component. Never includes raw OS text (that rides on `details`). - private static func displayName(of folder: URL) -> String { - let localized = (try? folder.resourceValues(forKeys: [.localizedNameKey]))?.localizedName - let name = localized ?? folder.lastPathComponent - return name.isEmpty ? folder.path : name - } - - // MARK: - Kind classification (boundary: UTType → the AppKit-free FileKind) - - /// Map a directory flag + the entry's `UTType` to the coarse, AppKit-free `FileKind` the view uses to - /// pick a row glyph (the concrete SF Symbol stays in the view layer). A directory is always `.folder` - /// regardless of its UTI (e.g. a bundle), matching `FileEntry`'s contract. Classification is by UTI - /// conformance so subtypes (e.g. a specific image format) fold into their family. - static func fileKind(isDirectory: Bool, contentType: UTType?) -> FileKind { - if isDirectory { return .folder } - guard let type = contentType else { return .other } - if type.conforms(to: .image) { return .image } - if type.conforms(to: .movie) || type.conforms(to: .video) { return .video } - if type.conforms(to: .audio) { return .audio } - if type.conforms(to: .pdf) { return .pdf } - if type.conforms(to: .archive) { return .archive } - if type.conforms(to: .sourceCode) || type.conforms(to: .script) { return .sourceCode } - if type.conforms(to: .application) { return .application } - if type.conforms(to: .text) { return .text } - return .other - } -} - -/// The configurable order a folder's entries are listed in (spec: "ordered by a configurable sort -/// order"). A pure, AppKit-free value so it travels with the Files domain and is unit-testable; the Hub's -/// behavior page and `AppSettings` persist the user's choice and hand it to the lister. -/// -/// Every order is **folders-first** (a Finder-mimic convention) and breaks ties with a localized, -/// case-insensitive name compare so the result is deterministic — a re-list of an unchanged folder yields -/// an identical ordering, which (with the path-stable `FileEntry.id`) is what keeps the highlight from -/// jumping. -enum FilesSortOrder: String, Equatable, CaseIterable, Codable, Sendable { - /// A→Z by name (the default). - case name - /// Most-recently-modified first (entries without a date sort last). - case dateModified - /// File kind grouped, then by name. - case kind - - /// The default order applied when the user has expressed no preference. - static let `default`: FilesSortOrder = .name - - /// Sort `entries` by this order, folders-first, with a stable name tiebreak. Pure and total. - func sorted(_ entries: [FileEntry]) -> [FileEntry] { - entries.sorted { lhs, rhs in - // Folders always lead, regardless of the chosen order (Finder-mimic). - if lhs.isDirectory != rhs.isDirectory { return lhs.isDirectory } - switch self { - case .name: - break - case .dateModified: - let l = lhs.modificationDate, r = rhs.modificationDate - if l != r { - // Newer first; a missing date sorts after any real date. - switch (l, r) { - case let (l?, r?): return l > r - case (_?, nil): return true - case (nil, _?): return false - case (nil, nil): break - } - } - case .kind: - if lhs.kind != rhs.kind { return lhs.kind.rawValue < rhs.kind.rawValue } - } - // Tiebreak (and the whole comparison for `.name`): localized, case-insensitive name, then the - // stable path id so equal names never compare equal (keeps the order total / deterministic). - let byName = lhs.name.localizedCaseInsensitiveCompare(rhs.name) - if byName != .orderedSame { return byName == .orderedAscending } - return lhs.id < rhs.id - } - } -} diff --git a/Sources/ThreeFingerSwitcher/Files/FileEntry.swift b/Sources/ThreeFingerSwitcher/Files/FileEntry.swift deleted file mode 100644 index 7c9a2b2..0000000 --- a/Sources/ThreeFingerSwitcher/Files/FileEntry.swift +++ /dev/null @@ -1,103 +0,0 @@ -import Foundation - -/// The Files-band filesystem data model. -/// -/// Like `LaunchItem` and `ClipboardEntry`, everything here is a pure value type with **no AppKit/SwiftUI -/// dependency**, so the navigation / listing / open logic that consumes it stays unit-testable without a -/// running app or a live filesystem. Turning a directory listing into `FileEntry`s (reading resource -/// values off a `URL`) lives in the boundary layer (`DirectoryLister`); choosing a row glyph for a -/// `FileKind` lives in the view layer. -/// -/// A `FileEntry` is **ephemeral**: the Files band is rebuilt from the live filesystem on every launcher -/// open and is **never persisted** into the authored favorites. Like `ClipboardEntry` (which the Files -/// band is modeled on, design D2), ephemerality is enforced at the **persistence boundary** — the -/// `FilesBandBuilder` produces synthetic items and the favorites store never sees them — not by refusing -/// `Codable`: the enclosing `LaunchItemKind.fileEntry` case lives on a `Codable` enum, so the payload must -/// itself be `Codable` for that synthesis to hold, exactly as `ClipboardEntry` is. - -/// A coarse, AppKit-free classification of an entry, just rich enough to pick a row glyph. The boundary -/// layer derives this from the entry's `UTType` (a directory becomes `.folder` regardless of its UTI); -/// the mapping from a `FileKind` to a concrete SF Symbol stays in the view layer so this stays -/// dependency-light. -enum FileKind: String, Codable, Equatable, CaseIterable { - case folder - case image - case audio - case video - case pdf - case archive - case sourceCode - case text - case application - /// Anything not matched above (the neutral document glyph). - case other -} - -/// One filesystem entry in a listed folder: a **stable identity derived from the absolute path** plus the -/// handful of fields the column navigator needs to display and act on it. -/// -/// The id is the file's **standardized absolute path** (not a fresh `UUID`): re-listing the same folder — -/// on re-entry, or because a file changed on disk — yields the **same** id for the same path, so the -/// SwiftUI selection highlight keeps a stable target and never strobes or jumps (design D2). This mirrors -/// how `ClipboardBandBuilder` reuses `ClipboardEntry.id` as the `LaunchItem.id` for stable identity across -/// rebuilds — here the stable value is path-derived rather than capture-assigned, because the same file -/// has no persistent UUID of its own. -struct FileEntry: Codable, Equatable, Identifiable { - /// Stable identity: the entry's standardized absolute path. Two listings of the same path produce - /// equal ids; a downstream `LaunchItem` reuses this so re-listings don't restart the highlight. - let id: String - /// The file URL this entry refers to (a `fileURL`, standardized). - let url: URL - /// Display name shown in the list column (the URL's last path component). - let name: String - /// True for a directory (drives descend vs. open-in-default and the preview target). - let isDirectory: Bool - /// Last content-modification date, when the filesystem reported one (nil if unavailable). - let modificationDate: Date? - /// Coarse type used to choose a row glyph. - let kind: FileKind - - /// The entry's absolute path, derived from `url`. (Equal to `id`.) - var path: String { url.path } - - init(url: URL, name: String, isDirectory: Bool, modificationDate: Date?, kind: FileKind) { - let standardized = url.standardizedFileURL - self.id = standardized.path - self.url = standardized - self.name = name - self.isDirectory = isDirectory - self.modificationDate = modificationDate - self.kind = kind - } - - // MARK: Codable - // - // Required only so the enclosing `Codable` `LaunchItemKind.fileEntry` synthesizes (see the type doc); - // `FileEntry`s are never actually written to the favorites store. `id`/`path` are derived, not stored, - // so decoding rebuilds the value through the standardizing memberwise init — this re-derives `id` - // from `url` and keeps the `id == url.standardizedFileURL.path` invariant rather than trusting a - // separately-encoded id. - private enum CodingKeys: String, CodingKey { - case url, name, isDirectory, modificationDate, kind - } - - init(from decoder: Decoder) throws { - let c = try decoder.container(keyedBy: CodingKeys.self) - self.init( - url: try c.decode(URL.self, forKey: .url), - name: try c.decode(String.self, forKey: .name), - isDirectory: try c.decode(Bool.self, forKey: .isDirectory), - modificationDate: try c.decodeIfPresent(Date.self, forKey: .modificationDate), - kind: try c.decode(FileKind.self, forKey: .kind) - ) - } - - func encode(to encoder: Encoder) throws { - var c = encoder.container(keyedBy: CodingKeys.self) - try c.encode(url, forKey: .url) - try c.encode(name, forKey: .name) - try c.encode(isDirectory, forKey: .isDirectory) - try c.encodeIfPresent(modificationDate, forKey: .modificationDate) - try c.encode(kind, forKey: .kind) - } -} diff --git a/Sources/ThreeFingerSwitcher/Files/FileOpenService.swift b/Sources/ThreeFingerSwitcher/Files/FileOpenService.swift deleted file mode 100644 index ffc74e0..0000000 --- a/Sources/ThreeFingerSwitcher/Files/FileOpenService.swift +++ /dev/null @@ -1,283 +0,0 @@ -import Foundation -import Combine - -/// One Open-With choice for a file: an `AppCandidate` (its `url` + display `name`) plus whether it is the -/// file's **default** application, so the menu can mark the default. -/// -/// `AppCandidate` (defined in `BandsCanvas.swift`) is reused verbatim — it is the Core's existing -/// `{ url, name }` value and already drives the favorites app browser — because it is exactly the shape an -/// Open-With row needs. `AppCandidate` itself carries no default-indication slot (it is a plain installed-app -/// descriptor), so the "default app indicated" the spec requires is added here, alongside it, rather than by -/// widening `AppCandidate`. -struct OpenWithCandidate: Identifiable, Equatable { - /// The application (its bundle URL and display name). - let app: AppCandidate - /// True when this is the file's default application (the one a plain open would launch). - let isDefault: Bool - - /// Stable identity for the list: the app's path (so re-querying the same associations doesn't strobe). - var id: String { app.id } - - static func == (lhs: OpenWithCandidate, rhs: OpenWithCandidate) -> Bool { - lhs.app.url == rhs.app.url && lhs.isDefault == rhs.isDefault - } -} - -/// Opens a highlighted `FileEntry` for real — a file in its default (or a chosen) application, a folder as a -/// Finder window — and enumerates the Open-With candidates for a file. The opened window lands on the -/// **current Space** natively: the open is routed through the `FileWorkspace` seam (whose conformer sets -/// `configuration.activates = true`), NOT through `SpaceWindowMover` (design D9) — so nothing teleports the -/// user to another Space. -/// -/// Modeled on `AICommandExecutor`: `@MainActor` + `ObservableObject`, holding the observable `state` a -/// surface (the column navigator's failure row) binds to, so a failed open is **observable, never a silent -/// false success** (spec: "Failures are observable, never silent"). Every workspace/OS error is mapped into -/// the shared `FileActionError` taxonomy at the `FileWorkspace` boundary, so a `.failed` headline is always a -/// clean per-case sentence — never raw error text. -/// -/// All opens target the **captured front-app context** — the app the user was looking at when the launcher -/// opened — not whichever app is frontmost at the instant of firing (the overlay is non-activating). That -/// context is injected (`activateFrontAppContext`, run just before the open fires) so the service never -/// reaches into AppKit for `frontmostApplication` itself and stays unit-testable against a stub workspace. -@MainActor -final class FileOpenService: ObservableObject { - - /// The service's observable state — the contract the failure surface binds to. Mirrors the AI canvas's - /// state shape: an open is in flight (`.opening`), landed (`.opened`), or surfaced a clean bounded - /// failure (`.failed`); `.idle` is the resting state (no open, or one defused before it fired). - enum State: Equatable { - /// Nothing in flight (also where a defused / discarded open leaves the service — opening nothing). - case idle - /// A committed open is firing (between commit and the workspace returning). - case opening - /// The open actually launched (never reached unless the workspace returned without throwing). - case opened - /// A typed failure with a clean, bounded, user-facing `headline` (never raw error text) plus the - /// opt-in copyable `details` (the raw workspace/OS text captured at the boundary — `nil` when the - /// headline already says everything), so a surface can offer a "Show details / Copy" disclosure - /// without ever putting raw text in the headline. The user can retry or discard from this state - /// (spec: "A failed open surfaces a clean, bounded message"). - case failed(headline: String, details: String?) - } - - @Published private(set) var state: State = .idle - - /// The system-workspace seam (open / Open-With / app association). Injected so tests drive a stub that - /// records calls and simulates failures, and so Core stays AppKit-free; production injects - /// `SystemFileWorkspace` (which wraps `NSWorkspace` and maps errors to `FileActionError` at the boundary). - private let workspace: FileWorkspace - /// The captured front-app context: a side effect the service runs **before** firing an open so the open - /// targets the app the user was looking at, never the frontmost app at fire time. Returns to the resting - /// app context; no-op by default / in tests. Modeled on `LaunchService`/`SelectionService`'s injected - /// `frontAppProvider` — kept a bare closure here so the service (and its Core taxonomy) stays AppKit-free. - private let activateFrontAppContext: () -> Void - - /// The pending open awaiting its commit/cancel (the defusable held state). Retained so a `cancel()` - /// (discard) can defuse it before its fuse fires; cleared after it commits, cancels, or its fuse lands. - private(set) var pendingOpen: PendingOpen? - - init(workspace: FileWorkspace, - activateFrontAppContext: @escaping () -> Void = {}) { - self.workspace = workspace - self.activateFrontAppContext = activateFrontAppContext - } - - // MARK: - Open-With enumeration - - /// The applications that can open **this file**, in the system's order, with the file's default - /// application indicated. Derived on demand from the workspace's association of apps to the file - /// (`urlsForApplications(toOpen:)`); the default is `urlForApplication(toOpen:)`. Empty when no installed - /// application handles the file (the caller then has nothing to offer / surfaces "no app"). - /// - /// Open-With is a **file** action only: a folder's default open is a Finder window, so this returns empty - /// for a directory entry (the navigator does not offer Open-With for folders). - func openWithCandidates(for file: FileEntry) -> [OpenWithCandidate] { - guard !file.isDirectory else { return [] } - let defaultApp = workspace.urlForApplication(toOpen: file.url) - return workspace.urlsForApplications(toOpen: file.url).map { appURL in - OpenWithCandidate(app: AppCandidate(url: appURL), - isDefault: appURL == defaultApp) - } - } - - /// Surface the "no installed application can open this file" outcome as observable bounded state: an - /// Open-With with an EMPTY candidate list has nothing to pick, so rather than silently doing nothing the - /// service transitions to `.failed` carrying the clean `FileActionError.noApplicationForFile` headline - /// (spec: "Failures are observable, never silent"). Mapped into the taxonomy here, at the service - /// boundary, so the headline is a clean per-file sentence — never raw error text. The surface (the - /// navigator's failure row) binds `state`; it is bounded + non-blocking, never an app-modal alert. - func surfaceNoApplication(for file: FileEntry) { - state = Self.failure(for: FileActionError.noApplicationForFile(name: file.name), fallbackName: file.name) - } - - // MARK: - Defusable opens (prepare → commit / cancel) - - /// Prepare a **default** open of `entry` (a file in its default app, a folder as a Finder window) as a - /// defusable `PendingOpen`: the open does NOT fire here — it fires on the returned pending's `commit()` - /// (optionally after a short fuse), and a `cancel()` before then opens nothing. A new prepare supersedes - /// any still-pending one (it is cancelled first), so only one open is ever in flight. - @discardableResult - func prepareOpen(_ entry: FileEntry) -> PendingOpen { - prepare { [weak self] in - await self?.performOpenDefault(entry) - } - } - - /// Prepare an **Open-With** of `file` using the application at `appURL`, as a defusable `PendingOpen` - /// (same held → commit/cancel lifecycle as `prepareOpen`). - @discardableResult - func prepareOpenWith(_ file: FileEntry, appURL: URL) -> PendingOpen { - prepare { [weak self] in - await self?.performOpenWith(file, appURL: appURL) - } - } - - /// Cancel (defuse) the current pending open, if any: nothing opens, and the service returns to `.idle`. - /// Safe to call when there is no pending open (a stray discard is a no-op) and **never terminates an - /// already-running application** — it only defuses a not-yet-fired open (spec: "Defusable open"). - func cancelPending() { - pendingOpen?.cancel() - } - - /// Build a `PendingOpen` around `fire` (the deferred open effect), wiring the held → commit/cancel - /// lifecycle: committing runs the captured-front-app activation, transitions to `.opening`, then runs the - /// effect; cancelling defuses (opening nothing) and rests at `.idle`. The pending reference is cleared in - /// both terminal paths so a later discard can't re-fire it. - private func prepare(_ fire: @escaping () async -> Void) -> PendingOpen { - pendingOpen?.cancel() // a new prepare supersedes any still-pending open - let pending = PendingOpen( - onCommit: { [weak self] in - guard let self else { return } - self.pendingOpen = nil - // Target the captured front-app context (not the frontmost app at fire time) before opening. - self.activateFrontAppContext() - self.state = .opening - await fire() - }, - onCancel: { [weak self] in - guard let self else { return } - self.pendingOpen = nil - self.state = .idle // defused: nothing opened, never kills a running app - } - ) - pendingOpen = pending - state = .idle // held: visible/resting until the commit (down-swipe) or discard - return pending - } - - // MARK: - Open effects (workspace boundary) - - /// Fire a default open of `entry` through the workspace, surfacing the outcome: a clean bounded `.failed` - /// (its `FileActionError` headline) if the open did not launch, `.opened` only when it actually did. The - /// workspace already lands the window on the current Space and maps any OS error to `FileActionError`. - private func performOpenDefault(_ entry: FileEntry) async { - do { - try await workspace.open(entry.url) - state = .opened - } catch { - state = Self.failure(for: error, fallbackName: entry.name) - } - } - - /// Fire an Open-With of `file` using the app at `appURL` through the workspace, with the same - /// outcome surfacing as `performOpenDefault`. - private func performOpenWith(_ file: FileEntry, appURL: URL) async { - do { - try await workspace.open(file.url, withApplicationAt: appURL) - state = .opened - } catch { - state = Self.failure(for: error, fallbackName: file.name) - } - } - - // MARK: - Messaging - - /// The `.failed` State for an open failure: a clean, user-facing **headline** plus the opt-in copyable - /// **details** (the raw text, surfaced only behind a "Show details / Copy" disclosure). A `FileActionError` - /// (mapped at the workspace boundary) is self-describing via `LocalizedError`, so its `errorDescription` - /// is the headline directly and its `copyableDetails` carries the raw OS/workspace text; any other error - /// never reaches a headline — it falls back to a clean, file-named sentence with no details (the raw text - /// is confined to the boundary's logs, never interpolated into the headline — spec: "No raw error text in - /// user-facing strings"). - private static func failure(for error: Error, fallbackName: String) -> State { - if let fileError = error as? FileActionError, let description = fileError.errorDescription { - return .failed(headline: description, details: fileError.copyableDetails) - } - // The boundary maps every workspace error into `FileActionError`, so this is a defensive floor: a - // clean per-file sentence (no details), never a reflected dump of `error`. - return .failed(headline: FileActionError.openFailed(name: fallbackName, details: nil).errorDescription ?? "", - details: nil) - } -} - -/// A held, **defusable** open: the open effect is captured but not yet fired, so a discard issued before it -/// fires opens nothing (spec: "Defusable open"). Modeled on the AI canvas's held → commit/cancel lifecycle -/// (`AICommandExecutor`'s `isCommittable` / `commit` / `cancel`): a four-finger DOWN swipe `commit()`s it, a -/// horizontal discard `cancel()`s it, and a stray re-lift is a no-op (the firing lift already resolved it). -/// -/// `commit()` optionally arms a short **fuse** (a retained `Task` you can cancel) before the effect fires, so -/// a discard within the fuse window still defuses it; once the effect has actually run there is nothing to -/// defuse — and defusing **never terminates an already-running application**, it only prevents a not-yet-fired -/// open. The pending is **one-shot**: after it commits or cancels, every later `commit()`/`cancel()` is a -/// no-op, so a stray re-lift cannot double-open or cancel a window that is already up. -@MainActor -final class PendingOpen { - /// The resolution lifecycle. `held` is the live state a down-swipe commits / a horizontal swipe cancels; - /// `fusing` is committed-but-not-yet-fired (the fuse is counting down and can still be defused by a - /// discard); `committed` / `cancelled` are terminal. Mirrors the canvas's "resolve once" rule. - private enum Phase { case held, fusing, committed, cancelled } - private var phase: Phase = .held - - /// Whether a DOWN-swipe commit would still COMMIT (the open is held and not yet resolved). Mirrors - /// `AICommandExecutor.State.isCommittable`: false once committed/cancelled (or while a fuse is mid-flight), - /// which is what makes a stray re-lift a no-op. - var isCommittable: Bool { phase == .held } - - /// Whether the open can still be **defused** by a discard: while held, or while its fuse is counting down - /// (committed but not yet fired). Once the effect has actually fired there is nothing left to defuse. - var isDefusable: Bool { phase == .held || phase == .fusing } - - /// Fired (once) when the open commits — runs the actual workspace open via the service. - private let onCommit: () async -> Void - /// Fired (once) when the open is cancelled/defused — resets the service, opening nothing. - private let onCancel: () -> Void - /// The retained fuse, when `commit(afterFuse:)` armed one. Cancelling it (via `cancel()`) before it fires - /// defuses the open. `nil` when the commit fired the effect immediately. - private var fuse: Task? - - init(onCommit: @escaping () async -> Void, onCancel: @escaping () -> Void) { - self.onCommit = onCommit - self.onCancel = onCancel - } - - /// Commit the open: fire it now (or, if `afterFuse` > 0, after that delay via a retained, cancellable - /// `Task` — so a discard within the window still defuses it). One-shot: a second `commit()` (or a - /// `commit()` after a `cancel()`) is ignored, so a stray re-lift never double-opens. - func commit(afterFuse fuse: Duration = .zero) { - guard phase == .held else { return } - guard fuse > .zero else { - phase = .committed - Task { await onCommit() } - return - } - // Armed but not fired: stays defusable (a discard within the window cancels the fuse → opens nothing). - phase = .fusing - self.fuse = Task { [weak self] in - try? await Task.sleep(for: fuse) - guard let self, !Task.isCancelled, self.phase == .fusing else { return } - self.phase = .committed - await self.onCommit() - } - } - - /// Cancel (defuse) the open: cancel any armed fuse and open nothing. One-shot and idempotent — a - /// discard after the open already fired (or was already cancelled) is a no-op, and it **never** terminates - /// an already-running app (there is no terminate path here at all — only a not-yet-fired open is prevented). - func cancel() { - guard isDefusable else { return } // a discard after the open already fired is a no-op - phase = .cancelled - fuse?.cancel() - fuse = nil - onCancel() - } -} diff --git a/Sources/ThreeFingerSwitcher/Files/FileWorkspace.swift b/Sources/ThreeFingerSwitcher/Files/FileWorkspace.swift deleted file mode 100644 index e33ab28..0000000 --- a/Sources/ThreeFingerSwitcher/Files/FileWorkspace.swift +++ /dev/null @@ -1,145 +0,0 @@ -import Foundation - -/// The Files band's single seam onto the system workspace (open / Open-With / app association). -/// -/// Mirrors the `LLMRuntime` seam idea: the open/Open-With logic depends ONLY on this protocol — never on -/// `NSWorkspace` directly — so it is unit-testable against a stub that records calls and simulates -/// failures. The protocol is **dependency-light** (Foundation only, no AppKit) so a test stub conforms -/// without importing AppKit; the real `SystemFileWorkspace` conformer wraps `NSWorkspace` and lives behind -/// `#if canImport(AppKit)`. -/// -/// The open operations are `async throws`: a conformer maps any underlying workspace/OS failure into the -/// shared `FileActionError` taxonomy **at this boundary** (so callers only ever see `FileActionError`, never -/// a raw `NSError`), and a non-throwing return means the open actually launched (never a false success). -/// The two association queries are synchronous and non-throwing — they only read the system's app -/// associations and naturally yield an empty list / `nil` when nothing handles the file. -protocol FileWorkspace { - /// Open `url` in its default application. Throws a `FileActionError` if the open did not launch. - func open(_ url: URL) async throws - - /// Open `url` with the application at `applicationURL`. Throws a `FileActionError` on failure. - func open(_ url: URL, withApplicationAt applicationURL: URL) async throws - - /// The applications capable of opening `url`, in the system's order (default app first). Empty when no - /// installed application handles the file. - func urlsForApplications(toOpen url: URL) -> [URL] - - /// The default application for `url`, or `nil` when the system has no association for it. - func urlForApplication(toOpen url: URL) -> URL? -} - -#if canImport(AppKit) -import AppKit - -/// The production `FileWorkspace`, wrapping `NSWorkspace`. Uses the modern async `open(_:configuration:)` -/// / `open(_:withApplicationAt:configuration:)` (so a launch failure is an awaited `throw`, mapped to the -/// taxonomy here) and `urlsForApplications(toOpen:)` / `urlForApplication(toOpen:)` for the associations. -/// -/// `configuration.activates = true` so the opened window comes to the front on the **current** Space — the -/// new window lands natively where the user is; the Files band deliberately does NOT route opens through -/// `SpaceWindowMover` (design D9). -struct SystemFileWorkspace: FileWorkspace { - init() {} - - func open(_ url: URL) async throws { - let config = NSWorkspace.OpenConfiguration() - config.activates = true - do { - _ = try await NSWorkspace.shared.open(url, configuration: config) - } catch { - // Map at the boundary: a raw NSWorkspace error never escapes into feature/UI code. The raw - // text is stringified into the opt-in `details` here (and is the only place it survives). - throw FileActionError.openFailed(name: url.lastPathComponent, - details: String(describing: error)) - } - } - - func open(_ url: URL, withApplicationAt applicationURL: URL) async throws { - let config = NSWorkspace.OpenConfiguration() - config.activates = true - do { - _ = try await NSWorkspace.shared.open([url], withApplicationAt: applicationURL, - configuration: config) - } catch { - throw FileActionError.openFailed(name: url.lastPathComponent, - details: String(describing: error)) - } - } - - func urlsForApplications(toOpen url: URL) -> [URL] { - NSWorkspace.shared.urlsForApplications(toOpen: url) - } - - func urlForApplication(toOpen url: URL) -> URL? { - NSWorkspace.shared.urlForApplication(toOpen: url) - } -} -#endif - -/// Failures the Files band's side effects (directory listing, open, Open-With) can report. -/// -/// This is the Files-domain parallel to the AI feature's `RuntimeError`: a small Core taxonomy conforming -/// to `LocalizedError`, with a clean, per-case, user-facing string for every case — so a failure surfaces -/// as a bounded headline that reads the same everywhere. The AI `RuntimeError` deliberately is NOT reused: -/// its cases (`modelMissing`, `integrityFailed`, `unsupportedModality`…) are model-layer concepts with no -/// meaning for a filesystem action, and folding file failures into it would muddy both taxonomies. -/// -/// **Map at the boundary:** `FileManager` errors (listing) and `NSWorkspace`/OS errors (open) are converted -/// into these cases where they cross into app code (`DirectoryLister`, `SystemFileWorkspace.open`), so Core -/// stays free of vendor/OS error types. The raw error is **stringified into the opt-in `details` payload at -/// that boundary** — it is kept ONLY for an opt-in "Show details / Copy" disclosure and logs, and is NEVER -/// used as the headline (`errorDescription`). Carrying `details` as a `String?` (rather than a raw `Error`) -/// keeps the enum `Equatable` exactly like `RuntimeError.modelLoadFailed(detail:)`. -enum FileActionError: Error, Equatable { - /// A folder's contents could not be read (e.g. permission denied, or it was removed). `name` is the - /// folder's display name; `details` is opt-in copyable text (the raw `FileManager`/OS error, off the - /// headline) — surfaced only as a disclosure / in logs. - case folderUnreadable(name: String, details: String?) - /// Opening a file or folder did not launch (e.g. the item was removed, or the app failed to start). - /// `name` is the item's display name; `details` is opt-in copyable text (the raw workspace/OS error). - case openFailed(name: String, details: String?) - /// No installed application can open this file, so there is nothing to open it with. - case noApplicationForFile(name: String) - /// A Paste-into copy could not complete (e.g. permission denied, the destination is read-only, or the - /// source was removed). `name` is the destination folder's display name; `details` is opt-in copyable - /// text (the raw `FileManager`/OS error). The band's single mutating op (`files-action-menu`). - case pasteFailed(name: String, details: String?) - /// A Delete (move-to-Trash) could not complete (e.g. permission denied, or the item was already removed). - /// `name` is the entry's display name; `details` is opt-in copyable text (the raw `FileManager`/OS error). - /// The band only ever **trashes** (recoverable) — there is no permanent-delete error because there is no - /// permanent delete. - case trashFailed(name: String, details: String?) -} - -/// Self-describing, user-facing messages for every case — clean per-case sentences, so the "clean path" -/// (reading `errorDescription`) never falls back to a reflected enum dump or raw OS text. These are the -/// canonical headlines a surfacing translator returns; raw error text appears only in `copyableDetails` -/// (→ opt-in disclosure) and logs, never here (spec: "No raw error text in user-facing strings"). -extension FileActionError: LocalizedError { - var errorDescription: String? { - switch self { - case let .folderUnreadable(name, _): - return "Couldn't read “\(name)”. You may not have permission, or it was moved." - case let .openFailed(name, _): - return "Couldn't open “\(name)”. It may have been moved or removed." - case let .noApplicationForFile(name): - return "No app on this Mac can open “\(name)”." - case let .pasteFailed(name, _): - return "Couldn't paste into “\(name)”. You may not have permission, or it was moved." - case let .trashFailed(name, _): - return "Couldn't move “\(name)” to the Trash. You may not have permission, or it was already removed." - } - } - - /// The opt-in copyable detail (the raw error text captured at the boundary), for a "Show details / Copy" - /// disclosure and logs only. `nil` when the headline already says everything (e.g. no app for the file). - var copyableDetails: String? { - switch self { - case let .folderUnreadable(_, details): return details - case let .openFailed(_, details): return details - case .noApplicationForFile: return nil - case let .pasteFailed(_, details): return details - case let .trashFailed(_, details): return details - } - } -} diff --git a/Sources/ThreeFingerSwitcher/Files/FilesActionMenu.swift b/Sources/ThreeFingerSwitcher/Files/FilesActionMenu.swift deleted file mode 100644 index 5ea3a08..0000000 --- a/Sources/ThreeFingerSwitcher/Files/FilesActionMenu.swift +++ /dev/null @@ -1,157 +0,0 @@ -import Foundation - -// MARK: - Files action-menu model (pure, MLX-free Core) - -/// One action offered by the Files-band **action menu** (`files-action-menu`) — summoned by the menu -/// excursion (default the `+1`-finger lift) over a highlighted file or folder. A pure catalog enum; the -/// concrete effect runs at the boundary (`AppCoordinator`), the row label/glyph in the view layer. -/// -/// Deliberately distinct from `GestureBindings.FilesAction` (the drill's `{open, openWith, discard}` -/// *resolution* vocabulary): this is the **menu item** catalog, several of which a single drill excursion -/// (the menu trigger) opens. -enum FilesMenuAction: String, Codable, CaseIterable, Identifiable, Sendable { - /// Copy the entry's absolute path into the clipboard history + live pasteboard. - case copyAsPath - /// Copy the entry's file/folder object (the `fileURL`) to the pasteboard. - case copy - /// Mark the entry for a MOVE: writes its `fileURL` to the pasteboard and records the cut, so the next - /// `pasteInto` (while the pasteboard is still this cut) MOVES it instead of copying (Finder ⌘X). - case cut - /// Paste the pasteboard's file(s) INTO the folder (for a file: its containing folder) — **dual-mode**: - /// a MOVE when fulfilling a `cut`, else a COPY; keep-both on conflict either way (never overwrites). - case pasteInto - /// The app grid (Open-With generalized): capable apps (file) / folder-openers (folder). - case openIn - /// Expands to one row per enabled terminal — opens the folder as that terminal's working directory. - case openInTerminals - /// Opt-in. Expands to one row per enabled editor — opens the folder in that editor. - case openInEditor - /// Opt-in. Reveal the entry selected in a Finder window. - case revealInFinder - /// Opt-in. Pin the entry into the launcher as a favorite. - case addToFavorites - /// Opt-in. Copy just the entry's display name (last path component). - case copyName - /// Move the entry to the **Trash** (recoverable from Finder) — never a permanent delete. - case delete - - public var id: String { rawValue } - - /// The actions that are **defaults** (shown unless the user customizes); the rest are opt-in extras the - /// user may add. Used by the Hub editor to present "available to add" vs. "in the menu." - static let defaultCatalog: [FilesMenuAction] = [.copyAsPath, .copy, .cut, .pasteInto, .openInTerminals, .openIn, .delete] - static let extras: [FilesMenuAction] = [.openInEditor, .revealInFinder, .addToFavorites, .copyName] -} - -/// An external tool (terminal or editor) that can open a folder as its working directory. Auto-detected at -/// the boundary (a bundle-id probe); the pure menu model only needs its identity + display name + role to -/// build rows. `enabled` reflects the user's curation (the allow-list in `tunable-settings`). -struct FilesTool: Codable, Equatable, Identifiable, Sendable { - enum Role: String, Codable, Sendable { case terminal, editor } - let bundleID: String - let name: String - let role: Role - var enabled: Bool - - var id: String { bundleID } - - init(bundleID: String, name: String, role: Role, enabled: Bool = true) { - self.bundleID = bundleID - self.name = name - self.role = role - self.enabled = enabled - } -} - -/// One concrete, rendered row of the action menu after expanding groups and applying visibility rules — the -/// view iterates these, the controller dispatches on the committed one. `tool` carries which catalog action -/// produced it (terminals vs. editors) so the boundary opens with the right role. -enum FilesMenuRow: Identifiable, Equatable { - /// A plain action row (everything except the tool groups). - case action(FilesMenuAction) - /// A terminal/editor row produced by `openInTerminals` / `openInEditor`. - case tool(FilesMenuAction, FilesTool) - - var id: String { - switch self { - case let .action(a): return "action:\(a.rawValue)" - case let .tool(a, tool): return "tool:\(a.rawValue):\(tool.bundleID)" - } - } -} - -/// The user-configurable Files action-menu contents, per entry type (`files-action-menu`, -/// `tunable-settings`). Pure value; persisted via `AppSettings`. Defaults are **exactly** the user's -/// specified menus; any deviation is a customization. -struct FilesActionMenu: Codable, Equatable, Sendable { - /// Ordered catalog items shown for a highlighted **file**. - var fileItems: [FilesMenuAction] - /// Ordered catalog items shown for a highlighted **folder**. - var folderItems: [FilesMenuAction] - - init(fileItems: [FilesMenuAction] = FilesActionMenu.defaultFileItems, - folderItems: [FilesMenuAction] = FilesActionMenu.defaultFolderItems) { - self.fileItems = fileItems - self.folderItems = folderItems - } - - /// File default: Copy as path · Copy · Cut · Paste · Open in ▸ · Delete (Delete last, set apart). - static let defaultFileItems: [FilesMenuAction] = [.copyAsPath, .copy, .cut, .pasteInto, .openIn, .delete] - /// Folder default: Copy as path · Copy · Cut · Paste · ‹terminals› · Open in ▸ · Delete. - static let defaultFolderItems: [FilesMenuAction] = [.copyAsPath, .copy, .cut, .pasteInto, .openInTerminals, .openIn, .delete] - /// Both menus at their specified defaults. - static let `default` = FilesActionMenu() - - /// The configured catalog order for a given entry type. - func items(forFolder: Bool) -> [FilesMenuAction] { forFolder ? folderItems : fileItems } - - /// Resolve the configured catalog into the concrete, ordered rows for `entry`, applying runtime context: - /// whether the live pasteboard currently holds a file reference (gates `pasteInto`) and the enabled - /// tools (the `openInTerminals` / `openInEditor` groups expand to one row per enabled tool, and vanish - /// when there are none). Pure → unit-tested. - func visibleRows(for entry: FileEntry, - pasteboardHasFile: Bool, - terminals: [FilesTool], - editors: [FilesTool]) -> [FilesMenuRow] { - items(forFolder: entry.isDirectory).flatMap { action -> [FilesMenuRow] in - switch action { - case .pasteInto: - return pasteboardHasFile ? [.action(.pasteInto)] : [] - case .openInTerminals: - return terminals.filter(\.enabled).map { .tool(.openInTerminals, $0) } - case .openInEditor: - return editors.filter(\.enabled).map { .tool(.openInEditor, $0) } - default: - return [.action(action)] - } - } - } -} - -// MARK: - Paste-into name resolution (pure) - -/// Pure **keep-both** name resolver for the action menu's Paste-into (`files-action-menu`). Given a desired -/// file name and the set of names already present in the destination folder, return a name that does not -/// collide — the original if free, else `"name copy"`, `"name copy 2"`, … (Finder's convention), preserving -/// the file extension. Its whole job is to pick a **fresh** name so the incoming copy keeps both and the -/// existing item is never overwritten. -enum FilesPasteName { - static func uniqueName(for desired: String, existing: Set) -> String { - guard existing.contains(desired) else { return desired } - - let ns = desired as NSString - let ext = ns.pathExtension - let base = ns.deletingPathExtension - func compose(_ stem: String) -> String { ext.isEmpty ? stem : "\(stem).\(ext)" } - - // Finder's first duplicate is " copy", then " copy 2", " copy 3", … - let firstCopy = compose("\(base) copy") - if !existing.contains(firstCopy) { return firstCopy } - var n = 2 - while true { - let candidate = compose("\(base) copy \(n)") - if !existing.contains(candidate) { return candidate } - n += 1 - } - } -} diff --git a/Sources/ThreeFingerSwitcher/Files/FilesBandBuilder.swift b/Sources/ThreeFingerSwitcher/Files/FilesBandBuilder.swift deleted file mode 100644 index b512401..0000000 --- a/Sources/ThreeFingerSwitcher/Files/FilesBandBuilder.swift +++ /dev/null @@ -1,92 +0,0 @@ -import Foundation -import CryptoKit - -/// Builds the **synthetic, ephemeral** Files band shown as a launcher band alongside the authored -/// favorites and the synthetic Clipboard band. The band is projected fresh on every launcher open from -/// the *current* directory column (the navigator's `visibleEntries`); it is never written into the -/// persisted `Favorites` record and is never the home band — exactly like `ClipboardBandBuilder`. -/// -/// Each `FileEntry` becomes a `LaunchItem` whose kind is `.fileEntry`, so the band flows through the -/// existing `LauncherModel` plumbing as data. (A chosen entry is **not** resolved through the generic -/// `LaunchService.fire` — see that case's no-op — but through the Files band's own drill-down / open -/// path: folders descend in place, files open via `FileOpenService`. That path is owned by the Files -/// band's dedicated views/coordination, not this builder.) -/// -/// **Stable identity (design D2).** A `LaunchItem.id` is a `UUID`, but a `FileEntry.id` is its -/// standardized absolute *path* `String`. Re-listing the same folder must yield the **same** item id -/// for the same path, or the SwiftUI selection highlight strobes/jumps on every re-list. We therefore -/// derive a **deterministic** UUID from the path (`uuid(forPath:)`) rather than minting a fresh one — -/// mirroring how `ClipboardBandBuilder` reuses `ClipboardEntry.id` as the `LaunchItem.id`, except the -/// stable value here is path-derived because a file has no persistent UUID of its own. -enum FilesBandBuilder { - /// Sentinel band id so the overlay can recognize the Files band among the launcher's bands - /// (distinct from `ClipboardBandBuilder.bandID` and `AIBand.bandID`). "F11E5" ≈ "FILES". - static let bandID = UUID(uuidString: "F11E5000-0000-4000-8000-000000000001")! - static let name = "Files" - /// The Files band's accent tint as a default, AppKit-free `ItemColor` matching the calm blue - /// `AppSettings.Defaults.filesBandTint` (`#3B82C4`). The user-configurable `filesBandTint` hex is - /// resolved to a `Color`/`ItemColor` at the view/overlay boundary (a later wiring stage); this is the - /// neutral fallback used when the builder runs without that context. `#3B82C4` = (59, 130, 196)/255. - static let color = ItemColor(red: 0.231, green: 0.510, blue: 0.769) - /// The Files band's dedicated, preset launcher icon (not user-editable — it's a synthetic band). - static let icon: ItemIcon = .sfSymbol("folder.fill") - - /// Build the Files band from the current directory column's entries (already sorted by the - /// navigator). Each entry maps to a `.fileEntry` `LaunchItem` with a path-stable id and a - /// `FileKind`-derived glyph; the band itself carries the synthetic sentinel id. - static func build(currentColumn entries: [FileEntry]) -> ContextBand { - let items = entries.map { item(for: $0) } - return ContextBand(id: bandID, name: name, color: color, icon: icon, items: items) - } - - /// One `.fileEntry` band item for a filesystem entry. The item id is derived deterministically from - /// the entry's path (`uuid(forPath:)`) so re-listings keep a stable SwiftUI identity; the title is - /// the entry's display name and the icon is a `FileKind`-derived SF Symbol (`glyph(for:)`). - static func item(for entry: FileEntry) -> LaunchItem { - LaunchItem(id: uuid(forPath: entry.id), title: entry.name, - icon: glyph(for: entry.kind), kind: .fileEntry(entry)) - } - - /// True for a band produced by this builder (matched by the sentinel id). - static func isFilesBand(_ band: ContextBand) -> Bool { band.id == bandID } - - // MARK: - Stable path → UUID - - /// Derive a **stable, deterministic** `UUID` from a file path (the entry's standardized absolute - /// path / `FileEntry.id`). The same path always produces the same UUID, so a `.fileEntry` item keeps - /// its SwiftUI identity across re-lists (no highlight strobe, design D2); different paths effectively - /// never collide (128-bit SHA-256 prefix). Implemented as the first 16 bytes of `SHA256(path)` — - /// CryptoKit is already a Core dependency (see `ModelManager`). The version/variant bits are left as - /// the hash produced them (this is an internal identity token, not an RFC-4122-typed UUID), exactly - /// the way other stable identities are derived from content here. - static func uuid(forPath path: String) -> UUID { - let digest = SHA256.hash(data: Data(path.utf8)) - var bytes = [UInt8](digest) // 32 bytes; take the first 16 for the UUID - bytes.removeLast(bytes.count - 16) - return UUID(uuid: (bytes[0], bytes[1], bytes[2], bytes[3], - bytes[4], bytes[5], bytes[6], bytes[7], - bytes[8], bytes[9], bytes[10], bytes[11], - bytes[12], bytes[13], bytes[14], bytes[15])) - } - - // MARK: - Glyphs - - /// The SF Symbol standing in for a `FileKind`, used as the `.fileEntry` item's launcher icon (the - /// view's `.sfSymbol` branch renders it). Mirrors `ClipboardBandBuilder.glyph(for:)`. The Files - /// band's own column/preview views may show richer, real file icons; this is the AppKit-free, - /// kind-coarse fallback carried on the model item. - static func glyph(for kind: FileKind) -> ItemIcon { - switch kind { - case .folder: return .sfSymbol("folder.fill") - case .image: return .sfSymbol("photo") - case .audio: return .sfSymbol("music.note") - case .video: return .sfSymbol("film") - case .pdf: return .sfSymbol("doc.richtext") - case .archive: return .sfSymbol("doc.zipper") - case .sourceCode: return .sfSymbol("chevron.left.forwardslash.chevron.right") - case .text: return .sfSymbol("doc.text") - case .application: return .sfSymbol("app") - case .other: return .sfSymbol("doc") - } - } -} diff --git a/Sources/ThreeFingerSwitcher/Files/FilesColumnController.swift b/Sources/ThreeFingerSwitcher/Files/FilesColumnController.swift deleted file mode 100644 index b53a489..0000000 --- a/Sources/ThreeFingerSwitcher/Files/FilesColumnController.swift +++ /dev/null @@ -1,307 +0,0 @@ -import Foundation - -/// Bridges the Files band's **pure, synchronous** `FilesNavigationModel` to the **async, off-main** -/// `DirectoryLister` (design D6 / "the key seam"). The navigation model only learns a folder's contents -/// through a *synchronous* `(URL) -> [FileEntry]` closure, but the real listing crosses the `FileManager` -/// boundary asynchronously; this controller resolves that mismatch with a listing CACHE it owns: -/// -/// - The synchronous lister handed to `FilesNavigationModel` reads `cache[folderPath] ?? []` — never the -/// filesystem. So the model stays pure and instantaneous; a not-yet-listed folder simply shows empty for -/// one beat. -/// - On a cache **miss** (the model descended / ascended into a folder we haven't listed) the controller -/// kicks off `DirectoryLister.contents(of:sortedBy:)` off-main. When it returns, the result is stored in -/// the cache, the model's *current* column is re-fed (`navigation.reloadCurrentColumn()`, which re-reads -/// through the same now-warm lister and preserves the highlight), and `onColumnChanged` fires so the -/// owner (`LauncherModel`) rebuilds the band's items and republishes — the view updates with the real -/// contents. -/// - The preview's folder-peek (`FilesNavigationModel.previewTarget`'s `.folder(_, contents:)`) flows -/// through the **same** cache, so a render never triggers a fresh `FileManager` read; an un-cached peek -/// warms the cache (and refreshes) exactly like a column miss. -/// -/// `@MainActor` because it mutates the model the main-thread view reads and hops back onto the actor to -/// store listings; the only thing it does off the actor is the `DirectoryLister` read itself, which is -/// `nonisolated` and pure-Foundation. -@MainActor -final class FilesColumnController { - - // MARK: - Injected - - /// Lists a folder's local contents off-main, already sorted by the supplied order (the live - /// `DirectoryLister`; a test injects a synchronous fixture wrapped to look async). The descending - /// reversal is applied here, around the lister, so the cached column is in its final display order. - private let lister: (URL, FilesSortOrder) async -> [FileEntry] - - /// The order to list folders in, recomputed from settings (`filesSortField`) each launcher open. A - /// `var` so a settings change between opens is honoured on the next listing without rebuilding the - /// controller. The ascending/descending *direction* rides separately in `sortDirection`. - var sortOrder: FilesSortOrder - - /// Ascending vs. descending, applied (folders-first) on top of `sortOrder` after a listing returns — - /// `FilesSortOrder` itself carries no direction (see `applyingDirection(_:to:)`). - var sortDirection: FilesSortDirection - - /// Whether each (re)build of the navigation model opens displaying the remembered deepest folder rather - /// than the roots list (the Hub "restore last folder" toggle; the owner passes - /// `AppSettings.filesRememberLocation`). A stored property so `reset(roots:remembered:)` honours the same - /// landing on a roots change. The deepest-location tracking is unaffected by this — it only governs the - /// initial landing (see `FilesNavigationModel.init`). - var restoreLastLocation: Bool - - /// Fired (on the main actor) whenever the current column's *visible contents* change — an async listing - /// landed and was re-fed into the model. The owner rebuilds the Files band's `items` and republishes. - var onColumnChanged: () -> Void = {} - - // MARK: - State - - /// The pure column-navigation state machine. Rebuilt by `reset(roots:remembered:)`; mutated in place by - /// `descend()` / `ascend()` / `highlightUp()` / `highlightDown()`. Read-only to the owner via the - /// forwarding accessors below so the cache-feeding invariants live solely here. - private(set) var navigation: FilesNavigationModel - - /// The listing cache: standardized folder path → its listed entries (in final display order). The single - /// source the synchronous lister (and the preview folder-peek) read from; populated only by completed - /// async listings. - private(set) var cache: [String: [FileEntry]] = [:] - - /// Folders an async listing is already in flight for (by standardized path), so a burst of misses for - /// the same folder (a column feed + a preview peek in the same beat) coalesces to one read. - private var inFlight: Set = [] - - /// The currently-spawned listing tasks, retained so callers (the initial-warm path, and tests) can - /// `await settle()` until every in-flight listing has stored and re-fed. Drained as each task finishes. - private var listingTasks: [Task] = [] - - // MARK: - Init - - /// Build the controller over `roots`, restoring `remembered` per-root locations. The navigation model is - /// wired to a synchronous lister that reads *this* controller's cache, so the model never blocks; the - /// landing column (and any restored remembered depth) is then warmed asynchronously. - /// - /// - Parameters: - /// - roots: the configured local root folders (the entry column). - /// - remembered: previously-persisted `root → deepest folder` map. - /// - sortOrder: the listing key, mapped from settings by the owner. - /// - sortDirection: ascending/descending, applied folders-first after listing. - /// - restoreLastLocation: open displaying the remembered deepest folder (the owner passes - /// `AppSettings.filesRememberLocation`) rather than the roots list. Threaded into the navigation model - /// so the band OPENS on that folder; the restored column (and its ancestors) is warmed below so the - /// first frame isn't empty. - /// - seededCache: folders already listed (standardized path → entries) to pre-warm the cache, so the - /// landing column (and tests over a fixed tree) is populated synchronously with no listing round-trip. - /// - lister: the async directory lister (defaults to the live `DirectoryLister`). - init(roots: [URL], - remembered: [URL: URL], - sortOrder: FilesSortOrder, - sortDirection: FilesSortDirection, - restoreLastLocation: Bool = false, - seededCache: [String: [FileEntry]] = [:], - lister: @escaping (URL, FilesSortOrder) async -> [FileEntry] = FilesColumnController.systemLister) { - self.lister = lister - self.sortOrder = sortOrder - self.sortDirection = sortDirection - self.restoreLastLocation = restoreLastLocation - self.cache = seededCache - // `navigation` must be set before `self` is captured, so seed it with a roots-only model, then rebind - // to the live cache (the rebind re-creates it with the same inputs but a `self`-capturing lister that - // reads the now-seeded cache). Any folder not in `seededCache` lists empty until `warm…` fills it. - self.navigation = FilesNavigationModel(roots: roots, remembered: remembered, lister: { _ in [] }) - rebindNavigationToLiveCache(roots: roots, remembered: remembered) - warmCurrentColumnIfNeeded() - warmAncestorColumns() - warmPreviewTargetIfNeeded() - } - - // MARK: - Forwarded read state (the owner builds the band from these) - - /// The current column's filtered entries — what the Files band's items are built from. - var visibleEntries: [FileEntry] { navigation.visibleEntries } - /// The highlighted row index into `visibleEntries` (drives the band's selected index). - var highlightedIndex: Int { navigation.highlightedIndex } - /// The highlighted entry, or nil on an empty column. - var highlightedEntry: FileEntry? { navigation.highlightedEntry } - /// The preview target for the current highlight (file-self vs folder-peek), or nil. - var previewTarget: FilesNavigationModel.PreviewTarget? { navigation.previewTarget } - /// Whether an ascend from here would leave the current column. - var canAscend: Bool { navigation.canAscend } - /// The per-root remembered locations to persist (the owner writes these back on depth change). - var rememberedLocations: [URL: URL] { navigation.rememberedLocations } - /// The current location (roots vs. a concrete folder) — lets the owner gate "did the depth change?". - var current: FilesNavigationModel.Location { navigation.current } - /// The ancestor folders above `current`, oldest first — drives any ancestor-rail / ancestor warming. - var ancestors: [URL] { navigation.ancestors } - /// The ordered breadcrumb path (root → … → highlighted item) for the bottom bar (refinement 4). Updates - /// live as the highlight / folder changes. - var breadcrumb: [FilesBreadcrumbComponent] { navigation.breadcrumb } - - // MARK: - Driving the column (the owner routes the recognizer here) - - /// Descend into the highlighted folder (horizontal, descend-direction). Warms the new column / preview - /// if they aren't cached yet. - func descend() { - navigation.descend() - warmCurrentColumnIfNeeded() - warmPreviewTargetIfNeeded() - } - - /// Ascend one level / back-out-to-roots (horizontal, the other direction). Warms as needed (the column - /// we ascend to is usually already cached, but a cold restore may not be). - func ascend() { - navigation.ascend() - warmCurrentColumnIfNeeded() - warmPreviewTargetIfNeeded() - } - - /// Move the highlight up one row (clamped at the top). Warms the new highlight's preview folder-peek if - /// needed. - func highlightUp() { - navigation.highlightUp() - warmPreviewTargetIfNeeded() - } - - /// Move the highlight down one row. Warms the new highlight's preview folder-peek if needed. - func highlightDown() { - navigation.highlightDown() - warmPreviewTargetIfNeeded() - } - - // MARK: - Cache feeding (the async bridge) - - /// If the current column is a folder we haven't listed yet, kick off an async listing for it. A no-op - /// at the roots list (the roots column is synthesized, not listed) and when the folder is already cached - /// or in flight. - private func warmCurrentColumnIfNeeded() { - guard case let .folder(folder) = navigation.current else { return } - fetchIfNeeded(folder) - } - - /// Warm each restored ancestor folder's column (refinement 2: restore AT OPEN with no empty first - /// frame). When the band opens deep — via the restore-last-location landing — the ancestor stack is - /// already populated but their listings aren't cached; listing them now means an ascend (and any - /// ancestor-rail peek) shows real contents immediately rather than empty-for-one-beat. Cheap and - /// coalesced: `fetchIfNeeded` no-ops for anything already cached or in flight. A no-op when the stack is - /// empty (the common shallow open). - private func warmAncestorColumns() { - for ancestor in navigation.ancestors { - fetchIfNeeded(ancestor) - } - } - - /// If the highlighted entry is a folder whose contents aren't cached (the preview folder-peek would show - /// empty), kick off a listing for it so the peek fills in. A no-op for a file highlight or an empty - /// column. - private func warmPreviewTargetIfNeeded() { - guard let entry = navigation.highlightedEntry, entry.isDirectory else { return } - fetchIfNeeded(entry.url) - } - - /// List `folder` off-main unless it's already cached or a listing is in flight; on return store the - /// result, re-feed the model's current column, and notify the owner. Coalesces duplicate requests. - private func fetchIfNeeded(_ folder: URL) { - let key = folder.standardizedFileURL.path - guard cache[key] == nil, !inFlight.contains(key) else { return } - inFlight.insert(key) - let order = sortOrder - let direction = sortDirection - let task = Task { [weak self] in - let listed = await self?.lister(folder, order) ?? [] - let ordered = Self.applyingDirection(direction, to: listed) - self?.store(ordered, forFolderPath: key) - } - listingTasks.append(task) - } - - /// Await every in-flight listing (and the cascade they trigger — a descend warms its new column, whose - /// store re-feeds and may warm a newly-highlighted folder's preview) until the controller is quiescent. - /// The live UI never needs to block on this (the view refreshes reactively via `onColumnChanged`); it - /// exists so an initial-warm caller — and the unit tests — can drive the async bridge to completion - /// deterministically without sleeping. Loops because completing one task can enqueue the next. - func settle() async { - while !listingTasks.isEmpty { - let pending = listingTasks - listingTasks = [] - for task in pending { await task.value } - } - } - - /// Store a completed listing into the cache, drop the in-flight mark, re-feed the model's current column - /// (so a column whose miss triggered this listing now shows the real entries — and the highlight is - /// preserved), and notify the owner to rebuild + republish when something visible changed. Runs on the - /// main actor. - private func store(_ entries: [FileEntry], forFolderPath key: String) { - cache[key] = entries - inFlight.remove(key) - // Re-feed the current column unconditionally (cheap: re-reads through the warm cache). Notify only - // when this listing actually changed a visible surface — the current column, or the folder-peek of - // the *highlighted* folder — so an unrelated prefetch doesn't churn the view. - let affectsCurrent = (navigation.current.folderURL?.standardizedFileURL.path == key) - let highlighted = navigation.highlightedEntry - let affectsPreview = (highlighted?.isDirectory == true) - && (highlighted?.url.standardizedFileURL.path == key) - navigation.reloadCurrentColumn() - if affectsCurrent || affectsPreview { onColumnChanged() } - } - - // MARK: - Reset (a fresh launcher open / a roots change) - - /// Rebuild the navigator over a new `roots`/`remembered` set (e.g. the user edited the roots, or a fresh - /// launcher open with changed settings). Clears the cache and re-warms the landing column. - func reset(roots: [URL], remembered: [URL: URL]) { - cache = [:] - inFlight = [] - rebindNavigationToLiveCache(roots: roots, remembered: remembered) - warmCurrentColumnIfNeeded() - warmAncestorColumns() - warmPreviewTargetIfNeeded() - } - - /// Build (or rebuild) `navigation` with a synchronous lister bound to *this controller's* live cache, so - /// every read the model makes (column reload, ascend re-highlight, preview folder-peek) goes through the - /// cache rather than the filesystem. - private func rebindNavigationToLiveCache(roots: [URL], remembered: [URL: URL]) { - navigation = FilesNavigationModel(roots: roots, - remembered: remembered, - restoreLastLocation: restoreLastLocation, - lister: { [weak self] url in - self?.cache[url.standardizedFileURL.path] ?? [] - }) - } - - // MARK: - Live system lister - - /// The default async lister: a fresh `DirectoryLister` read mapped to a non-throwing `[FileEntry]`. A - /// read failure yields an empty column here — the typed `FileActionError` is still surfaced by the open - /// path; a column *miss* degrades to empty rather than throwing through the pure model. The primary sort - /// key is applied by the lister; the controller applies the direction (folders-first) on return. - nonisolated static func systemLister(_ folder: URL, _ order: FilesSortOrder) async -> [FileEntry] { - let lister = DirectoryLister() - return (try? await lister.contents(of: folder, sortedBy: order)) ?? [] - } - - // MARK: - Sort mapping (settings → FilesSortOrder, folders-first descending) - - /// Map the persisted `FilesSortField` to the lister's `FilesSortOrder` (the primary key only — the - /// ascending/descending direction is applied separately by `applyingDirection(_:to:)` because - /// `FilesSortOrder` is directionless and always folders-first ascending). - nonisolated static func sortOrder(field: FilesSortField) -> FilesSortOrder { - switch field { - case .name: return .name - case .date: return .dateModified - case .kind: return .kind - } - } - - /// Apply `direction` to an already-`FilesSortOrder`-sorted (ascending, folders-first) list, reversing - /// **within the folders-first partition** for `.descending`. - /// - /// `FilesSortOrder` has no direction and always sorts ascending, folders-first. Reversing the *whole* - /// array would put folders last (the leading folder run lands at the tail), breaking folders-first. - /// Instead we partition into [folders] + [files] and reverse each partition independently, so folders - /// stay on top but descending within their group (files likewise). `.ascending` returns the input - /// unchanged. Pure — used by the live lister wrapper and exercised directly in tests. - nonisolated static func applyingDirection(_ direction: FilesSortDirection, to entries: [FileEntry]) -> [FileEntry] { - guard direction == .descending else { return entries } - let folders = entries.filter { $0.isDirectory } - let files = entries.filter { !$0.isDirectory } - return Array(folders.reversed()) + Array(files.reversed()) - } -} diff --git a/Sources/ThreeFingerSwitcher/Files/FilesDelivery.swift b/Sources/ThreeFingerSwitcher/Files/FilesDelivery.swift deleted file mode 100644 index 45df4eb..0000000 --- a/Sources/ThreeFingerSwitcher/Files/FilesDelivery.swift +++ /dev/null @@ -1,45 +0,0 @@ -import Foundation - -// MARK: - Contextual delivery (pure, MLX-free Core) - -/// What the Files band writes to the pasteboard when **delivering** a highlighted entry to the captured -/// front app (`files-contextual-delivery`). Dual-representation **by design**: the same item carries the -/// file's reference (`fileURL`) AND its absolute path as text, and the *receiver* picks the form it -/// understands — a text field / terminal / editor consumes the path string, a Finder window consumes the -/// file reference (copying it in). The app never inspects the front context to choose which to send; -/// macOS's paste contract does the routing. Pure value — the actual `NSPasteboard` write happens at the -/// boundary (`SystemPasteboard.setFileDelivery`). -struct FilesDeliveryPayload: Equatable { - /// The file reference a Finder window consumes (a standardized `fileURL`). - let url: URL - /// The standardized absolute path a text target consumes. - let path: String -} - -/// Builds the dual-representation delivery payload for an entry. Pure and testable. -enum FilesDelivery { - /// The payload for delivering `entry`. The path is the entry's **standardized** absolute path (equal to - /// `FileEntry.id`), so a delivered path is canonical regardless of how the root was configured. - static func payload(for entry: FileEntry) -> FilesDeliveryPayload { - let standardized = entry.url.standardizedFileURL - return FilesDeliveryPayload(url: standardized, path: standardized.path) - } -} - -// MARK: - Lift action - -/// What the Files-band **lift** (the drill's primary resolve excursion — by default the plain lift) does -/// when committed (`files-band`, `tunable-settings`). `deliver` (the default) pastes the highlighted entry -/// into the captured front app; `open` opens it (file → default app, folder → Finder window). -/// -/// This is **orthogonal** to the gesture *binding* (`GestureBindings.FilesDrillBinding`, which decides -/// *which excursion* is the primary resolve vs. the menu vs. discard): the binding says which physical move -/// is "the primary resolve," and this says what that move's commit performs. Keeping it separate avoids -/// restructuring the already-shipped one-to-one drill binding. -enum FilesLiftAction: String, Codable, CaseIterable, Identifiable, Sendable { - /// Deliver the entry to the captured front app (the default — `files-contextual-delivery`). - case deliver - /// Open the entry (file → default app, folder → Finder window) on the current Space. - case open - public var id: String { rawValue } -} diff --git a/Sources/ThreeFingerSwitcher/Files/FilesNavigationModel.swift b/Sources/ThreeFingerSwitcher/Files/FilesNavigationModel.swift deleted file mode 100644 index 9537160..0000000 --- a/Sources/ThreeFingerSwitcher/Files/FilesNavigationModel.swift +++ /dev/null @@ -1,445 +0,0 @@ -import Foundation - -/// One step of the Files band's bottom breadcrumb bar (refinement 4): a display `name` and the `url` it -/// points at, ordered root → … → currently-highlighted item. A value type so the view can diff the path -/// and the model can recompute it implicitly as the highlight / folder changes. -struct FilesBreadcrumbComponent: Equatable { - /// The component's display name (a folder's or entry's last path component; a root's prettifiable name). - let name: String - /// The file URL this component refers to (standardized). - let url: URL -} - -/// The Files band's **pure** column-navigation state machine (design D6 / spec "Column navigation -/// model"). It owns the ancestors stack, the current folder, the highlighted entry, and the per-root -/// remembered locations — and **nothing else**: it never touches `FileManager`. -/// -/// Determinism by injection: the only way it learns a folder's contents is the injected `lister` -/// closure (the live app wires that to `DirectoryLister`; a test wires a fixture). Because every input is -/// either a value handed in or that one closure, the whole machine is synchronous and exhaustively -/// unit-testable without a filesystem or a running app. The view layer reads `visibleEntries` / -/// `highlightedIndex` / `previewTarget` and renders; the recognizer drives `descend` / `ascend` / -/// `highlightUp` / `highlightDown`. -/// -/// Mirrors how `LauncherModel`'s navigation is "pure, knows item counts + columns": this is the Files -/// analogue, just over a folder stack rather than a band grid. -struct FilesNavigationModel { - - // MARK: - Location - - /// Where the current column is rooted: the configured roots list (the entry column), or a concrete - /// folder reached by descending. Backing out past a root returns to `.roots` (spec). - enum Location: Equatable { - /// The entry column: the configured root folders. - case roots - /// A concrete folder whose live contents fill the current column. - case folder(URL) - - /// The folder URL when in a folder, else `nil` (the roots list has no single folder URL). - var folderURL: URL? { - if case let .folder(url) = self { return url } - return nil - } - } - - // MARK: - Injected dependencies - - /// The configured local root folders, in order — shown as the initial current column. Standardized - /// on init so a remembered-location prefix test (`hasPrefix`) lines up with listed entry ids. - let roots: [URL] - - /// Lists a folder's entries on demand (the live app: `DirectoryLister`; a test: a fixture map). The - /// model calls this exactly when the current folder changes (descend / ascend / restore), caching the - /// result in `entries` until the next move — it never re-lists speculatively. - private let lister: (URL) -> [FileEntry] - - // MARK: - State (current column) - - /// The ancestor folders above `current`, oldest first — i.e. the folders a left/ascend step pops back - /// through. Empty when `current` is a root's top level or the roots list itself. Surfaced for the - /// icon-rail (design D6). - private(set) var ancestors: [URL] = [] - - /// Where the current column is rooted. - private(set) var current: Location = .roots - - /// The current column's entries (the roots as folder entries when at `.roots`, else the listed folder - /// contents). The unfiltered backing list; the view shows `visibleEntries`. - private(set) var entries: [FileEntry] = [] - - /// The highlighted row's index into `visibleEntries`, clamped to it. `0` on an empty column. - private(set) var highlightedIndex: Int = 0 - - /// Per-root remembered **deepest** location: `root → last folder visited within it`. Injected in (the - /// caller restores it from persistence) and surfaced out via `rememberedLocations` so the caller can - /// persist it again — the model itself never persists. Keyed by standardized root URL. - private var remembered: [URL: URL] = [:] - - /// Whether restoring the per-root remembered deepest location is enabled (the Hub "remember and reopen - /// the last folder" toggle). When ON, the band both **opens** on the remembered folder (the `init` - /// landing) AND, on a later descend into a root from the roots list, jumps straight to that root's - /// remembered deepest location (`enterRoot`). When OFF, the band opens on the roots list and descending - /// into a root lands on the root's **top level** — never the remembered folder. Tracking the deepest - /// location continues regardless (so flipping the toggle back ON restores correctly); only *using* it to - /// land is gated by this flag. - private let restoreLastLocation: Bool - - // MARK: - Init - - /// Build the navigator over `roots`, restoring any previously-`remembered` per-root locations. `roots` - /// and the remembered keys/values are standardized so path-prefix math and entry ids line up. - /// - /// Landing: when `restoreLastLocation` is true (the Hub "restore last folder" toggle, default ON) AND a - /// `remembered` deepest location exists for a configured root, the model starts **at that folder** — - /// `current = .folder(lastFolder)`, ancestors reconstructed and contents listed — so the band OPENS - /// DISPLAYING the last folder (refinement 2: restore AT OPEN, not on the first descend). When false (or - /// nothing is remembered) it lands on the roots list as before. The deepest-location *tracking* runs on - /// every later move regardless; this flag only governs the initial landing. - /// - /// - Parameters: - /// - roots: the configured local root folders (the entry column). - /// - remembered: previously-persisted `root → deepest folder` map (default empty). - /// - restoreLastLocation: open displaying the remembered deepest folder (true) vs. the roots list - /// (false). Default false so callers that haven't opted in keep the roots-list landing. - /// - lister: lists a folder's entries on demand. - init(roots: [URL], - remembered: [URL: URL] = [:], - restoreLastLocation: Bool = false, - lister: @escaping (URL) -> [FileEntry]) { - self.roots = roots.map { $0.standardizedFileURL } - self.lister = lister - self.restoreLastLocation = restoreLastLocation - var standardizedRemembered: [URL: URL] = [:] - for (root, location) in remembered { - standardizedRemembered[root.standardizedFileURL] = location.standardizedFileURL - } - self.remembered = standardizedRemembered - if restoreLastLocation, let landing = restorableLanding() { - // Open straight onto the remembered folder: same ancestor reconstruction + listing as `enterRoot`, - // but performed AT INIT so the displayed column matches where crossing horizontally will land. - ancestors = intermediateFolders(from: landing.root, to: landing.folder) - setCurrentFolder(landing.folder) - } else { - reloadCurrentColumn(resetHighlight: true) - } - } - - // MARK: - Derived view state - - /// The entries shown in the current column. Kept as a distinct accessor — the view, controller, and - /// band builder all read the column through this name — though with type-to-filter search removed it - /// now mirrors `entries` directly (the column is never filtered). - var visibleEntries: [FileEntry] { entries } - - /// The highlighted entry within `visibleEntries`, or `nil` when the column is empty. - var highlightedEntry: FileEntry? { - let visible = visibleEntries - guard visible.indices.contains(highlightedIndex) else { return nil } - return visible[highlightedIndex] - } - - /// The per-root remembered locations to persist (`root → deepest folder visited`). The caller writes - /// this back to storage; the model never persists on its own. - var rememberedLocations: [URL: URL] { remembered } - - /// Whether an ascend step from here would leave the current column (pop an ancestor, or step a root's - /// top level back to the roots list). False only on the roots list itself, where ascend is a no-op. - var canAscend: Bool { current != .roots } - - /// The ordered path the bottom breadcrumb bar shows (refinement 4): root → … → the currently-HIGHLIGHTED - /// item. In a folder it is the ancestors, then the current folder, then the highlighted entry (when one - /// is highlighted); at the roots list it is just the highlighted root (a root is the whole path). Recomputes - /// implicitly as the highlight / folder changes — naming each URL by its last path component, the same - /// AppKit-free convention `rootEntry(for:)` uses (the view can prettify). - var breadcrumb: [FilesBreadcrumbComponent] { - switch current { - case .roots: - // At the entry column the highlighted root IS the whole path; nothing above it. - guard let root = highlightedEntry else { return [] } - return [FilesBreadcrumbComponent(name: root.name, url: root.url)] - case let .folder(folder): - var components = ancestors.map { Self.breadcrumbComponent(for: $0) } - components.append(Self.breadcrumbComponent(for: folder)) - // The highlighted entry is the leaf — but only when it isn't already the current folder (it never - // is; the highlight is a child) and one exists (an empty/over-filtered column stops at the folder). - if let highlighted = highlightedEntry { - components.append(FilesBreadcrumbComponent(name: highlighted.name, url: highlighted.url)) - } - return components - } - } - - // MARK: - Preview target - - /// What the highlighted entry previews (spec "preview-target derivation"): a **file** previews itself; - /// a **folder** previews *its own contents* (the column descending would promote). `nil` when nothing - /// is highlighted. The folder case carries the same listing the model would make current on descend, - /// so the view's folder-contents peek and a subsequent descend agree. - enum PreviewTarget: Equatable { - /// Preview this file itself (QuickLook / icon fallback in the view). - case file(FileEntry) - /// Peek this folder's listed contents (what descending would make the current column). - case folder(FileEntry, contents: [FileEntry]) - } - - /// The preview target for the current highlight, or `nil` when the column is empty. - var previewTarget: PreviewTarget? { - guard let entry = highlightedEntry else { return nil } - if entry.isDirectory { - return .folder(entry, contents: lister(entry.url)) - } - return .file(entry) - } - - // MARK: - Depth transitions (horizontal axis) - - /// Descend into the highlighted **folder**: push the prior current folder onto `ancestors` and make - /// the highlighted folder current, listing its contents (spec). A no-op when nothing is highlighted or - /// the highlight is a file (files open; they don't descend). Descending always resets the highlight to - /// the top of the new column. - /// - /// From the **roots list**, descending into a root makes that root the current folder with an *empty* - /// ancestor stack (a root is the base of its own column, not an ancestor) — and if that root has a - /// remembered deeper location, restore straight to it (spec "re-entering that root restores it"). - mutating func descend() { - guard let entry = highlightedEntry, entry.isDirectory else { return } - let target = entry.url - - switch current { - case .roots: - // Entering a root: restore its remembered deepest location if we have one, else land on it. - enterRoot(target) - case let .folder(folder): - ancestors.append(folder) - setCurrentFolder(target) - } - rememberCurrentLocation() - } - - /// Ascend one level (spec): pop the deepest ancestor back to current, or — at a root's top level (no - /// ancestors) — return to the roots list. A no-op on the roots list itself. Resets the highlight and - /// re-highlights the folder we came up from so the column doesn't feel lost. - mutating func ascend() { - switch current { - case .roots: - return // already at the top; nothing to pop - case let .folder(folder): - if let parent = ancestors.popLast() { - setCurrentFolder(parent, highlighting: folder) - } else { - returnToRoots(highlighting: folder) - } - } - rememberCurrentLocation() - } - - // MARK: - Highlight transitions (vertical axis) - - /// Move the highlight **down** one row (toward higher indices), clamped at the last visible row. A - /// no-op (but harmless) on an empty column. - mutating func highlightDown() { - let count = visibleEntries.count - guard count > 0 else { highlightedIndex = 0; return } - highlightedIndex = min(highlightedIndex + 1, count - 1) - } - - /// Move the highlight **up** one row (toward index 0), clamped at the top. An up-step while already at - /// index 0 is a no-op — the highlight simply stays on the top row. (There is no type-to-filter search to - /// overflow into; the navigator stays pure-trackpad.) - mutating func highlightUp() { - highlightedIndex = max(highlightedIndex - 1, 0) - } - - // MARK: - Re-feed (async-listing bridge) - - /// Re-read the current column through the injected `lister` **without** moving — used when the column's - /// contents change underneath a stationary navigator. The Files band lists folders asynchronously - /// (off-main), but this model is synchronous and only re-reads on a move; when a late listing lands in - /// the controller's cache (which backs the live `lister`), the controller calls this to pull the now-warm - /// contents into the current column. The highlight is **preserved** (re-clamped, not reset) so a row the - /// user is already on doesn't jump when its folder finishes loading; the location is untouched. A - /// no-op-shaped read at `.roots` (the roots column is synthesized, not listed). - mutating func reloadCurrentColumn() { - reloadCurrentColumn(resetHighlight: false) - } - - // MARK: - Private: column (re)loading - - /// Make `folder` the current column and list it, optionally re-highlighting a known child (used on - /// ascend so the folder we came up from is selected). - private mutating func setCurrentFolder(_ folder: URL, highlighting child: URL? = nil) { - current = .folder(folder) - reloadCurrentColumn(resetHighlight: child == nil) - if let child { highlight(url: child) } - } - - /// Return to the roots list (back-out-to-roots), optionally re-highlighting the root we came up from. - /// Clears ancestors. - private mutating func returnToRoots(highlighting child: URL? = nil) { - ancestors = [] - current = .roots - reloadCurrentColumn(resetHighlight: child == nil) - if let child { highlight(url: child) } - } - - /// The folder the band should OPEN displaying when restore-last-location is on, or `nil` to fall back to - /// the roots list. `folder` is a remembered location that is still itself or a descendant of a configured - /// root — i.e. a root the user has actually been into; `intermediateFolders(from:to:)` then rebuilds the - /// ancestor chain for it (yielding `[]` when `folder == root`). Stale paths that no longer sit under their - /// root (the root moved) are skipped, exactly like `enterRoot`. - /// - /// Choice when several roots are remembered: the persisted seam (`AppSettings.filesRememberedLocations`) - /// is per-root with **no timestamp**, so a true global "last visited" isn't recoverable. We therefore pick - /// the **deepest** valid remembered path — the most specific "where you left off" — breaking ties by - /// configured-root order so the result is deterministic. (The common single-root setup has exactly one - /// candidate, so this only matters across multiple deep roots.) - private func restorableLanding() -> (root: URL, folder: URL)? { - var best: (root: URL, folder: URL)? - for root in roots { - guard let deepest = remembered[root] else { continue } - let candidate: URL - if deepest == root { - candidate = root // root top level (valid, depth = the root's own) - } else if isDescendant(deepest, of: root) { - candidate = deepest // a deeper remembered location under this root - } else { - continue // stale: outside the root → ignore - } - // Deepest path wins; configured-root order is the tie-break (strict `>` keeps the earlier root). - if best == nil || candidate.standardizedFileURL.path.count > best!.folder.standardizedFileURL.path.count { - best = (root, candidate) - } - } - return best - } - - /// Enter `root` from the roots list. When `restoreLastLocation` is ON, restore its remembered deepest - /// location if one exists and is still a descendant of the root (spec "restore where you left off"), - /// reconstructing the ancestor stack so ascending walks back up correctly; otherwise — and **always** - /// when the toggle is OFF — land on the root's **top level**. Gating on `restoreLastLocation` here (not - /// just at `init`) is what keeps the toggle honest: with it off, descending into a root must NOT jump to - /// the last-visited folder (the deepest-location map is still tracked for when the toggle is turned on). - private mutating func enterRoot(_ root: URL) { - ancestors = [] - if restoreLastLocation, - let deepest = remembered[root], deepest != root, isDescendant(deepest, of: root) { - // Rebuild ancestors = [root, …intermediate folders…] up to (but excluding) `deepest`. - ancestors = intermediateFolders(from: root, to: deepest) - setCurrentFolder(deepest) - } else { - setCurrentFolder(root) - } - } - - /// List the current location into `entries`, optionally resetting the highlight to the top. At - /// `.roots` the entries are the configured roots projected as folder `FileEntry`s (no `FileManager` - /// read); in a folder they come from the injected `lister`. - private mutating func reloadCurrentColumn(resetHighlight: Bool) { - switch current { - case .roots: - entries = roots.map(Self.rootEntry(for:)) - case let .folder(folder): - entries = lister(folder) - } - if resetHighlight { highlightedIndex = 0 } - clampHighlight() - } - - /// Re-clamp `highlightedIndex` into the current `visibleEntries` so it never points off the end (after - /// a re-list, a search, or an ascend that re-highlights). Lands on `0` for an empty column. - private mutating func clampHighlight() { - let count = visibleEntries.count - guard count > 0 else { highlightedIndex = 0; return } - highlightedIndex = min(max(highlightedIndex, 0), count - 1) - } - - /// Highlight the entry whose URL matches `url` within the current (unfiltered) column, if present — - /// used to re-select the folder we just came up from on ascend. Falls back to leaving the clamp at 0. - private mutating func highlight(url: URL) { - let target = url.standardizedFileURL.path - if let index = visibleEntries.firstIndex(where: { $0.id == target }) { - highlightedIndex = index - } else { - clampHighlight() - } - } - - // MARK: - Private: remembered-location bookkeeping - - /// Record the current folder as the deepest-visited location for whichever root it lives under, so a - /// later re-entry restores it (spec: "remember the **deepest** location"). A no-op at the roots list. - /// The "deepest" contract is explicit: this runs on *every* move (descend AND ascend), so it must - /// never let an ascend shrink the remembered depth — it overwrites only when the current folder is at - /// least as deep as what's already remembered (i.e. the existing value isn't a strict descendant of - /// `folder`). Descending deeper updates it; ascending back up keeps the deeper mark. - private mutating func rememberCurrentLocation() { - guard case let .folder(folder) = current, let root = owningRoot(of: folder) else { return } - if let existing = remembered[root], isDescendant(existing, of: folder) { return } - remembered[root] = folder - } - - /// The configured root that `folder` lives under (itself or a descendant), or `nil` if none — the - /// longest matching root wins so nested roots remember independently. - private func owningRoot(of folder: URL) -> URL? { - roots - .filter { folder == $0 || isDescendant(folder, of: $0) } - .max { $0.path.count < $1.path.count } - } - - // MARK: - Private: path helpers (pure) - - /// A root projected as a folder `FileEntry` for the roots column. The display name is the root's - /// last path component (the view can prettify; the model stays AppKit-free), `isDirectory == true`. - private static func rootEntry(for root: URL) -> FileEntry { - let name = root.lastPathComponent.isEmpty ? root.path : root.lastPathComponent - return FileEntry(url: root, name: name, isDirectory: true, modificationDate: nil, kind: .folder) - } - - /// A folder URL as a breadcrumb component — its last path component (falling back to the full path for a - /// filesystem root), matching `rootEntry(for:)`'s naming so an ancestor reads the same in the rail and - /// the breadcrumb. The view can prettify; the model stays AppKit-free. - private static func breadcrumbComponent(for folder: URL) -> FilesBreadcrumbComponent { - let name = folder.lastPathComponent.isEmpty ? folder.path : folder.lastPathComponent - return FilesBreadcrumbComponent(name: name, url: folder) - } - - /// True when `url` is a strict descendant of `ancestor` (a deeper path under it). Path-prefix math on - /// standardized paths, boundary-aware (so `/a/bc` is not treated as under `/a/b`). - private func isDescendant(_ url: URL, of ancestor: URL) -> Bool { - let child = url.standardizedFileURL.path - var base = ancestor.standardizedFileURL.path - if !base.hasSuffix("/") { base += "/" } - return child.hasPrefix(base) && child.count > base.count - } - - /// The chain of folders to push as ancestors when restoring from `root` down to `deepest`: `[root, - /// …each intermediate folder…]`, excluding `deepest` itself (which becomes current). - /// - /// Works in **path-space** and rebuilds each ancestor via `URL(fileURLWithPath:)` so the result uses - /// the SAME no-trailing-slash convention as every other URL in the model (`FileEntry.url`, the roots, - /// `current`). `deletingLastPathComponent()` would instead yield directory URLs *with* a trailing - /// slash that fail `==` against those bare URLs — which `ascend`'s `popLast()` → `setCurrentFolder` - /// comparison (and the tests) depend on. The caller guarantees `deepest` is a strict descendant of - /// `root`, so the path-component prefix below always lines up. - private func intermediateFolders(from root: URL, to deepest: URL) -> [URL] { - let rootPath = root.standardizedFileURL.path - let rootComponents = pathComponents(rootPath) - let deepComponents = pathComponents(deepest.standardizedFileURL.path) - guard deepComponents.count > rootComponents.count else { return [] } - - // Ancestors run from the root down to the folder just above `deepest` (its parent), inclusive of - // the root, exclusive of `deepest`. e.g. root /Home, deepest /Home/Docs/Sub → [/Home, /Home/Docs]. - var chain: [URL] = [] - for end in rootComponents.count...(deepComponents.count - 1) { - let path = "/" + deepComponents.prefix(end).joined(separator: "/") - chain.append(URL(fileURLWithPath: path)) - } - return chain - } - - /// The non-empty path segments of an absolute POSIX path (`/Home/Docs` → `["Home", "Docs"]`; `/` → - /// `[]`). Pure string splitting — used so ancestor URLs are rebuilt component-by-component. - private func pathComponents(_ path: String) -> [String] { - path.split(separator: "/").map(String.init) - } -} diff --git a/Sources/ThreeFingerSwitcher/Files/FilesToolCatalog.swift b/Sources/ThreeFingerSwitcher/Files/FilesToolCatalog.swift deleted file mode 100644 index a7c6420..0000000 --- a/Sources/ThreeFingerSwitcher/Files/FilesToolCatalog.swift +++ /dev/null @@ -1,28 +0,0 @@ -import Foundation - -/// Known terminals and editors the Files action menu can "open a folder in" (`files-action-menu`). Pure -/// data — bundle-id seeds + display names, in a sensible default order. The boundary (`AppCoordinator`) -/// probes which are actually installed (via `NSWorkspace.urlForApplication(withBundleIdentifier:)`) and -/// applies the user's curation (`AppSettings.filesToolsDisabled`); this file holds no AppKit dependency so -/// the seed list stays unit-inspectable. -enum FilesToolCatalog { - /// `(bundleID, display name)` for terminals, Apple Terminal first. - static let terminals: [(bundleID: String, name: String)] = [ - ("com.apple.Terminal", "Terminal"), - ("com.googlecode.iterm2", "iTerm"), - ("dev.warp.Warp-Stable", "Warp"), - ("com.github.wez.wezterm", "WezTerm"), - ("net.kovidgoyal.kitty", "kitty"), - ("org.alacritty", "Alacritty"), - ("com.mitchellh.ghostty", "Ghostty"), - ("co.zeit.hyper", "Hyper") - ] - /// `(bundleID, display name)` for code editors. - static let editors: [(bundleID: String, name: String)] = [ - ("com.microsoft.VSCode", "VS Code"), - ("com.todesktop.230313mzl4w4u92", "Cursor"), - ("com.sublimetext.4", "Sublime Text"), - ("dev.zed.Zed", "Zed"), - ("com.apple.dt.Xcode", "Xcode") - ] -} diff --git a/Sources/ThreeFingerSwitcher/Files/OpenWithEntry.swift b/Sources/ThreeFingerSwitcher/Files/OpenWithEntry.swift deleted file mode 100644 index d6b0f9a..0000000 --- a/Sources/ThreeFingerSwitcher/Files/OpenWithEntry.swift +++ /dev/null @@ -1,39 +0,0 @@ -import Foundation - -/// One row in the Files Open-With picker (`files-band`). The picker lists the external applications that can -/// open the highlighted file. A pure value type — the controller acts on the case (an external open), the -/// view renders `label`. -enum OpenWithEntry: Identifiable, Equatable { - /// Open in an external application (the system candidate). - case external(OpenWithCandidate) - - /// Stable list identity (so re-querying associations doesn't strobe the highlight). - var id: String { - switch self { - case let .external(candidate): return "external:\(candidate.id)" - } - } - - /// The human-facing row label: the application's display name. - var label: String { - switch self { - case let .external(candidate): - return candidate.app.name - } - } - - /// True when this is the file's default external application (so the view can mark it). - var isDefault: Bool { - if case let .external(candidate) = self { return candidate.isDefault } - return false - } -} - -/// Builds the Open-With picker rows for a highlighted file: the external applications, in system order. -/// Pure and testable. -enum OpenWithEntries { - /// - `externalApps`: the system Open-With candidates, in system order. - static func build(externalApps: [OpenWithCandidate]) -> [OpenWithEntry] { - externalApps.map(OpenWithEntry.external) - } -} diff --git a/Sources/ThreeFingerSwitcher/Gesture/FlickExcursionClassifier.swift b/Sources/ThreeFingerSwitcher/Gesture/FlickExcursionClassifier.swift deleted file mode 100644 index ce63374..0000000 --- a/Sources/ThreeFingerSwitcher/Gesture/FlickExcursionClassifier.swift +++ /dev/null @@ -1,103 +0,0 @@ -import Foundation -import CoreGraphics - -/// The pure fast-vs-soft flick classifier (D4 of the canvas grammar, extracted by -/// `notch-conversation-gestures` so the launcher canvas and the expanded notch conversation share ONE -/// implementation — the constants and feel can never drift between surfaces). -/// -/// One instance tracks ONE excursion: `begin` seeds the origin, `track` accumulates the per-axis PEAK of -/// the smoothed centroid speed plus the timing of the last genuinely-fast frame (the lift frame is empty -/// with `.zero` velocity, so the flick speed must come from the last in-contact frame), and -/// `classifyOnLift` renders the D4 verdict: a FLICK requires (a) the dominant axis's travel to have -/// crossed the floor, (b) its peak velocity to have crossed the threshold, AND (c) the lift to have -/// arrived within the lift window of the last fast frame. A slow reading-scrub (sub-threshold peak) or a -/// decelerated hold-then-lift is a SCROLL and classifies as nil — the caller emits nothing and native -/// scrolling is untouched. -/// -/// Pure value type — no clocks, no settings; every threshold is an input, so it is deterministic under -/// `swift test`. The caller owns routing (which frames feed it) and one-shot emission. -struct FlickExcursionClassifier { - /// Whether an excursion is being tracked (`begin` ran; `reset` clears it). - private(set) var started = false - - private var start = CGPoint.zero - /// Running PEAK of `abs(centroidVelocity)` per axis across the in-contact frames. - private var peakVelX: CGFloat = 0 - private var peakVelY: CGFloat = 0 - /// Timestamp of the most recent in-contact frame whose DOMINANT-axis speed crossed the threshold, - /// and the most recent in-contact frame's timestamp (the lift window is measured between them). - private var lastFastTime: CFTimeInterval = 0 - private var lastContactTime: CFTimeInterval = 0 - /// Signed dominant-axis travel captured from the last in-contact frame, so the (empty) lift frame - /// can still classify direction and the travel floor. - private var lastDX: CGFloat = 0 - private var lastDY: CGFloat = 0 - /// Whether any in-contact frame crossed the velocity threshold on its dominant axis. - private var sawFastFrame = false - - /// Seed a fresh excursion at the first qualifying contact frame. - mutating func begin(at centroid: CGPoint, time: CFTimeInterval) { - reset() - started = true - start = centroid - lastContactTime = time - } - - /// Accumulate one in-contact frame: travel from the origin, per-axis velocity peaks, and the - /// last-fast-frame timing (dominant axis chosen per-frame by `axisLockRatio`, exactly as the canvas - /// path always did). No-op until `begin`. - mutating func track(centroid: CGPoint, velocity: CGVector, time: CFTimeInterval, - velocityThreshold: CGFloat, axisLockRatio: CGFloat) { - guard started else { return } - let dx = centroid.x - start.x - let dy = centroid.y - start.y - let verticalDominant = abs(dy) >= axisLockRatio * abs(dx) - let vx = abs(velocity.dx) - let vy = abs(velocity.dy) - peakVelX = max(peakVelX, vx) - peakVelY = max(peakVelY, vy) - lastContactTime = time - lastDX = dx - lastDY = dy - let dominantSpeed = verticalDominant ? vy : vx - if dominantSpeed >= velocityThreshold { - sawFastFrame = true - lastFastTime = time - } - } - - /// Classify the just-ended excursion (call on the lift frame, `fingerCount == 0`): an axis-locked - /// flick — exactly one of `dx`/`dy` non-zero, `+1` right/up, `-1` left/down — or nil for a soft - /// scrub, a decelerated lift, or travel under the floor. Does not mutate; the caller `reset()`s. - func classifyOnLift(travelFloor: CGFloat, velocityThreshold: CGFloat, - liftWindow: CFTimeInterval, axisLockRatio: CGFloat) -> (dx: Int, dy: Int)? { - guard started else { return nil } - let verticalDominant = abs(lastDY) >= axisLockRatio * abs(lastDX) - // (a) Travel floor: the dominant axis must have crossed it. - let dominantTravel = verticalDominant ? abs(lastDY) : abs(lastDX) - guard dominantTravel >= travelFloor else { return nil } - // (b) Peak velocity: the dominant axis must have flicked fast at some point. - let dominantPeak = verticalDominant ? peakVelY : peakVelX - guard dominantPeak >= velocityThreshold else { return nil } // slow scrub → SCROLL - // (c) Prompt lift: the lift must follow the last fast frame within the window (a pause before - // lifting means the fingers decelerated to a hold/scroll). - guard sawFastFrame, lastContactTime - lastFastTime <= liftWindow else { return nil } - if verticalDominant { - return (dx: 0, dy: lastDY > 0 ? 1 : -1) // dy>0 = up, dy<0 = down - } - return (dx: lastDX > 0 ? 1 : -1, dy: 0) - } - - /// Clear all per-excursion state so peaks, timing, and travel never leak across gestures. - mutating func reset() { - started = false - start = .zero - peakVelX = 0 - peakVelY = 0 - lastFastTime = 0 - lastContactTime = 0 - lastDX = 0 - lastDY = 0 - sawFastFrame = false - } -} diff --git a/Sources/ThreeFingerSwitcher/Gesture/GestureBindings.swift b/Sources/ThreeFingerSwitcher/Gesture/GestureBindings.swift index d604206..215ee21 100644 --- a/Sources/ThreeFingerSwitcher/Gesture/GestureBindings.swift +++ b/Sources/ThreeFingerSwitcher/Gesture/GestureBindings.swift @@ -1,34 +1,20 @@ import Foundation -// MARK: - Gesture bindings (pure, MLX-free Core) +// MARK: - Gesture bindings (pure Core) -/// User-configurable **resolution**-gesture bindings for the three remappable open surfaces — the AI -/// command canvas, the Files-band drill, and the window switcher's scrub axes. +/// User-configurable gesture bindings — today, the window switcher's per-axis scrub directions. /// -/// Each surface has its OWN action set and its OWN excursion vocabulary; they are deliberately distinct -/// grammars and are NOT unified into one remap (CLAUDE.md: "do not generalize it to this navigation -/// surface"; the spec delta forbids unification). The model is a pure value type — the recognizer's -/// raw-direction emission is unchanged; only the *action* an excursion maps to is configurable. +/// The model is a pure value type — the recognizer's raw-direction emission is unchanged; only the +/// *action* an excursion maps to is configurable. Defaults equal today's hardcoded behavior exactly: +/// both switcher axes scrub normally. /// -/// Defaults equal today's hardcoded behavior exactly: the canvas down-swipe commits / horizontal -/// dismisses / up is ignored; the Files drill lift opens / +1-finger lift opens-with / four-finger -/// horizontal discards; both switcher axes scrub normally. -/// -/// The vocabularies exclude reserved/invalid excursions by construction: single-finger motion (never a -/// trigger anywhere) and the AI canvas's sub-threshold two-finger pan (which stays "read/scroll the -/// canvas") are simply not members of any excursion enum, so they can never be bound. +/// A stored blob written by an older build may carry retired surfaces (the former AI-canvas and +/// Files-drill bindings) as extra JSON keys; `JSONDecoder` ignores unknown keys, so those records +/// still decode into just the switcher binding. public struct GestureBindings: Codable, Equatable, Sendable { - public var canvas: CanvasBinding - public var filesDrill: FilesDrillBinding public var switcher: SwitcherBinding - public init( - canvas: CanvasBinding = .default, - filesDrill: FilesDrillBinding = .default, - switcher: SwitcherBinding = .default - ) { - self.canvas = canvas - self.filesDrill = filesDrill + public init(switcher: SwitcherBinding = .default) { self.switcher = switcher } @@ -36,149 +22,6 @@ public struct GestureBindings: Codable, Equatable, Sendable { public static let `default` = GestureBindings() } -// MARK: - AI command canvas - -extension GestureBindings { - /// What an AI-canvas resolve excursion does. The canvas's `{commit, dismiss, ignore}` actions. - public enum CanvasAction: String, Codable, CaseIterable, Identifiable, Sendable { - case commit - case dismiss - case ignore - public var id: String { rawValue } - } - - /// The two-finger resolve excursions the canvas can bind. Deliberately excludes the sub-threshold - /// two-finger pan (which scrolls/reads the canvas) and any single-finger motion — neither is a member. - public enum CanvasExcursion: String, Codable, CaseIterable, Identifiable, Sendable { - case swipeUp - case swipeDown - case swipeLeft - case swipeRight - public var id: String { rawValue } - } - - /// The canvas action→excursion mapping. A strict one-to-one map: each of the three actions owns one - /// distinct excursion. Four excursions exist, so exactly one excursion is the **spare** (unbound) — - /// the consumer treats the spare per the surface's fallback (today: any horizontal excursion that is - /// not explicitly bound elsewhere discards, preserving "horizontal = dismiss"). - public struct CanvasBinding: Codable, Equatable, Sendable { - public var commit: CanvasExcursion - public var dismiss: CanvasExcursion - public var ignore: CanvasExcursion - - public init(commit: CanvasExcursion, dismiss: CanvasExcursion, ignore: CanvasExcursion) { - self.commit = commit - self.dismiss = dismiss - self.ignore = ignore - } - - /// Today's behavior: down = commit, horizontal = dismiss (bound to left; right is the spare and - /// also discards in the consumer's "any horizontal" fallback), up = ignore. - public static let `default` = CanvasBinding(commit: .swipeDown, dismiss: .swipeLeft, ignore: .swipeUp) - - /// The excursion currently bound to `action`. - public func excursion(for action: CanvasAction) -> CanvasExcursion { - switch action { - case .commit: return commit - case .dismiss: return dismiss - case .ignore: return ignore - } - } - - /// Return a renormalized binding that maps `action → excursion`, keeping a strict one-to-one - /// mapping by **swapping** with whichever action currently holds `excursion`. Pure; `self` is - /// unchanged. If `action` already holds `excursion`, the binding is returned unchanged. - public func assigning(_ excursion: CanvasExcursion, to action: CanvasAction) -> Self { - let previous = self.excursion(for: action) - guard previous != excursion else { return self } - var result = self - for other in CanvasAction.allCases - where other != action && result.excursion(for: other) == excursion { - result.set(previous, for: other) // the conflicting action inherits the old excursion - } - result.set(excursion, for: action) - return result - } - - private mutating func set(_ excursion: CanvasExcursion, for action: CanvasAction) { - switch action { - case .commit: commit = excursion - case .dismiss: dismiss = excursion - case .ignore: ignore = excursion - } - } - } -} - -// MARK: - Files-band drill - -extension GestureBindings { - /// What a Files-drill resolve excursion does. The drill's `{open, openWith, discard}` actions. - public enum FilesAction: String, Codable, CaseIterable, Identifiable, Sendable { - case open - case openWith - case discard - public var id: String { rawValue } - } - - /// The Files-drill resolution excursions. Deliberately excludes single-finger motion. - public enum FilesExcursion: String, Codable, CaseIterable, Identifiable, Sendable { - case lift - case plusOneFingerLift - case fourFingerHorizontal - public var id: String { rawValue } - } - - /// The Files action→excursion mapping. A strict one-to-one map over the three excursions. - public struct FilesDrillBinding: Codable, Equatable, Sendable { - public var open: FilesExcursion - public var openWith: FilesExcursion - public var discard: FilesExcursion - - public init(open: FilesExcursion, openWith: FilesExcursion, discard: FilesExcursion) { - self.open = open - self.openWith = openWith - self.discard = discard - } - - /// Today's behavior: lift = open, +1-finger lift = Open-With, four-finger horizontal = discard. - public static let `default` = FilesDrillBinding( - open: .lift, openWith: .plusOneFingerLift, discard: .fourFingerHorizontal - ) - - /// The excursion currently bound to `action`. - public func excursion(for action: FilesAction) -> FilesExcursion { - switch action { - case .open: return open - case .openWith: return openWith - case .discard: return discard - } - } - - /// Return a renormalized binding that maps `action → excursion`, swapping with whichever action - /// currently holds `excursion` so the result stays one-to-one. Pure; `self` is unchanged. - public func assigning(_ excursion: FilesExcursion, to action: FilesAction) -> Self { - let previous = self.excursion(for: action) - guard previous != excursion else { return self } - var result = self - for other in FilesAction.allCases - where other != action && result.excursion(for: other) == excursion { - result.set(previous, for: other) - } - result.set(excursion, for: action) - return result - } - - private mutating func set(_ excursion: FilesExcursion, for action: FilesAction) { - switch action { - case .open: open = excursion - case .openWith: openWith = excursion - case .discard: discard = excursion - } - } - } -} - // MARK: - Window switcher extension GestureBindings { diff --git a/Sources/ThreeFingerSwitcher/Gesture/GesturePose.swift b/Sources/ThreeFingerSwitcher/Gesture/GesturePose.swift index f8dc6be..3a6b347 100644 --- a/Sources/ThreeFingerSwitcher/Gesture/GesturePose.swift +++ b/Sources/ThreeFingerSwitcher/Gesture/GesturePose.swift @@ -1,7 +1,7 @@ import CoreGraphics import Foundation -/// A pure, MLX-free ghost-hand pose generator — the "self-playing clip" engine behind both the +/// A pure ghost-hand pose generator — the "self-playing clip" engine behind both the /// First Touch wizard's attract loop and the Hub's gesture previews. Given a continuous `phase` /// it returns one frame: a centroid plus a hand-like arc of fingertips, all normalized to 0..1 /// trackpad space and clamped to `[0.05, 0.95]` (so the hand never leaves the pad). Generalized @@ -180,24 +180,23 @@ enum GesturePose { } } - /// The in-surface gesture a band journey (or a canvas resolve) ends on / demonstrates — a directed - /// excursion in one of four cardinal directions, or a `.lift` (rest-and-open, no excursion). Mirrors - /// the `HubGesturePreview.BandInSurfaceGesture` semantics, lifted into MLX-free Core so the pose - /// driver can build directed strokes from it. `.swipeHorizontal` is the dismiss-style horizontal - /// excursion (a left-going stroke), kept as an alias of the canvas-dismiss default for parity with - /// the existing Hub vocabulary. + /// The in-surface gesture a band journey ends on / demonstrates — a directed excursion in one of + /// four cardinal directions, or a `.lift` (rest-and-open, no excursion). Mirrors the + /// `HubGesturePreview.BandInSurfaceGesture` semantics, lifted into Core so the pose driver can + /// build directed strokes from it. `.swipeHorizontal` is the dismiss-style horizontal excursion + /// (a left-going stroke), kept for parity with the existing Hub vocabulary. enum BandInSurfaceGesture: Equatable { - /// Rest on the band and lift (Files / Clipboard land-and-open) — no directional excursion. + /// Rest on the band and lift (Clipboard land-and-open) — no directional excursion. case lift - /// A downward two-finger resolve (the canvas commit default: top-middle → center-middle). + /// A downward two-finger excursion (top-middle → center-middle). case swipeDown - /// An upward two-finger resolve (the canvas ignore default). + /// An upward two-finger excursion. case swipeUp - /// A leftward two-finger resolve. + /// A leftward two-finger excursion. case swipeLeft - /// A rightward two-finger resolve. + /// A rightward two-finger excursion. case swipeRight - /// A horizontal two-finger resolve (the canvas dismiss default) — a leftward stroke. + /// A horizontal two-finger excursion — a leftward stroke. case swipeHorizontal } @@ -361,15 +360,6 @@ enum GesturePose { return DemoGesture(strokes: strokes, liftGap: 0.5) } - /// A standalone **canvas resolve** demonstration: a single decisive **two-finger** directed swipe in `dir` - /// (e.g. `.swipeDown` = top-middle → center-middle), carrying the hand angle/bow, then a lift and a loop. - nonisolated static func canvasResolve(_ dir: BandInSurfaceGesture) -> DemoGesture { - let center = CGPoint(x: 0.5, y: 0.5) - let stroke = inSurfaceStroke(dir, at: center.x, mid: center.y) - ?? Stroke(fingers: 2, from: center, to: center) - return DemoGesture(strokes: [stroke], liftGap: 0.6) - } - /// The directed two-finger stroke for an in-surface excursion, anchored at the landing column `land` /// (X) and resting row `mid` (Y). `.lift` has no directional stroke (returns nil — the journey simply /// rests on the band). Directions stroke from one edge of the safe pad toward the centre/landing. diff --git a/Sources/ThreeFingerSwitcher/Gesture/GestureRecognizer.swift b/Sources/ThreeFingerSwitcher/Gesture/GestureRecognizer.swift index dd143a2..cdfb54a 100644 --- a/Sources/ThreeFingerSwitcher/Gesture/GestureRecognizer.swift +++ b/Sources/ThreeFingerSwitcher/Gesture/GestureRecognizer.swift @@ -46,38 +46,6 @@ protocol GestureRecognizerDelegate: AnyObject { /// and the finer item-step to in-grid item movement — so the two can be tuned independently. /// Defaults to false. func launcherFocusIsOnBandList() -> Bool - /// While the launcher's AI preview canvas is open, a fresh four-finger swipe RESOLVES it instead of - /// navigating: `dx != 0` is a horizontal swipe (discard); `dy` is vertical (`+1` up, `-1` down) — a - /// DOWN swipe applies the result ("bring it into the document"). Emitted once per gesture, only - /// while `launcherCanvasResolutionActive`. - func launcherCanvasResolve(dx: Int, dy: Int) - /// While a notch conversation is EXPANDED, a fresh two-finger FLICK resolves it - /// (`notch-conversation-gestures`): `dy == +1` (fast up) minimizes it into the notch dock, - /// `dx == +1` (fast right) purge-deletes the session; fast down/left are reserved no-ops at the - /// consumer. Soft scrubs never emit (the D4 classifier ignores them). Emitted at most once per - /// excursion, only while `notchConversationActive`. - func notchConversationResolve(dx: Int, dy: Int) - - // MARK: Files-drill intents (emitted only while `filesDrillActive`; the controller drives entry/exit). - // The recognizer emits pure intents — directory navigation, preview, search-field focus (an up-step - // while already at the top of the list), arm, and fire all live in the model/controller, not here. - // Default no-op implementations are provided below so non-Files conformers compile unchanged. - - /// While the Files column navigator is open, horizontal travel past one item-step steps the depth - /// (descend / ascend), already direction-adjusted (`+1` / `-1`). The controller maps it to descend - /// the current folder / ascend to the parent. - func filesDepth(_ direction: Int) - /// Vertical travel past one item-step moves the highlighted entry, already direction-adjusted - /// (`+1` / `-1`). An up-step while already clamped at the top of the list is interpreted by the - /// controller/model as focus-search; the recognizer just keeps emitting steps. - func filesHighlight(_ direction: Int) - /// The resolving lift with no added finger: open the highlighted entry (default action). One-shot. - func filesOpen() - /// The resolving lift after a relative +1 finger was added: Open-With the highlighted entry. One-shot. - func filesOpenWith() - /// A fresh deliberate four-finger horizontal swipe-away while drilled: discard (defuse a held open / - /// dismiss the navigator). One-shot. - func filesDiscard() } extension GestureRecognizerDelegate { @@ -88,13 +56,6 @@ extension GestureRecognizerDelegate { func launcherDidCancel() {} func launcherEdgeChanged(dx: Int, dy: Int) {} func launcherFocusIsOnBandList() -> Bool { false } - func launcherCanvasResolve(dx: Int, dy: Int) {} - func notchConversationResolve(dx: Int, dy: Int) {} - func filesDepth(_ direction: Int) {} - func filesHighlight(_ direction: Int) {} - func filesOpen() {} - func filesOpenWith() {} - func filesDiscard() {} } /// Multi-finger scrub state machine. The active finger count is **latched at gesture start**: @@ -133,57 +94,6 @@ final class GestureRecognizer { /// have actually been freed). When false, four fingers behave exactly as before. var launcherEnabled = false - /// While true (the launcher's AI preview canvas is open), a fresh four-finger swipe is interpreted - /// as a one-shot canvas RESOLUTION (horizontal = discard, down = apply) via `launcherCanvasResolve`, - /// bypassing the normal launcher/switcher latch. The coordinator sets it from the canvas state. - var launcherCanvasResolutionActive = false - private var canvasResResolved = false - /// The per-excursion D4 flick state for the canvas resolve — the shared classifier - /// (`FlickExcursionClassifier`), one instance per surface so excursions never cross-talk. - private var canvasFlick = FlickExcursionClassifier() - - /// While true (a notch conversation is EXPANDED — `notch-conversation-gestures`), a fresh - /// **two-finger** flick resolves the conversation via `notchConversationResolve` (up = minimize to - /// the dock, right = purge-delete; soft scrubs classify as nothing and scroll natively). Unlike the - /// canvas mode this NEVER swallows the wider grammar: it watches two-finger excursions only, begins - /// them only while the normal machine is idle (a switcher relaxed to two fingers keeps its frames), - /// and a 2→3+ morph resets the tracker and hands the frame straight to the normal latch — the - /// switcher/launcher stay fully usable while a chat is open. The coordinator sets it from - /// `ParkController.onExpandedChanged`. - var notchConversationActive = false { - didSet { if notchConversationActive != oldValue { notchFlick.reset() } } - } - private var notchFlick = FlickExcursionClassifier() - - /// While true (the Files column navigator is open), every frame routes to `trackFilesDrill` and the - /// normal finger-count latch is bypassed — a fresh contact during the drill never opens the switcher - /// or a second launcher. The controller flips it from the navigator's open/close state (mirroring - /// `launcherCanvasResolutionActive`). Setting it `true` re-seeds a fresh drill session; while `false` - /// (the default + all non-Files use) everything below is byte-identical to before. - var filesDrillActive = false { - didSet { if filesDrillActive && !oldValue { resetDrill() } } - } - /// Whether the current drill session has seeded its baseline yet (a fresh contact seeds it). Cleared - /// on entry and on a true lift so the next contact re-seeds the origin. - private var drillStarted = false - /// Set once the session has resolved (open / open-with / discard). The resolution is **one-shot** for - /// the whole session: while set, no further intent is emitted, so a stray re-lift is a no-op. Cleared - /// only when the controller re-enters the sub-state. - private var drillResolved = false - /// Set once a relative +1 finger (a contact above the current relaxed baseline) is seen, so the - /// resolving lift emits Open-With instead of a plain Open. Latched for the session. - private var pendingOpenWith = false - /// Relaxed contact baseline of the drill, re-baselined on every contact-count change (the gesture - /// lives while ≥2 remain, so the user may relax fingers). A count rising ABOVE this is the relative - /// +1 Open-With morph (D4) — not an absolute three. - private var drillContacts = 0 - /// Reference origin for the drill, re-baselined (with the accumulators cleared) on every contact-count - /// change so a leaving or landing finger's centroid shift emits no spurious step. - private var drillStart = CGPoint.zero - private var drillLast = CGPoint.zero - private var drillAccumX: CGFloat = 0 // horizontal → depth steps - private var drillAccumY: CGFloat = 0 // vertical → highlight steps - private enum Axis { case undetermined, horizontal, vertical } private enum State { case idle, tracking } private enum Mode { case switcher, launcher } @@ -202,8 +112,7 @@ final class GestureRecognizer { private var launcherContacts = 0 /// Edge-hold state for auto-repeat, per axis (−1 / 0 / +1): `edgeDX +1` = right edge, `edgeDY +1` = /// top edge. Emitted to the delegate (`launcherEdgeChanged`) when either changes; the controller drives - /// the edge-triggered auto-repeat off it. Shared by the launcher and the Files drill (mutually exclusive - /// sub-states). + /// the edge-triggered auto-repeat off it. private var edgeDX = 0 private var edgeDY = 0 /// Normalized distance from a trackpad edge within which a held contact triggers auto-repeat, with @@ -228,44 +137,11 @@ final class GestureRecognizer { /// Control / App Exposé — larger than axis detection so it isn't twitchy. private let missionControlThreshold: CGFloat = 0.10 - /// Deliberate travel (normalized) before a fresh TWO-finger swipe resolves the AI canvas (change - /// `positional-navigation`, D5). Deliberately **larger than incidental two-finger scrolling** so - /// reading/scrolling the canvas is never mistaken for a commit/discard. - private let canvasResolveThreshold: CGFloat = 0.12 - - /// Peak smoothed centroid speed (normalized/sec) the dominant axis must reach for a canvas excursion's - /// lift to count as a FLICK rather than a reading-scroll (D4). Sourced from `settings.flickVelocityThreshold`. - private var flickVelocityThreshold: CGFloat { CGFloat(settings.flickVelocityThreshold) } - /// Maximum gap (seconds) between the last high-velocity in-contact frame and the lift for that lift to - /// count as a flick (D4). Sourced from `settings.flickLiftWindow`. - private var flickLiftWindow: CFTimeInterval { CFTimeInterval(settings.flickLiftWindow) } - init(settings: AppSettings) { self.settings = settings } func feed(_ frame: TouchFrame) { - // While the launcher's AI preview canvas is open, a fresh four-finger swipe RESOLVES it - // (horizontal = discard, down = apply) — bypassing the normal launcher/switcher latch. When the - // flag is off (the default + all normal use), everything below is byte-identical to before. - if launcherCanvasResolutionActive { - trackCanvasResolution(frame) - return - } - // While the Files column navigator is open, route every frame to the sustained drill tracker - // BEFORE the idle re-latch below, so a fresh contact during drill-in never opens the switcher or - // a second launcher on top of the navigator. Off by default → byte-identical to before. - if filesDrillActive { - trackFilesDrill(frame) - return - } - // While a notch conversation is EXPANDED (`notch-conversation-gestures`), watch two-finger flick - // excursions — but never at the wider grammar's expense: an excursion may BEGIN only while the - // normal machine is idle (a switcher relaxed to two fingers keeps every frame), and any frame the - // handler declines (0/1/3/4 fingers, or a 2→3+ morph) falls straight through to the machine below. - if notchConversationActive, notchFlick.started || state == .idle { - if handleNotchConversationFrame(frame) { return } - } let switcherTarget = settings.requireExactlyThree ? (frame.fingerCount == 3) : (frame.fingerCount >= 3) switch state { @@ -299,89 +175,6 @@ final class GestureRecognizer { } } - /// One-shot canvas-resolution tracking (see `launcherCanvasResolutionActive`). A fresh **two-finger** - /// FLICK-LIFT reports a single `launcherCanvasResolve`: vertical-dominant → `dy` (`+1` up, `-1` down; - /// down applies, up is ignored / parks upstream), else horizontal → `dx` (discard). Two-finger - /// resolution (change `positional-navigation`, D5) aligns the grammar — 4 fingers open / dismiss the - /// platform, 2 fingers act within it. Runs INSTEAD of the normal state machine while the canvas is open, - /// so it never opens the launcher or switcher; it self-resets on lift. - /// - /// D4 (scroll-vs-flick): the resolve does NOT fire the instant travel crosses `canvasResolveThreshold` — - /// that floor is a MINIMUM only. While fingers are down we accumulate the signed dominant-axis travel, a - /// running PEAK of `abs(centroidVelocity)` per axis, and the timing of the last high-velocity frame - /// (`frame.centroidVelocity`/`frame.time` from the engine — the lift frame is empty with `.zero` - /// velocity, so the flick speed must come from the LAST in-contact frame). On the lift frame we classify: - /// a FLICK (→ emit) requires (a) the travel floor was crossed, (b) the dominant axis's peak velocity - /// exceeded `flickVelocityThreshold`, AND (c) the lift arrived within `flickLiftWindow` of the last - /// high-velocity frame. A slow continuous scrub (sub-threshold peak, or held without a prompt lift) is - /// SCROLL and emits nothing. - private func trackCanvasResolution(_ frame: TouchFrame) { - let count = frame.fingerCount - if count == 0 { // lift → classify the just-ended excursion, then re-arm - if canvasFlick.started, !canvasResResolved, - let flick = canvasFlick.classifyOnLift(travelFloor: canvasResolveThreshold, - velocityThreshold: flickVelocityThreshold, - liftWindow: flickLiftWindow, - axisLockRatio: CGFloat(settings.axisLockRatio)) { - canvasResResolved = true - delegate?.launcherCanvasResolve(dx: flick.dx, dy: flick.dy) - } - canvasFlick.reset() - canvasResResolved = false - return - } - if !canvasFlick.started { - guard count >= 2 else { return } // require a fresh (≥) two-finger contact to begin - canvasResResolved = false - canvasFlick.begin(at: frame.centroid, time: frame.time) - return - } - guard !canvasResResolved, count >= 2 else { return } - // Accumulate travel + per-axis velocity peaks + last-fast-frame timing in the shared classifier. - // `canvasResolveThreshold` stays a travel FLOOR only — nothing ever fires mid-contact. - canvasFlick.track(centroid: frame.centroid, velocity: frame.centroidVelocity, time: frame.time, - velocityThreshold: flickVelocityThreshold, - axisLockRatio: CGFloat(settings.axisLockRatio)) - } - - // MARK: - Notch conversation flick (two-finger only, falls through — `notch-conversation-gestures`) - - /// Route one frame while a notch conversation is expanded. Returns true when the frame was consumed - /// by the flick tracker; false hands the SAME frame to the normal machine (fall-through). Routing: - /// a started excursion's lift classifies + emits one-shot `notchConversationResolve`; a 2→3+ morph - /// resets and falls through (the user is growing a switcher/launcher gesture out of a scroll — that - /// must win); an un-started tracker begins only on exactly two fingers (and only from the machine's - /// idle state — the caller gates that), so 0/1/3/4-finger frames always belong to the normal grammar. - private func handleNotchConversationFrame(_ frame: TouchFrame) -> Bool { - let count = frame.fingerCount - if notchFlick.started { - if count == 0 { // lift → classify, emit at most once, re-arm - if let flick = notchFlick.classifyOnLift(travelFloor: canvasResolveThreshold, - velocityThreshold: flickVelocityThreshold, - liftWindow: flickLiftWindow, - axisLockRatio: CGFloat(settings.axisLockRatio)) { - delegate?.notchConversationResolve(dx: flick.dx, dy: flick.dy) - } - notchFlick.reset() - return true - } - if count >= 3 { // growing contact → the wider grammar owns this frame - notchFlick.reset() - return false - } - if count == 2 { - notchFlick.track(centroid: frame.centroid, velocity: frame.centroidVelocity, - time: frame.time, - velocityThreshold: flickVelocityThreshold, - axisLockRatio: CGFloat(settings.axisLockRatio)) - } - return true // a transient 1-finger dip is ignored, state retained - } - guard count == 2 else { return false } - notchFlick.begin(at: frame.centroid, time: frame.time) - return true - } - // MARK: - Switcher (three-finger; relaxes to two after activation) private func trackSwitcher(_ frame: TouchFrame) { @@ -711,173 +504,4 @@ final class GestureRecognizer { activated = false if wasActivated { delegate?.launcherDidCancel() } } - - // MARK: - Files drill (sustained modal sub-state; bypasses the latch while the navigator is open) - - /// Re-seed a fresh drill session. Called when the controller ENTERS the sub-state (the `didSet` - /// false→true edge); the next contact re-baselines the origin. A truly one-shot resolution means - /// `drillResolved` survives a lift WITHIN a session and only clears here, on the next entry. - private func resetDrill() { - drillStarted = false - drillResolved = false - pendingOpenWith = false - drillContacts = 0 - belowTargetFrames = 0 - drillAccumX = 0 - drillAccumY = 0 - clearEdges() // no held auto-repeat carried into a fresh drill session - } - - /// Re-arm the SAME drill session for a fresh resolution **without** toggling `filesDrillActive`. After - /// the +1-finger lift resolved the drill (`filesOpenWith`, one-shot — `drillResolved` latched), the - /// Open-With picker opens and needs fresh gesture input to scrub it: this clears the one-shot resolution - /// latch and re-seeds the drill scalars (exactly as a fresh entry would) so the next contact re-baselines - /// and navigation resumes. It is NOT a re-entry — `filesDrillActive` stays true throughout — so the - /// controller drives it explicitly right after entering the picker (mirroring how a re-entry would seed, - /// but in place). The lift that opens the picker has already raised the fingers, so seeding from scratch - /// here is correct: the picker is scrubbed by a brand-new gesture. - func rearmDrill() { - resetDrill() - } - - /// Sustained drill tracking (see `filesDrillActive`). Unlike the one-shot canvas tracker this lives - /// for the whole session while ≥2 contacts remain, emitting many depth/highlight steps. The origin is - /// re-baselined (and carry cleared) on EVERY contact-count change so a leaving/landing finger emits no - /// phantom step. Navigation (depth/highlight) happens at the relaxed posture (≤3 contacts); a FULL - /// four-finger contact is the resolution-arming posture — a deliberate horizontal swipe-away there is a - /// discard, a plain lift is an open (Open-With if a relative +1 finger was added). The resolving lift - /// (count below two, with the standard below-target debounce) is a ONE-SHOT resolution; a stray re-lift - /// after that emits nothing. - private func trackFilesDrill(_ frame: TouchFrame) { - let count = frame.fingerCount - - if count >= 2 { - belowTargetFrames = 0 - if !drillStarted { - drillStarted = true - drillContacts = count - drillStart = frame.centroid // baseline for the 4-finger discard swipe - drillLast = frame.centroid - drillAccumX = 0 - drillAccumY = 0 - return - } - // A contact-count change shifts the centroid as fingers leave or land. Re-baseline the origin - // (and clear carry) so the jump emits no step, and stop any held auto-repeat across the - // re-baseline; a count rising ABOVE the relaxed baseline is the relative +1 Open-With morph - // (latched for the lift). The baseline then follows the count. While resolved, the session is inert. - if count != drillContacts { - if count > drillContacts && !drillResolved { pendingOpenWith = true } - drillContacts = count - drillStart = frame.centroid - drillLast = frame.centroid - drillAccumX = 0 - drillAccumY = 0 - clearEdges() - return - } - guard !drillResolved else { return } - updateFilesDrill(frame) - } else { - // Below two contacts ("lift"): a real lift reports 0 immediately; an edge flicker dips to 1 for - // a frame or two. Resolve on a true lift or a sustained drop (the same debounce as the launcher). - belowTargetFrames += 1 - if count == 0 || belowTargetFrames >= 2 { - resolveFilesDrillLift() - } - } - } - - private func updateFilesDrill(_ frame: TouchFrame) { - let c = frame.centroid - - // A full four-finger contact is the resolution-arming posture (the +1 morph past the navigation - // postures). It does NOT navigate — instead a fresh deliberate horizontal swipe-away past the - // activation threshold is a one-shot DISCARD (mirroring the canvas-resolution swipe). A plain lift - // from here resolves Open-With (the +1 latch). Measuring from the re-baselined `drillStart` (set - // when the 4th finger landed) means a small depth-sized nudge won't trip it; a deliberate sweep will. - if drillContacts >= 4 { - let dx = c.x - drillStart.x - let dy = c.y - drillStart.y - let threshold = CGFloat(settings.launcherActivationThreshold) - let ratio = CGFloat(settings.axisLockRatio) - if abs(dx) >= threshold && abs(dx) >= ratio * abs(dy) { - drillResolved = true - // The physical four-finger horizontal swipe-away → its bound action - // (`add-gesture-previews-and-bindings` §9.4; default: discard). - resolveFilesDrillExcursion(.fourFingerHorizontal) - } - drillLast = c - return - } - - // Relaxed navigation posture (≥2, ≤3 contacts): ODOMETER (restored v0.11.0 model). Accumulate signed - // travel and emit discrete steps with carry — HIGHLIGHT (vertical) moves the selection, DEPTH - // (horizontal) descends/ascends folders. Unlike the launcher's flat grid a horizontal step here - // mutates the folder stack, so holding at the trackpad edge AUTO-DRILLS through the tree (uniform - // edge auto-repeat on BOTH axes — the user opted into this). - drillAccumX += (c.x - drillLast.x) - drillAccumY += (c.y - drillLast.y) - drillLast = c - - let step = CGFloat(max(settings.launcherStepDistance, 0.005)) - while drillAccumX >= step { drillAccumX -= step; emitDrillDepth(forward: true) } - while drillAccumX <= -step { drillAccumX += step; emitDrillDepth(forward: false) } - while drillAccumY >= step { drillAccumY -= step; emitDrillHighlight(up: true) } - while drillAccumY <= -step { drillAccumY += step; emitDrillHighlight(up: false) } - - updateEdges(c) // both axes auto-repeat at the edge: highlight (vertical) + depth auto-drill (horizontal) - } - - /// The resolving lift: a one-shot Open-With (if a relative +1 finger was added) or plain Open. A lift - /// after the session already resolved (e.g. a stray re-lift, or a four-finger discard) emits nothing. - private func resolveFilesDrillLift() { - clearEdges() // the lift stops any held highlight auto-repeat - guard !drillResolved else { - drillStarted = false - return - } - // Only resolve a session that actually started (a sub-threshold flicker before any contact does - // nothing); seeding requires a real ≥2-finger contact. - if drillStarted { - drillResolved = true - // The physical lift → its bound action: a relative +1-finger lift is `.plusOneFingerLift`, a - // plain lift is `.lift` (`add-gesture-previews-and-bindings` §9.4; defaults: Open-With / open). - resolveFilesDrillExcursion(pendingOpenWith ? .plusOneFingerLift : .lift) - } - drillStarted = false - } - - /// Map a DETECTED physical Files-drill excursion to its bound action via the user's configured - /// `filesDrill` binding (`add-gesture-previews-and-bindings` §9.4), then fire the matching delegate - /// intent. The recognizer detects only the *physical* move; which intent it carries is the binding's - /// job — defaults reproduce today's grammar (lift → open, +1-finger lift → Open-With, four-finger - /// horizontal → discard). All downstream guards (one-shot `drillResolved`, the defuse window, and - /// "discard never terminates a running app") live in the delegate/controller and are untouched. - private func resolveFilesDrillExcursion(_ excursion: GestureBindings.FilesExcursion) { - let binding = settings.gestureBindings.filesDrill - // Find which action is bound to the detected excursion (the binding is strictly one-to-one). - let action = GestureBindings.FilesAction.allCases.first { - binding.excursion(for: $0) == excursion - } - switch action { - case .open: delegate?.filesOpen() - case .openWith: delegate?.filesOpenWith() - case .discard: delegate?.filesDiscard() - case .none: break // unreachable: every excursion is bound to exactly one action - } - } - - private func emitDrillDepth(forward: Bool) { - var dir = forward ? 1 : -1 - if settings.reverseDirection { dir = -dir } - delegate?.filesDepth(dir) - } - - private func emitDrillHighlight(up: Bool) { - var dir = up ? 1 : -1 - if settings.reverseVerticalDirection { dir = -dir } - delegate?.filesHighlight(dir) - } - } diff --git a/Sources/ThreeFingerSwitcher/Gesture/LauncherTourEngine.swift b/Sources/ThreeFingerSwitcher/Gesture/LauncherTourEngine.swift index 825cec4..331c6e6 100644 --- a/Sources/ThreeFingerSwitcher/Gesture/LauncherTourEngine.swift +++ b/Sources/ThreeFingerSwitcher/Gesture/LauncherTourEngine.swift @@ -1,6 +1,6 @@ import CoreGraphics -/// The pure, MLX-free **launcher tour brain** (§13 / design D12) — one settings-driven odometer that turns a +/// The pure **launcher tour brain** (§13 / design D12) — one settings-driven odometer that turns a /// stream of trackpad frames (finger count + normalized centroid) into discrete launcher intents, so the Hub /// launcher/band previews navigate exactly like the real launcher and the wizard tour. It folds the three /// today-triplicated copies of this logic (`GestureRecognizer`'s launcher activation/step math, the wizard's diff --git a/Sources/ThreeFingerSwitcher/Hub/BandsCanvas.swift b/Sources/ThreeFingerSwitcher/Hub/BandsCanvas.swift index ba7e14b..7961103 100644 --- a/Sources/ThreeFingerSwitcher/Hub/BandsCanvas.swift +++ b/Sources/ThreeFingerSwitcher/Hub/BandsCanvas.swift @@ -51,7 +51,6 @@ private enum SourceCategory: String, CaseIterable, Identifiable { case scripts = "Scripts" case actions = "Actions" case automations = "Automations" - case aiCommands = "AI Command" case claudeProject = "Claude Project" case terminal = "Open in Terminal" case claudeProjectPrompt = "Claude (Pick Folder)" @@ -67,7 +66,6 @@ private enum SourceCategory: String, CaseIterable, Identifiable { case .scripts: return "terminal.fill" case .actions: return "bolt.horizontal.fill" case .automations: return "gearshape.2.fill" - case .aiCommands: return "wand.and.stars" case .claudeProject: return "sparkles" case .terminal: return "terminal" case .claudeProjectPrompt: return "sparkles" @@ -79,7 +77,7 @@ private enum SourceCategory: String, CaseIterable, Identifiable { var hint: String { switch self { case .urls, .scripts, .paths, .claudeProject, .terminal, .claudeProjectPrompt, .terminalPrompt: return "Add" - case .apps, .shortcuts, .actions, .automations, .aiCommands, .presets: return "Browse" + case .apps, .shortcuts, .actions, .automations, .presets: return "Browse" } } } @@ -244,7 +242,6 @@ private struct SourcePicker: View { case .shortcuts: ShortcutBrowser { add($0) } case .actions: ActionBrowser { add($0) } case .automations: AutomationBrowser { add($0) } - case .aiCommands: AICommandSource(store: store) { add($0) } case .presets: PresetComposer(store: store) { add($0) } // Immediate-add sources never drill in (handled by `activate`); never reached. case .urls, .scripts, .paths, .claudeProject, .terminal, .claudeProjectPrompt, .terminalPrompt: EmptyView() @@ -425,81 +422,6 @@ private struct AutomationBrowser: View { } } -/// Source for AI commands: a CATALOG BROWSER over `AICommandCatalog`, mirroring `ActionBrowser` — a -/// `List` with one `Section` per `Category`, each preset a row that adds a fresh copy (`copy(of:)` -/// mints a new id) to the active band and auto-selects it so its inspector (right pane) opens for -/// editing. Each section header carries an "Add all as a band" affordance that creates a new band -/// named after the category (carrying its color) populated with that category's presets. A trailing -/// "Custom command" entry adds the blank editable command. Authoring lives inline in the item -/// inspector (configuration-hub fold-in; the standalone AI-command editor is gone). -private struct AICommandSource: View { - @ObservedObject var store: FavoritesStore - let onPick: (LaunchItem) -> Void - - var body: some View { - ScrollView { - LazyVStack(alignment: .leading, spacing: 12) { - ForEach(AICommandCatalog.Category.allCases) { category in - HStack { - Label(category.title, systemImage: category.sfSymbol).font(.caption) - Spacer() - Button("Add all as a band") { addCategoryAsBand(category) } - .buttonStyle(.borderless) - .controlSize(.small) - .font(.caption) - .help("Create a new \"\(category.title)\" band populated with these presets.") - } - .padding(.horizontal, 12).padding(.top, 4) - LazyVGrid(columns: sourceGridColumns, spacing: 12) { - ForEach(AICommandCatalog.commands(in: category)) { preset in - Button { - onPick(AIBand.item(for: AICommandCatalog.copy(of: preset))) - } label: { - GridTile(title: preset.name, subtitle: "AI Command") { SourceSymbol(name: symbolName(preset.icon)) } - } - .buttonStyle(PickTileButtonStyle()) - .help(preset.promptTemplate) - } - } - .padding(.horizontal, 12) - } - LazyVGrid(columns: sourceGridColumns, spacing: 12) { - Button { - let cmd = AICommand(name: "New Command", icon: .sfSymbol("wand.and.stars"), - inputs: AICommand.defaultInputs, promptTemplate: "{input}", - outputs: AICommand.defaultOutputs) - onPick(AIBand.item(for: cmd)) - } label: { - GridTile(title: "Custom command", subtitle: "AI Command") { SourceSymbol(name: "wand.and.stars") } - } - .buttonStyle(PickTileButtonStyle()) - .help("Add a blank AI command, then edit its prompt, input, and output on the right.") - } - .padding(.horizontal, 12) - } - .padding(.vertical, 8) - } - } - - /// Create a new band named after the category (carrying its color), populated with that category's - /// presets — each a fresh copy (`copy(of:)`). Appending is correct even if a same-named band exists - /// (no dedupe/merge). Does not require the AI opt-in. - private func addCategoryAsBand(_ category: AICommandCatalog.Category) { - let items = AICommandCatalog.commands(in: category) - .map { AIBand.item(for: AICommandCatalog.copy(of: $0)) } - let band = ContextBand(name: category.title, color: category.tint, - icon: .sfSymbol(category.sfSymbol), items: items) - store.mutate { $0.bands.append(band) } - } - - /// The SF Symbol name behind a preset's `ItemIcon` (presets are always `.sfSymbol`; fall back to the - /// AI glyph for any non-symbol icon a future preset might carry). - private func symbolName(_ icon: ItemIcon) -> String { - if case let .sfSymbol(name) = icon { return name } - return "wand.and.stars" - } -} - private struct PresetComposer: View { @ObservedObject var store: FavoritesStore let onPick: (LaunchItem) -> Void @@ -880,7 +802,6 @@ private struct ItemInspector: View { let item: LaunchItem @Binding var autoFocusItemID: UUID? @State private var title: String - @State private var prompt: String /// Local edit buffer for the link address (a `.url` stores a `URL`, so we commit only when it parses). @State private var address: String /// Local edit buffer for an inline script body (shell / AppleScript). @@ -898,11 +819,6 @@ private struct ItemInspector: View { self.store = store; self.bandID = bandID; self.item = item _autoFocusItemID = autoFocusItemID _title = State(initialValue: item.title) - if case let .aiCommand(cmd) = item.kind { - _prompt = State(initialValue: cmd.promptTemplate) - } else { - _prompt = State(initialValue: "") - } if case let .url(u, _, _) = item.kind, u != blankLinkURL { _address = State(initialValue: u.absoluteString) } else { @@ -938,7 +854,7 @@ private struct ItemInspector: View { } var body: some View { - if liveAICommand != nil { aiForm } else { standardForm } + standardForm } // MARK: Standard item form (app / file / url / shortcut / script / action / preset) @@ -1492,318 +1408,6 @@ private struct ItemInspector: View { store.updateItem(item.id, inBand: bandID) { $0.kind = .app(bundleURL: url, strategy: strategy) } } } - - // MARK: - AI command editing (the embedded `.aiCommand` is authoritative; display fields mirror it) - - /// The live embedded command for this item, re-read from the store each render so structural edits - /// reflect immediately. `nil` when this item isn't an AI command (→ the standard form is shown). - private var liveAICommand: AICommand? { - guard let b = store.favorites.bands.first(where: { $0.id == bandID }), - let it = b.items.first(where: { $0.id == item.id }), - case let .aiCommand(cmd) = it.kind else { return nil } - return cmd - } - - /// A non-optional view of the command for the editors (the fallback is never rendered — `aiForm` - /// only appears when `liveAICommand != nil`). - private var ai: AICommand { - liveAICommand ?? AICommand(name: "", icon: .sfSymbol("wand.and.stars"), - inputs: AICommand.defaultInputs, promptTemplate: "", - outputs: AICommand.defaultOutputs) - } - - /// Apply an edit to the embedded command and persist it, mirroring name/icon/tint onto the - /// `LaunchItem`'s display fields so the grid and launcher render the command correctly. - private func updateCommand(_ block: (inout AICommand) -> Void) { - guard var cmd = liveAICommand else { return } - block(&cmd) - store.updateItem(item.id, inBand: bandID) { - $0.kind = .aiCommand(cmd) - $0.title = cmd.name - $0.icon = cmd.icon - $0.tint = cmd.tint - } - } - - private func insertToken(_ token: String) { - prompt += token - updateCommand { $0.promptTemplate = prompt } - } - - /// Whether any band other than this item's exists (gates the "Move to band" control + its divider). - private var hasOtherBands: Bool { store.favorites.bands.contains { $0.id != bandID } } - - /// A compact "Title: control" field — a `.labelsHidden()` picker sits right next to its title so - /// two can share a row without the Form's full-width label/value spread. - private func aiField(_ title: String, @ViewBuilder _ content: () -> Content) -> some View { - HStack(spacing: 6) { - Text("\(title):").foregroundStyle(.secondary) - content() - } - } - - private var aiForm: some View { - ScrollView { - // One unified container holds the whole command configuration: identity, input, prompt, - // output, model, reasoning, confirmation, and move — grouped by light inline subheaders - // instead of separate boxed Sections, so the inspector reads as a single item card. - Form { - Section { - HStack(spacing: 10) { - IconColorControl( - icon: Binding(get: { ai.icon }, set: { ic in updateCommand { $0.icon = ic } }), - tint: Binding(get: { ai.tint }, set: { t in updateCommand { $0.tint = t } }), - naturalIcon: nil) { - IconGlyphView(icon: ai.icon, tint: ai.tint, size: 20) - } - TextField("Name", text: $title) - .onChange(of: title) { updateCommand { $0.name = title } } - } - - // Sources + Result are CAPABILITY sets (change `ai-action-context-resolution`): the - // action senses the live environment at fire and resolves BOTH the input channel and - // the commit from what's enabled here — all-on is the smart default. Toggles narrow it. - aiField("Sources") { aiInputToggles } - Text("The action reads the first available source: your selection, then clipboard text, then a clipboard image. A selection commits as a replace; the clipboard commits as a paste at the cursor.") - .font(.caption).foregroundStyle(.secondary) - HStack(spacing: 18) { - aiField("Result") { aiOutputKindPicker.labelsHidden() } - Spacer(minLength: 0) - } - aiOutputDetail - - AITokenBar { insertToken($0) } - TextEditor(text: $prompt) - .frame(minHeight: 110) - .font(.system(.body, design: .monospaced)) - .border(.quaternary) - .onChange(of: prompt) { updateCommand { $0.promptTemplate = prompt } } - Text("Tokens are substituted at fire time: {input} the acquired text, {date} today, {app} the front app, {url} the front document URL. Unknown braces are left as-is.") - .font(.caption).foregroundStyle(.secondary) - - // Model + Reasoning on one tight row. - HStack(spacing: 18) { - aiField("Model") { aiModelPicker.labelsHidden() } - aiField("Reasoning") { aiReasoningPicker.labelsHidden() } - Spacer(minLength: 0) - } - - Toggle("Confirm before running", isOn: Binding( - get: { ai.confirmBeforeRun }, set: { on in updateCommand { $0.confirmBeforeRun = on } })) - - if hasOtherBands { Divider().padding(.vertical, 2); moveToBandControl } - } - } - .formStyle(.grouped) - .padding(.bottom, 8) - } - .frame(height: 380) - } - - // MARK: - Input capability toggles (design D1/D4) - - /// The input capability toggles: the ambient cascade (selection ▸ clipboard ▸ clipboard image) plus a - /// mutually-exclusive "screen region" (region-first, vision). Enabling screen region clears the ambient - /// set; while it is on, the ambient toggles are disabled. - private var aiInputToggles: some View { - let regionMode = ai.inputs.contains(.screenRegion) - return VStack(alignment: .leading, spacing: 4) { - Toggle(aiInputLabel(.selection), isOn: inputBinding(.selection)).disabled(regionMode) - Toggle(aiInputLabel(.clipboard), isOn: inputBinding(.clipboard)).disabled(regionMode) - Toggle(aiInputLabel(.clipboardImage), isOn: inputBinding(.clipboardImage)).disabled(regionMode) - Toggle(aiInputLabel(.screenRegion), isOn: inputBinding(.screenRegion)) - .help("Captures a screen region instead — exclusive of the other sources.") - } - .toggleStyle(.checkbox) - } - - private func inputBinding(_ s: InputSource) -> Binding { - Binding(get: { ai.inputs.contains(s) }, - set: { on in updateCommand { cmd in Self.setInput(&cmd, s, on) } }) - } - - /// Toggle one input capability, enforcing the screen-region exclusivity (design D4). - static func setInput(_ cmd: inout AICommand, _ s: InputSource, _ on: Bool) { - if on { - if s == .screenRegion { cmd.inputs = [.screenRegion] } // region-first: exclusive - else { cmd.inputs.remove(.screenRegion); cmd.inputs.insert(s) } - } else { - cmd.inputs.remove(s) - } - } - - // MARK: - Output capability (kind + in-place toggles) - - /// The result KIND: an in-place command (the replace/paste/preview toggles below), or a side-effecting - /// task / send-to (its own sub-editor). Derived from whether the outputs carry a side-effecting member. - private var aiOutputKindPicker: some View { - Picker("Result", selection: Binding(get: { outputKind(ai) }, set: { setOutputKind($0) })) { - ForEach(AIOutputKind.allCases) { Text($0.label).tag($0) } - } - } - - /// The in-place output capability toggles: which commit behaviors the action may use (resolved from the - /// input channel at fire — selection→replace, clipboard→paste, else preview). - private var aiInPlaceOutputToggles: some View { - VStack(alignment: .leading, spacing: 4) { - Toggle("Replace selection", isOn: inPlaceOutputBinding(.replaceSelection)) - Toggle("Paste at cursor", isOn: inPlaceOutputBinding(.pasteAtCursor)) - Toggle("Preview only", isOn: inPlaceOutputBinding(.previewOnly)) - } - .toggleStyle(.checkbox) - } - - private func inPlaceOutputBinding(_ o: OutputTarget) -> Binding { - Binding(get: { ai.outputs.contains(o) }, - set: { on in updateCommand { cmd in - if on { cmd.outputs.insert(o) } else { cmd.outputs.remove(o) } - } }) - } - - /// The output's conditional sub-editors: the in-place toggles, or the task/destination editor when the - /// result is a side-effecting task / send-to. - @ViewBuilder - private var aiOutputDetail: some View { - switch ai.sideEffect { - case let .runTask(kind): aiTaskKindEditor(kind) - case let .sendTo(dest): - aiDestinationEditor(dest) { newDest in updateCommand { $0.outputs = [.sendTo(newDest)] } } - default: - aiInPlaceOutputToggles - } - } - - @ViewBuilder - private func aiTaskKindEditor(_ kind: TaskKind) -> some View { - Picker("Task", selection: Binding(get: { taskChoice(kind) }, set: { setTaskChoice($0) })) { - ForEach(TaskChoice.allCases) { Text($0.label).tag($0) } - } - switch kind { - case .addToCalendar: - Text("Parses an event from the result and adds it to Calendar (asks for permission the first time).") - .font(.caption).foregroundStyle(.secondary) - case .addToReminder: - Text("Parses a to-do from the result and adds it to Reminders (asks for permission the first time).") - .font(.caption).foregroundStyle(.secondary) - case .newContact: - Text("Parses contact details from the result and creates a Contacts card (asks for permission the first time).") - .font(.caption).foregroundStyle(.secondary) - case let .saveToProject(project): - TextField("Project", text: Binding( - get: { project }, set: { p in updateCommand { $0.outputs = [.runTask(.saveToProject(project: p))] } })) - case let .openToolWithPayload(tool): - ToolTargetPicker(tool: Binding( - get: { tool }, set: { t in updateCommand { $0.outputs = [.runTask(.openToolWithPayload(tool: t))] } })) - case let .sendTo(dest): - aiDestinationEditor(dest) { newDest in updateCommand { $0.outputs = [.runTask(.sendTo(newDest))] } } - } - } - - @ViewBuilder - private func aiDestinationEditor(_ dest: Destination, onChange: @escaping (Destination) -> Void) -> some View { - Picker("Destination", selection: Binding( - get: { destinationChoice(dest) }, set: { onChange(blankDestination(for: $0, from: dest)) })) { - ForEach(DestinationChoice.allCases) { Text($0.label).tag($0) } - } - switch dest { - case let .shortcut(n): - ShortcutPicker(name: Binding(get: { n }, set: { onChange(.shortcut(name: $0)) })) - case let .urlScheme(s): - TextField("URL scheme (use {content})", text: Binding(get: { s }, set: { onChange(.urlScheme($0)) })) - case let .shell(c): - TextField("Shell command (content on stdin)", text: Binding(get: { c }, set: { onChange(.shell(command: $0)) })) - } - } - - private var aiModelPicker: some View { - let registry = ModelCatalog.standard - return Picker("Model", selection: Binding( - get: { selectedModelID(ai.model) }, set: { id in updateCommand { $0.model = .onDevice(modelID: id) } })) { - Text("Registry default").tag(String?.none) - ForEach(registry.models) { m in Text(m.displayName).tag(Optional(m.id)) } - } - } - - private var aiReasoningPicker: some View { - Picker("Reasoning", selection: Binding( - get: { ai.reasoning }, set: { r in updateCommand { $0.reasoning = r } })) { - Text("Default").tag(AIReasoning?.none) - Text("On").tag(Optional(AIReasoning.on)) - Text("Off").tag(Optional(AIReasoning.off)) - } - } - - private func outputKind(_ c: AICommand) -> AIOutputKind { - switch c.sideEffect { - case .runTask: return .runTask - case .sendTo: return .sendTo - default: return .inPlace - } - } - - /// Switch the result kind. Moving to in-place restores the all-on in-place toggles; moving to a task / - /// send-to seeds a default payload. Crossing the in-place ⇄ side-effect boundary re-derives - /// `confirmBeforeRun` (matching the prior single-picker behavior). - private func setOutputKind(_ kind: AIOutputKind) { - let wasSideEffecting = ai.isSideEffecting - updateCommand { cmd in - switch kind { - case .inPlace: cmd.outputs = AICommand.defaultOutputs - case .runTask: cmd.outputs = [.runTask(.addToCalendar)] - case .sendTo: cmd.outputs = [.sendTo(.shortcut(name: ""))] - } - if wasSideEffecting != cmd.isSideEffecting { - cmd.confirmBeforeRun = AICommand.defaultConfirmBeforeRun(for: cmd.outputs) - } - } - } - - private func taskChoice(_ k: TaskKind) -> TaskChoice { - switch k { - case .addToCalendar: return .addToCalendar - case .addToReminder: return .addToReminder - case .newContact: return .newContact - case .saveToProject: return .saveToProject - case .openToolWithPayload: return .openToolWithPayload - case .sendTo: return .sendTo - } - } - - private func setTaskChoice(_ choice: TaskChoice) { - let kind: TaskKind - switch choice { - case .addToCalendar: kind = .addToCalendar - case .addToReminder: kind = .addToReminder - case .newContact: kind = .newContact - case .saveToProject: kind = .saveToProject(project: "") - case .openToolWithPayload: kind = .openToolWithPayload(tool: "") - case .sendTo: kind = .sendTo(.shortcut(name: "")) - } - updateCommand { $0.outputs = [.runTask(kind)] } - } - - private func destinationChoice(_ d: Destination) -> DestinationChoice { - switch d { - case .shortcut: return .shortcut - case .urlScheme: return .urlScheme - case .shell: return .shell - } - } - - private func blankDestination(for choice: DestinationChoice, from current: Destination) -> Destination { - switch choice { - case .shortcut: if case .shortcut = current { return current }; return .shortcut(name: "") - case .urlScheme: if case .urlScheme = current { return current }; return .urlScheme("") - case .shell: if case .shell = current { return current }; return .shell(command: "") - } - } - - private func selectedModelID(_ m: ModelSelector) -> String? { - switch m { - case let .onDevice(id): return id - case .cloud: return nil - } - } } /// Compact appearance control: a clickable icon button (opens the SF Symbol picker) plus a color @@ -1869,7 +1473,7 @@ private func naturalIcon(for kind: LaunchItemKind) -> ItemIcon? { switch kind { case .app: return .appDefault case .path: return .fileIcon - case .url, .shortcut, .script, .action, .automation, .preset, .clipboardEntry, .fileEntry, .aiCommand, .claudeProject, .terminalCommand, .claudeProjectPrompt, .terminalCommandPrompt: return nil + case .url, .shortcut, .script, .action, .automation, .preset, .clipboardEntry, .claudeProject, .terminalCommand, .claudeProjectPrompt, .terminalCommandPrompt: return nil } } @@ -2092,8 +1696,6 @@ private func kindLabel(_ kind: LaunchItemKind) -> String { case .automation: return "Automation" case .preset: return "Preset" case .clipboardEntry: return "Clipboard" - case .fileEntry: return "File Entry" - case .aiCommand: return "AI Command" case .claudeProject: return "Claude Project" case .terminalCommand: return "Terminal" case .claudeProjectPrompt: return "Claude (Pick Folder)" @@ -2101,107 +1703,6 @@ private func kindLabel(_ kind: LaunchItemKind) -> String { } } -// MARK: - AI tool/destination pickers - -/// Menu picker for the "Open tool with payload" task target. The stored value is a bare string with -/// opener-defined semantics (see `WorkspaceToolOpener.defaultOpen`): an app *path* (contains "/" or -/// ends ".app") is launched as an app, anything else is run as a named Shortcut. So an app pick -/// stores `candidate.url.path` and a Shortcut pick stores its name. A "Custom…" escape hatch (also -/// auto-shown for an unrecognised value) exposes a `TextField` for not-yet-created Shortcuts. -private struct ToolTargetPicker: View { - @Binding var tool: String - @State private var apps: [AppCandidate] = [] - @State private var shortcuts: [String] = [] - @State private var showCustom = false - /// Set once the async lists have loaded, so a pre-existing listed value isn't briefly treated as - /// "custom" (the `matchesShortcut` check is meaningless against an empty, not-yet-loaded list). - @State private var loaded = false - - private var isAppPath: Bool { tool.contains("/") || tool.hasSuffix(".app") } - private var matchesShortcut: Bool { shortcuts.contains(tool) } - - /// Friendly label for the current value: app file name (no `.app`), shortcut name, or the raw value. - private var menuLabel: String { - if tool.isEmpty { return "Choose app or shortcut…" } - if isAppPath { return URL(fileURLWithPath: tool).deletingPathExtension().lastPathComponent } - return tool - } - - /// Show the free-text field when explicitly requested, or — once the lists have loaded — when the - /// current value matches neither a known app path nor a listed shortcut (so an unusual/typed target - /// stays editable without flashing for a pre-existing listed value while the list loads). - private var showsField: Bool { - showCustom || (loaded && !tool.isEmpty && !isAppPath && !matchesShortcut) - } - - var body: some View { - Menu { - if !shortcuts.isEmpty { - Section("Shortcuts") { - ForEach(shortcuts, id: \.self) { name in - Button { showCustom = false; tool = name } label: { Label(name, systemImage: "bolt.fill") } - } - } - } - Section("Apps") { - ForEach(apps) { app in - Button { - showCustom = false; tool = app.url.path - } label: { - Label { Text(app.name) } icon: { - Image(nsImage: NSWorkspace.shared.icon(forFile: app.url.path)) - .resizable().frame(width: 16, height: 16) - } - } - } - } - Divider() - Button("Custom…") { showCustom = true } - } label: { - if isAppPath { - Label { Text(menuLabel) } icon: { - Image(nsImage: NSWorkspace.shared.icon(forFile: tool)).resizable().frame(width: 16, height: 16) - } - } else { - Label(menuLabel, systemImage: matchesShortcut ? "bolt.fill" : "wand.and.stars") - } - } - .task { apps = await loadInstalledApps(); shortcuts = await loadShortcutNames(); loaded = true } - if showsField { - TextField("Tool (app path or shortcut name)", text: $tool) - } - } -} - -/// Menu picker for a `.shortcut(name:)` destination: the user's Shortcuts plus a "Custom…" → field -/// escape hatch for a not-yet-created Shortcut. Lazy-loaded; tolerates an empty list. -private struct ShortcutPicker: View { - @Binding var name: String - @State private var shortcuts: [String] = [] - @State private var showCustom = false - /// Set once the list loads, so a pre-existing listed Shortcut isn't briefly shown as "custom". - @State private var loaded = false - - private var menuLabel: String { name.isEmpty ? "Choose shortcut…" : name } - private var showsField: Bool { showCustom || (loaded && !name.isEmpty && !shortcuts.contains(name)) } - - var body: some View { - Menu { - ForEach(shortcuts, id: \.self) { n in - Button { showCustom = false; name = n } label: { Label(n, systemImage: "bolt.fill") } - } - Divider() - Button("Custom…") { showCustom = true } - } label: { - Label(menuLabel, systemImage: "bolt.fill") - } - .task { shortcuts = await loadShortcutNames(); loaded = true } - if showsField { - TextField("Shortcut name", text: $name) - } - } -} - // MARK: - Async source loaders struct AppCandidate: Identifiable { @@ -2271,74 +1772,3 @@ func loadShortcutNames() async -> [String] { .filter { !$0.trimmingCharacters(in: .whitespaces).isEmpty } }.value } - -// MARK: - AI command inspector helpers (ported from the former AI-command editor) - -/// Prompt-template token quick-insert bar for the AI command inspector. -private struct AITokenBar: View { - let onInsert: (String) -> Void - private let tokens = ["{input}", "{date}", "{app}", "{url}"] - var body: some View { - HStack(spacing: 6) { - Text("Insert:").font(.caption).foregroundStyle(.secondary) - ForEach(tokens, id: \.self) { token in - Button(token) { onInsert(token) } - .buttonStyle(.bordered).controlSize(.small) - .font(.system(.caption, design: .monospaced)) - } - Spacer() - } - } -} - -/// The result KIND of an AI command: an in-place command (whose replace/paste/preview behaviors are -/// capability toggles resolved from the input at fire) vs a side-effecting task / send-to. -private enum AIOutputKind: String, CaseIterable, Identifiable { - case inPlace, runTask, sendTo - var id: String { rawValue } - var label: String { - switch self { - case .inPlace: return "In-place (replace / paste / preview)" - case .runTask: return "Run a task" - case .sendTo: return "Send to…" - } - } -} - -private enum TaskChoice: String, CaseIterable, Identifiable { - case addToCalendar, addToReminder, newContact, saveToProject, openToolWithPayload, sendTo - var id: String { rawValue } - var label: String { - switch self { - case .addToCalendar: return "Add to Calendar" - case .addToReminder: return "Add to Reminders" - case .newContact: return "New Contact" - case .saveToProject: return "Save to project" - case .openToolWithPayload: return "Open tool with payload" - case .sendTo: return "Send to destination" - } - } -} - -private enum DestinationChoice: String, CaseIterable, Identifiable { - case shortcut, urlScheme, shell - var id: String { rawValue } - var label: String { - switch self { - case .shortcut: return "Shortcut" - case .urlScheme: return "URL scheme" - case .shell: return "Shell command" - } - } -} - -private func aiInputLabel(_ s: InputSource) -> String { - switch s { - case .selection: return "Selected text" - case .clipboard: return "Clipboard" - case .clipboardImage: return "Clipboard image (vision)" - case .screenRegion: return "Screen region (vision)" - case .none: return "No input" - } -} - diff --git a/Sources/ThreeFingerSwitcher/Hub/HubBackgroundAutonomy.swift b/Sources/ThreeFingerSwitcher/Hub/HubBackgroundAutonomy.swift deleted file mode 100644 index 3a54678..0000000 --- a/Sources/ThreeFingerSwitcher/Hub/HubBackgroundAutonomy.swift +++ /dev/null @@ -1,327 +0,0 @@ -import SwiftUI -import AppKit - -// The Hub AI page's **Background autonomy** section (`ai-background-autonomy`, §7): the user-editable -// whitelist editor (trusted folder prefixes + command patterns, persisted to `AppSettings`) and the -// append-only audit-log viewer ("what your agents did while you were away"). App-target view code -// (`xcodebuild` compile-verify; the user run-verifies the live feel). Both surfaces reuse the shared -// `HubSection` Liquid Glass presentation. A store-persist failure is surfaced as a bounded, non-blocking -// banner — NEVER an `NSAlert` (house rule), the headline routed through the single `AIError.message(for:)`. - -// MARK: - The whitelist editor (trusted paths + command patterns) - -/// The whitelist editor: add/remove trusted **folder path prefixes** (picked as local folders only) and -/// trusted **command patterns** (anchored globs). Binds directly to the persisted `AppSettings` arrays — -/// the single source of truth the agent's `BackgroundPolicyResolver` reads. Default-empty (a fresh install -/// trusts nothing arbitrary). -struct HubWhitelistEditor: View { - @Binding var trustedPaths: [String] - @Binding var trustedCommands: [String] - /// Disabled when the AI feature is off (the whitelist only matters while the agent runs). - let isEnabled: Bool - - /// The in-progress command pattern the user is typing before committing it with the add button. - @State private var newCommand = "" - - var body: some View { - HubSection("Background autonomy — trusted writes", - footnote: "Whitelisting a folder or command lets a parked agent run a matching write automatically (still audited). Dangerous operations — delete, overwrite an existing file, arbitrary shell — are NEVER made automatic by the whitelist. Empty by default: a fresh install trusts nothing on your wider filesystem.") { - trustedFolders - Divider() - trustedCommandPatterns - } - .disabled(!isEnabled) - } - - // MARK: Trusted folders (path prefixes) - - @ViewBuilder private var trustedFolders: some View { - VStack(alignment: .leading, spacing: 8) { - Text("Trusted folders").font(.callout).bold() - if trustedPaths.isEmpty { - Text("No trusted folders. A write under a trusted folder runs in the background; off-list writes still ask.") - .font(.caption).foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - } - ForEach(trustedPaths, id: \.self) { path in - HStack(spacing: 8) { - Image(systemName: "folder").foregroundStyle(.secondary) - Text(displayPath(path)).font(.callout).lineLimit(1).truncationMode(.middle) - .help(path) - Spacer(minLength: 8) - Button { trustedPaths.removeAll { $0 == path } } label: { - Image(systemName: "minus.circle.fill") - } - .buttonStyle(.borderless) - .foregroundStyle(.secondary) - .help("Stop trusting \(path)") - } - } - Button { addFolder() } label: { - Label("Add trusted folder…", systemImage: "plus") - } - .buttonStyle(.borderless) - } - } - - // MARK: Trusted command patterns (globs) - - @ViewBuilder private var trustedCommandPatterns: some View { - VStack(alignment: .leading, spacing: 8) { - Text("Trusted command patterns").font(.callout).bold() - if trustedCommands.isEmpty { - Text("No trusted commands. A command (tool / Shortcut / shell name) matching a pattern below runs in the background. Use * and ? as wildcards (e.g. git*).") - .font(.caption).foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - } - ForEach(trustedCommands, id: \.self) { pattern in - HStack(spacing: 8) { - Image(systemName: "terminal").foregroundStyle(.secondary) - Text(pattern).font(.system(.callout, design: .monospaced)).lineLimit(1).truncationMode(.middle) - Spacer(minLength: 8) - Button { trustedCommands.removeAll { $0 == pattern } } label: { - Image(systemName: "minus.circle.fill") - } - .buttonStyle(.borderless) - .foregroundStyle(.secondary) - .help("Stop trusting \(pattern)") - } - } - HStack(spacing: 8) { - TextField("Command pattern (e.g. git*)", text: $newCommand) - .textFieldStyle(.roundedBorder) - .font(.system(.callout, design: .monospaced)) - .onSubmit { addCommand() } - Button { addCommand() } label: { Image(systemName: "plus.circle.fill") } - .buttonStyle(.borderless) - .disabled(trimmedNewCommand.isEmpty) - } - } - } - - // MARK: Actions (pure, local-folder-only validation) - - private var trimmedNewCommand: String { - newCommand.trimmingCharacters(in: .whitespacesAndNewlines) - } - - /// Commit the typed command pattern (de-duplicated, trimmed). - private func addCommand() { - let pattern = trimmedNewCommand - guard !pattern.isEmpty, !trustedCommands.contains(pattern) else { newCommand = ""; return } - trustedCommands.append(pattern) - newCommand = "" - } - - /// Folder pick via `NSOpenPanel`, **local folders only** (network / iCloud-placeholder rejected per the - /// spec's "only local folders" scenario). Stores the standardized absolute path; de-duplicates. - private func addFolder() { - let panel = NSOpenPanel() - panel.canChooseDirectories = true - panel.canChooseFiles = false - panel.allowsMultipleSelection = true - panel.canCreateDirectories = false - panel.prompt = "Trust" - panel.message = "Choose a local folder a parked agent may write into automatically." - guard panel.runModal() == .OK else { return } - for url in panel.urls where HubWhitelistEditor.isLocalFolder(url) { - let std = url.standardizedFileURL.resolvingSymlinksInPath().path - if !trustedPaths.contains(std) { trustedPaths.append(std) } - } - } - - /// Accept only a genuinely local, non-placeholder folder (reject network volumes and iCloud - /// not-yet-downloaded placeholders). The spec's "only local folders can be added" scenario. - static func isLocalFolder(_ url: URL) -> Bool { - let keys: Set = [.isDirectoryKey, .volumeIsLocalKey, .isUbiquitousItemKey] - guard let values = try? url.resourceValues(forKeys: keys) else { return false } - guard values.isDirectory == true else { return false } - if values.isUbiquitousItem == true { return false } // iCloud placeholder - if values.volumeIsLocal == false { return false } // network volume - return true - } - - /// Abbreviate `/Users/me/Notes` to `~/Notes` for display (the full path stays in the row's `help`). - private func displayPath(_ path: String) -> String { - (path as NSString).abbreviatingWithTildeInPath - } -} - -// MARK: - The audit-log viewer ("what your agents did while you were away") - -/// The audit-log viewer (§7.2): a bounded, reverse-chronological ledger of recent tool steps — each row -/// shows the tool, the redacted args summary, the effective tier, the outcome (a `.failed` as a clean -/// headline + opt-in details disclosure), a timestamp, and whether it ran in the background. Reads the -/// Core `AuditLog` synchronously via the `HubContext` seam. A store-persist failure shows as a bounded, -/// non-blocking banner (NEVER an `NSAlert`). -struct HubAuditLogViewer: View { - /// Pulls the most-recent records on each render (synchronous read seam). - let records: (_ limit: Int) -> [AuditRecord] - /// A clean headline if the durable store last failed to persist (else `nil`) — already routed through - /// `AIError.message(for:)`, so the headline is safe and the details copyable. - let persistError: () -> AIPresentedError? - - /// How many recent records the viewer shows. - private let limit = 100 - - var body: some View { - HubSection("Audit log", - footnote: "Every tool step your agents ran — automatic, confirmed, skipped, or failed — newest first. “While you were away” marks steps that ran in the background while a session was parked.") { - if let error = persistError() { - AuditPersistBanner(error: error) - Divider() - } - let recent = records(limit) - if recent.isEmpty { - Text("No agent activity yet. When an agent runs a tool step, it appears here.") - .font(.caption).foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - } else { - ForEach(recent) { record in - AuditRow(record: record) - if record.id != recent.last?.id { Divider() } - } - } - } - } -} - -/// One audit row — tool + redacted summary on top, the effective tier · outcome · timestamp below, with a -/// "while you were away" tag for background steps and an opt-in details disclosure for a failure headline's -/// copyable detail. -private struct AuditRow: View { - let record: AuditRecord - @State private var showDetails = false - - var body: some View { - VStack(alignment: .leading, spacing: 3) { - HStack(spacing: 6) { - Image(systemName: outcomeIcon).foregroundStyle(outcomeTint) - Text(record.tool).font(.callout).bold().lineLimit(1).truncationMode(.middle) - if record.wasBackground { - Text("while you were away") - .font(.caption2).foregroundStyle(.secondary) - .padding(.horizontal, 6).padding(.vertical, 1) - .background(Capsule().fill(Color.secondary.opacity(0.15))) - } - Spacer(minLength: 8) - Text(record.timestamp, format: .relative(presentation: .named)) - .font(.caption).foregroundStyle(.secondary) - } - if !record.argumentsSummary.isEmpty { - Text(record.argumentsSummary) - .font(.system(.caption, design: .monospaced)) - .foregroundStyle(.secondary) - .lineLimit(2).truncationMode(.middle) - } - HStack(spacing: 6) { - tierBadge - Text(outcomeLabel).font(.caption).foregroundStyle(.secondary).lineLimit(1).truncationMode(.middle) - Spacer(minLength: 0) - } - if let detail = failureDetail { - DisclosureGroup(isExpanded: $showDetails) { - Text(detail) - .font(.system(.caption2, design: .monospaced)) - .foregroundStyle(.secondary) - .textSelection(.enabled) - .lineLimit(4).truncationMode(.middle) - } label: { - Text("Show details").font(.caption2) - } - } - } - .frame(maxWidth: .infinity, alignment: .leading) - } - - private var tierBadge: some View { - Text(tierLabel) - .font(.caption2.bold()) - .padding(.horizontal, 6).padding(.vertical, 1) - .background(Capsule().fill(tierTint.opacity(0.18))) - .foregroundStyle(tierTint) - } - - private var tierLabel: String { - switch record.policy { - case .auto: return "AUTO" - case .confirm: return "CONFIRM" - case .dangerous: return "DANGEROUS" - } - } - - private var tierTint: Color { - switch record.policy { - case .auto: return .green - case .confirm: return .blue - case .dangerous: return .orange - } - } - - private var outcomeLabel: String { - switch record.outcome { - case .done: return "Done" - case .awaitingApproval: return "Awaiting your approval" - case let .declined(reason): return reason.isEmpty ? "Skipped" : "Skipped — \(reason)" - case let .failed(headline): return headline - } - } - - private var outcomeIcon: String { - switch record.outcome { - case .done: return "checkmark.circle.fill" - case .awaitingApproval: return "hourglass.circle.fill" - case .declined: return "slash.circle.fill" - case .failed: return "exclamationmark.triangle.fill" - } - } - - private var outcomeTint: Color { - switch record.outcome { - case .done: return .green - case .awaitingApproval: return .blue - case .declined: return .secondary - case .failed: return .orange - } - } - - /// A failed outcome's headline is already clean; the disclosure simply re-shows it as the copyable - /// detail (the audit record stores only the headline, never raw OS text — by design). - private var failureDetail: String? { - if case let .failed(headline) = record.outcome { return headline } - return nil - } -} - -/// The bounded, non-blocking store-persist failure banner (house rule: never `NSAlert`; headline clean, -/// details opt-in). Mirrors the AI canvas's failure card. -private struct AuditPersistBanner: View { - let error: AIPresentedError - @State private var showDetails = false - - var body: some View { - VStack(alignment: .leading, spacing: 4) { - HStack(spacing: 6) { - Image(systemName: "exclamationmark.triangle.fill").foregroundStyle(.orange) - Text(error.headline) - .font(.callout) - .lineLimit(2).truncationMode(.middle) - Spacer(minLength: 0) - } - if let details = error.details { - DisclosureGroup(isExpanded: $showDetails) { - Text(details) - .font(.system(.caption2, design: .monospaced)) - .foregroundStyle(.secondary) - .textSelection(.enabled) - .lineLimit(4).truncationMode(.middle) - } label: { - Text("Show details").font(.caption2) - } - } - } - .frame(maxWidth: .infinity, alignment: .leading) - .padding(10) - .background(RoundedRectangle(cornerRadius: 8).fill(Color.orange.opacity(0.10))) - } -} diff --git a/Sources/ThreeFingerSwitcher/Hub/HubBindingPicker.swift b/Sources/ThreeFingerSwitcher/Hub/HubBindingPicker.swift index 20553f5..61547e5 100644 --- a/Sources/ThreeFingerSwitcher/Hub/HubBindingPicker.swift +++ b/Sources/ThreeFingerSwitcher/Hub/HubBindingPicker.swift @@ -1,15 +1,15 @@ import SwiftUI /// A reusable, presentation-only editor for a single remappable surface's gesture **bindings**, shown -/// beside that page's `HubGesturePreview`. It renders one row per surface *action* (e.g. the AI canvas's +/// beside that page's `HubGesturePreview`. It renders one row per surface *action* (e.g. the switcher's /// `commit` / `dismiss` / `ignore`): a label and a `Picker` listing the surface's whole *excursion* /// vocabulary. Choosing an excursion calls the page-supplied `assign` closure, which routes through the /// pure `GestureBindings.…assigning(_:to:)` — so the model (not this view) enforces per-surface /// mutual-exclusivity (assigning a taken excursion swaps it). This view holds no binding state itself. /// -/// It is generic over the action and excursion vocabularies so it serves all three surfaces (canvas, -/// Files drill, switcher direction) unchanged; a page supplies: -/// - `actions` — the surface's action cases (e.g. `GestureBindings.CanvasAction.allCases`); +/// It is generic over the action and excursion vocabularies so it serves any bindable surface +/// unchanged; a page supplies: +/// - `actions` — the surface's action cases; /// - `excursions` — the surface's bindable excursion vocabulary; /// - `actionLabel` / `excursionLabel` — human-readable strings for each (use `HubBindingLabels`); /// - `current(action)` — the excursion the binding currently maps that action to (the picker's value); @@ -97,42 +97,6 @@ struct HubBindingPicker String { - switch excursion { - case .swipeUp: return "Swipe up" - case .swipeDown: return "Swipe down" - case .swipeLeft: return "Swipe left" - case .swipeRight: return "Swipe right" - } - } - - static func canvasAction(_ action: GestureBindings.CanvasAction) -> String { - switch action { - case .commit: return "Commit" - case .dismiss: return "Dismiss" - case .ignore: return "Ignore" - } - } - - static func files(_ excursion: GestureBindings.FilesExcursion) -> String { - switch excursion { - case .lift: return "Lift" - case .plusOneFingerLift: return "+1 finger then lift" - case .fourFingerHorizontal: return "Four fingers sideways" - } - } - - static func filesAction(_ action: GestureBindings.FilesAction) -> String { - switch action { - // The Files-band-actions change repurposes these excursions: `open` is the primary resolve (it runs - // the configured lift action — deliver or open), and `openWith` opens the action menu (Open-With - // folds in as the menu's "Open in ▸"). The enum case names are kept; only the labels reflect this. - case .open: return "Lift action" - case .openWith: return "Action menu" - case .discard: return "Discard" - } - } - static func axisDirection(_ direction: GestureBindings.AxisDirection) -> String { switch direction { case .normal: return "Normal" @@ -141,26 +105,3 @@ enum HubBindingLabels { } } -#if DEBUG -private struct HubBindingPickerPreviewHost: View { - @State private var binding = GestureBindings.CanvasBinding.default - @State private var demo: GesturePose.Axis? - - var body: some View { - HubBindingPicker( - actions: GestureBindings.CanvasAction.allCases, - excursions: GestureBindings.CanvasExcursion.allCases, - actionLabel: HubBindingLabels.canvasAction, - excursionLabel: HubBindingLabels.canvas, - current: { binding.excursion(for: $0) }, - assign: { excursion, action in binding = binding.assigning(excursion, to: action) }, - demoAxis: { _ in .horizontal }, - demo: { demo = $0 } - ) - .frame(width: 360) - .padding() - } -} - -#Preview("HubBindingPicker — canvas") { HubBindingPickerPreviewHost() } -#endif diff --git a/Sources/ThreeFingerSwitcher/Hub/HubFeaturePages.swift b/Sources/ThreeFingerSwitcher/Hub/HubFeaturePages.swift index 0b7f8be..11a835e 100644 --- a/Sources/ThreeFingerSwitcher/Hub/HubFeaturePages.swift +++ b/Sources/ThreeFingerSwitcher/Hub/HubFeaturePages.swift @@ -228,7 +228,7 @@ struct LauncherPage: View { let models = HubPreviewModels(realWindowRows: context.realWindowRows, seedThumbnails: context.seedThumbnails, launcherBands: context.launcherBands) - demo.seed(from: models.makeLauncherModel(clipboardOn: false, aiOn: false, dwell: settings.dwellToArmDuration)) + demo.seed(from: models.makeLauncherModel(clipboardOn: false, dwell: settings.dwellToArmDuration)) } var body: some View { @@ -315,7 +315,7 @@ struct ClipboardPage: View { let models = HubPreviewModels(realWindowRows: context.realWindowRows, seedThumbnails: context.seedThumbnails, launcherBands: context.launcherBands) - demo.seed(from: models.makeLauncherModel(clipboardOn: true, aiOn: false, dwell: settings.dwellToArmDuration), + demo.seed(from: models.makeLauncherModel(clipboardOn: true, dwell: settings.dwellToArmDuration), landOnLastBand: true) } @@ -388,425 +388,6 @@ struct ClipboardPage: View { } } -// MARK: - AI - -struct AIPage: View { - let context: HubContext - @ObservedObject private var settings: AppSettings - @ObservedObject private var models: ModelManager - - /// §11.5 — the REAL launcher showing its AI band (the hero): a `LauncherView` seeded once with the AI - /// band on (the last band) and landed on it (the resting frame shows the band). The ghost-hand autoplay - /// plays the full path — 4-finger open → 2-finger traverse down to the AI band → the firing lift → a - /// directed two-finger commit swipe — and the preview's `sync` seam replays the journey on the model - /// (the resolve tail plays over it; the canvas isn't miniature); clockless, visibility-gated, never - /// grows (see `docs/postmortem-idle-cpu-spin.md`). - @StateObject private var demo = HubLauncherDemo() - @State private var seeded = false - /// The base autoplay journey, and the hover-demo override the canvas-resolve binding rows push in: - /// hovering a row demos that action's currently-bound excursion as a directed canvas-resolve swipe. - @State private var hoverGesture: GesturePose.DemoGesture? - /// The excursion the hovered binding row maps to — stashed by the picker's `demoAxis` closure (an - /// event-handler call) so the `demo` closure can build the matching directed candidate swipe. The - /// `HubBindingPicker` is a shared component that speaks `GesturePose.Axis`; this bridges its hover - /// signal to the preview's `DemoGesture` candidate without changing the component. - @State private var hoveredExcursion: GestureBindings.CanvasExcursion? - - init(context: HubContext) { - self.context = context - _settings = ObservedObject(wrappedValue: context.settings) - _models = ObservedObject(wrappedValue: context.models) - } - - /// The preview's attract journey: open the four-finger launcher → traverse DOWN the band list to the - /// AI band (the real band-rail grammar, so the driven miniature can follow stroke for stroke) → the - /// firing lift → a fresh directed downward canvas-commit swipe. The hover-demo override - /// (`hoverGesture`) plays a candidate resolve instead. - private static let aiJourney = HubLauncherDemo.bandJourneyGesture(openLength: 0.30, resolve: .swipeDown) - - /// The coarse axis a canvas excursion sweeps along (up/down ⇒ vertical, left/right ⇒ horizontal) — - /// the `GesturePose.Axis` the shared `HubBindingPicker` component expects from `demoAxis`. - private func axis(for excursion: GestureBindings.CanvasExcursion) -> GesturePose.Axis { - switch excursion { - case .swipeUp, .swipeDown: return .vertical - case .swipeLeft, .swipeRight: return .horizontal - } - } - - /// Map a canvas excursion to the directed resolve swipe its hover-demo should play (a standalone - /// two-finger `canvasResolve` in that direction) — pushed into the preview's `hoverGesture`. - private func candidate(for excursion: GestureBindings.CanvasExcursion) -> GesturePose.DemoGesture { - switch excursion { - case .swipeUp: return GesturePose.canvasResolve(.swipeUp) - case .swipeDown: return GesturePose.canvasResolve(.swipeDown) - case .swipeLeft: return GesturePose.canvasResolve(.swipeLeft) - case .swipeRight: return GesturePose.canvasResolve(.swipeRight) - } - } - - /// Seed once with the AI band on, landing the static preview on the last (AI) band. - private func seedIfNeeded() { - guard !seeded else { return } - seeded = true - let previewModels = HubPreviewModels(realWindowRows: context.realWindowRows, - seedThumbnails: context.seedThumbnails, - launcherBands: context.launcherBands) - demo.seed(from: previewModels.makeLauncherModel(clipboardOn: false, aiOn: true, dwell: settings.dwellToArmDuration), - landOnLastBand: true) - } - - /// Picker binding: maps `aiSelectedModelID` (nil = registry default) to the picker's optional-string. - private var modelSelection: Binding { - Binding(get: { settings.aiSelectedModelID }, - set: { settings.aiSelectedModelID = $0 }) - } - - /// The fleet roster's ACTIVE CHAT radio binding (`aiSelectedChatModelID`; nil = chat default). - /// Whether this OS can run the on-device transcriber (`SpeechAnalyzer`, macOS 26). - private var voiceOSSupported: Bool { - if #available(macOS 26.0, *) { return true } - return false - } - - /// The voice section's cost disclosure — includes the OS-floor message when it applies. - private var voiceFootnote: String { - if !voiceOSSupported { - return "Requires macOS 26 (on-device speech recognition)." - } - return "Push-to-talk with the on-device assistant. The microphone opens ONLY while the key is held (asked for on first press) — no wake word, never always-listening. Speech is transcribed on this Mac; audio never leaves the device." - } - - private var chatModelSelection: Binding { - Binding(get: { settings.aiSelectedChatModelID }, - set: { settings.aiSelectedChatModelID = $0 }) - } - - /// The fleet roster's ENABLED CAPABILITY toggles binding (`aiEnabledCapabilityModelIDs`). - private var capabilityModelSelection: Binding> { - Binding(get: { settings.aiEnabledCapabilityModelIDs }, - set: { settings.aiEnabledCapabilityModelIDs = $0 }) - } - - /// The model the management surface shows: the user's pinned selection if it resolves, else default. - private var selectedModelDescriptor: ModelDescriptor { - let registry = ModelCatalog.standard - if let id = settings.aiSelectedModelID, let d = registry.descriptor(id: id) { return d } - return registry.defaultDescriptor ?? registry.models[0] - } - - // MARK: - Context tuning + cost surface (tasks 5.2 / 5.3 / 5.4, design D5) - - /// The effective context-token budget for the chosen preset, clamped to the selected model's max. - /// `agentContextTokens` is the persisted resolution; for a non-custom preset it follows the preset. - private var effectiveContextTokens: Int { - let modelMax = selectedModelDescriptor.maxContextTokens - return settings.agentContextPreset.tokens(modelMax: modelMax, custom: settings.agentContextTokens) - } - - /// The live cost surface (estimated RAM + concurrent-stream count + relative speed) for the chosen - /// context — derived from the SAME pure `ConcurrencyBudget` the batched conformer uses (never silent - /// OOM, house requirement). Recomputes whenever the preset / toggle / model changes. - private var cost: AgentContextCostModel { - AgentContextCostModel(contextTokens: effectiveContextTokens, - compactKV: settings.agentCompactKV, - weightBytes: selectedModelDescriptor.sizeBytes) - } - - /// Picker binding: the preset segmented control writes the preset AND resolves `agentContextTokens` to - /// the preset's token value (clamped to the model max) so the persisted budget the runtime / compaction - /// reads always matches the chosen preset. - private var presetSelection: Binding { - Binding(get: { settings.agentContextPreset }, - set: { preset in - settings.agentContextPreset = preset - settings.agentContextTokens = preset.tokens(modelMax: selectedModelDescriptor.maxContextTokens, - custom: settings.agentContextTokens) - }) - } - - @ViewBuilder private var contextSection: some View { - HubSection("Context", - footnote: "Longer context remembers more of a conversation, skills, and memory — but the KV cache grows per background session, so more context means fewer concurrent background sessions and slower per-token speed. The estimate below updates as you choose; the foreground session always fits.") { - // The Balanced / Long / Max preset (custom is implicit when a heavy skill raises it). - Picker("Context size", selection: presetSelection) { - ForEach([AgentContextPreset.balanced, .long, .max], id: \.self) { preset in - Text(preset.title).tag(preset) - } - } - .pickerStyle(.segmented) - .disabled(!settings.aiCommandsEnabled) - - // The single comprehensible KV-quant lever (8-bit) — a longer context fits the same RAM. - Toggle("Compact long contexts (8-bit KV)", isOn: $settings.agentCompactKV) - .disabled(!settings.aiCommandsEnabled) - - // The cost surface (RAM · concurrent background sessions · relative speed) — live. - HStack(spacing: 6) { - Image(systemName: "memorychip") - .foregroundStyle(.secondary) - Text("\(effectiveContextTokens.formatted()) tokens · \(cost.summary)") - .font(.callout) - .foregroundStyle(.secondary) - .lineLimit(2) - .truncationMode(.middle) - Spacer(minLength: 0) - } - - // Per-skill override (read-only) — a heavy skill may raise the effective context (task 5.4). - // The override value rides on the skill file (`ai-skills-as-files`); this surface only displays - // that capability. With no skill source wired, the default resolves to no raise. - if let overrideNote = skillOverrideNote { - HStack(spacing: 6) { - Image(systemName: "wand.and.stars") - .foregroundStyle(.secondary) - Text(overrideNote) - .font(.footnote) - .foregroundStyle(.secondary) - .lineLimit(2) - Spacer(minLength: 0) - } - } - } - } - - // MARK: - Parked sessions (the opt-in auto-dismiss countdown) - - /// The auto-dismiss countdown surfaced in MINUTES: an idle, fully-seen session untouched this long is - /// dismissed forever — **0 = never (the default)**. `agentParkAutoDismissCountdown` is persisted in - /// SECONDS, so this binding divides on read / multiplies on write, keeping the Hub slider human. - /// Clamped to whole minutes so the readout is clean. - private var autoDismissMinutes: Binding { - Binding(get: { (settings.agentParkAutoDismissCountdown / 60).rounded() }, - set: { settings.agentParkAutoDismissCountdown = max(0, $0.rounded()) * 60 }) - } - - @ViewBuilder private var parkedSessionsSection: some View { - HubSection("Parked sessions", - footnote: "A conversation you park to the notch waits for you here — it keeps its results until you delete it. Set a countdown to auto-dismiss idle, already-seen sessions (0 = never; unseen results and sessions that need you always stay). Raise the cap to keep more parked at once.") { - LabeledSlider(title: "Reveal dwell (seconds)", value: $settings.agentNotchRevealDwell, - range: 0...1, format: "%.2f", - help: "How long the cursor must linger behind the notch before the dock opens. Keeps a quick pass through the notch — reaching for the menu bar or another corner — from popping it. Set to 0 to open instantly. Default 0.30s.") - .disabled(!settings.aiCommandsEnabled) - - LabeledSlider(title: "Auto-dismiss after (minutes, 0 = never)", value: autoDismissMinutes, - range: 0...30, format: "%.0f", - help: "How long an idle, already-seen session waits before it's dismissed forever. 0 (the default) keeps sessions until you delete them; sessions with unseen results never expire.") - .disabled(!settings.aiCommandsEnabled) - - LabeledIntSlider(title: "Maximum parked sessions", value: $settings.agentMaxParkedSessions, - range: 1...12, - help: "Soft cap on parked sessions. When exceeded, the least-recently-used idle one is evicted (never an active or needs-you session).") - .disabled(!settings.aiCommandsEnabled) - } - } - - /// The read-only per-skill-override note (task 5.4). Resolves the effective budget through the concrete - /// `AgentContextBudgetProvider` (∩ model max ∩ per-skill override) — when a heavy skill raises it above - /// the chosen preset, the note reports the raised number; otherwise it explains the capability. - private var skillOverrideNote: String? { - let provider = AgentContextBudgetProvider(userContextTokens: effectiveContextTokens, - modelMaxContextTokens: selectedModelDescriptor.maxContextTokens) - let resolved = provider.maxContextTokens - if resolved > effectiveContextTokens { - return "A heavy skill raises the effective context to \(resolved.formatted()) tokens for that session." - } - return "A heavy skill may raise the effective context for its own session (read-only — authored on the skill)." - } - - // MARK: - Release Full Potential (ai-full-potential-toggle, addendum §D1) - - /// Per-capability binding into `AppSettings` for the five sub-flags. The gate reads these at consult - /// time; the Hub writes them directly (turning the master off NEVER zeroes them — see `panic-off`). - private func subFlagBinding(_ capability: FullPotentialCapability) -> Binding { - switch capability { - case .cpuLane: return $settings.cpuLaneEnabled - case .batchedRuntime: return $settings.batchedRuntimeEnabled - case .mediaGen: return $settings.mediaGenEnabled - case .backgroundAutonomy: return $settings.backgroundAutonomyEnabled - case .fleetCloud: return $settings.fleetCloudEscalationEnabled - } - } - - /// The sub-toggle's human title. - private func subFlagTitle(_ capability: FullPotentialCapability) -> String { - switch capability { - case .cpuLane: return "CPU lane" - case .batchedRuntime: return "Batched runtime" - case .mediaGen: return "Media generation" - case .backgroundAutonomy: return "Background autonomy" - case .fleetCloud: return "Cloud escalation" - } - } - - /// The persistent, ALWAYS-VISIBLE cost line (RAM / heat / latency / $) — rendered inline as the row's - /// caption, never behind a tooltip (design Decision 4; the honest-surface ethos). The media + cloud - /// rows state the hard truths plainly: media evicts chat; cloud spends real money + sends data - /// off-device. - private func subFlagCost(_ capability: FullPotentialCapability) -> String { - switch capability { - case .cpuLane: - return "Heat / battery — a second (CPU) lane runs concurrently for fast small jobs. Short structured bursts only; CPU per-token is slower." - case .batchedRuntime: - return "RAM + latency — multiplexes several background sessions over one weight read. Larger context means more resident KV cache, and latency rises under load." - case .mediaGen: - return "RAM (eviction) + latency + disk — a heavy generation evicts chat: the assistant goes quiet while it paints. Minutes per clip; tens of gigabytes of weights." - case .backgroundAutonomy: - return "Unattended action — the agent may act while you're away. Only whitelisted, contained writes auto-run; dangerous ones still ask, and everything is audited." - case .fleetCloud: - return "$ + network — spends real money and sends data off-device to a paid cloud model (Claude / GLM-5.2). Budget-capped + audited; off until you arm it." - } - } - - /// The Full Potential section: the master toggle FIRST, then the five sub-toggles (disabled while the - /// master is off — visibly relocked — but their persisted values RETAINED). Each sub-row carries its - /// cost inline. Shared Liquid Glass presentation (`HubSection` card). The whole section is itself gated - /// behind the AI-commands opt-in (the fleet is a strict subset of the AI feature). - @ViewBuilder private var fullPotentialSection: some View { - HubSection("Release Full Potential", - footnote: "The agent ships calm: every heavy capability below is off until you release it here, and each states its own cost in the same breath it offers itself. Turning the master off relocks them all at once (your choices are kept, just inert) — a single calm panic-off. Cloud escalation stays off until you arm it: no surprise spend, no data leaving the device.") { - // The master toggle FIRST. No cost line of its own — it just lights up the section. - SwitchRow("Release Full Potential", isOn: $settings.fullPotentialEnabled, - caption: "Lights up the agent fleet. Each capability below states its own cost.") - .disabled(!settings.aiCommandsEnabled) - - Divider() - - // The five sub-toggles, rendered by iterating the capability enum. Disabled (visibly relocked) - // while the master is off; flipping the master off retains these stored values (no zeroing). - ForEach(FullPotentialCapability.allCases, id: \.self) { capability in - SwitchRow(subFlagTitle(capability), - isOn: subFlagBinding(capability), - caption: subFlagCost(capability)) - .disabled(!settings.aiCommandsEnabled || !settings.fullPotentialEnabled) - } - } - } - - var body: some View { - HubPage(HubDestination.ai.title, - subtitle: "Run on-device AI commands. Author the commands themselves on the Bands page.") { - HubSection(footnote: "Runs an on-device Gemma 4 model — turning this on starts a one-time multi-gigabyte download. No new permission or logout needed (a calendar task asks for Calendar access the first time it runs). Add AI commands to any band on the Bands page. Off by default.") { - HubFeatureHeader( - preview: HubGesturePreview( - gesture: Self.aiJourney, - hoverGesture: hoverGesture, - // The sync seam: the miniature replays the journey to the AI band; the resolve - // tail and any hovered candidate swipe play over it (the canvas isn't miniature). - sync: { demo.drive($0, script: .bandJourneyResolve) } - ) { - LauncherDemoMiniature(demo: demo) - }, - icon: HubDestination.ai.systemImage, - title: HubDestination.ai.title, - subtitle: "Run on-device AI commands on your selection, clipboard, or screen.", - isOn: $settings.aiCommandsEnabled - ) - .onAppear { seedIfNeeded() } - } - HubSection("Resolve gestures", - footnote: "Choose which two-finger swipe commits, dismisses, or is ignored while the AI command canvas is open. Each move maps to one action — picking a taken move swaps it. Hover a row to preview the move above.") { - HubBindingPicker( - actions: GestureBindings.CanvasAction.allCases, - excursions: GestureBindings.CanvasExcursion.allCases, - actionLabel: HubBindingLabels.canvasAction, - excursionLabel: HubBindingLabels.canvas, - current: { settings.gestureBindings.canvas.excursion(for: $0) }, - assign: { excursion, action in - settings.gestureBindings.canvas = settings.gestureBindings.canvas.assigning(excursion, to: action) - }, - demoAxis: { excursion in - // Stash the hovered excursion (event-handler context) so `demo` can build the - // matching directed candidate; return the coarse axis the component expects. - hoveredExcursion = excursion - return axis(for: excursion) - }, - demo: { axis in - // The component signals enter (non-nil axis) / exit (nil); translate to a - // directed candidate swipe for the hovered excursion, or clear the override. - hoverGesture = (axis == nil) ? nil : hoveredExcursion.map { candidate(for: $0) } - } - ) - .disabled(!settings.aiCommandsEnabled) - } - HubSection("Model", - footnote: "The fleet: a chat model + a small CPU model co-reside; an image or video model evicts chat while it generates (the assistant goes quiet, then reloads). Cloud models never run on-device and are off until you enable cloud escalation.") { - // The §C1 fleet roster (`ai-model-fleet`, D6): role / lane / provider / per-model status / - // honest residency cost, with the plan-driven evict-chat disclosure and the gated cloud - // rows. `fleetCloudEscalationEnabled` is owned by `ai-full-potential-toggle` — consumed - // here (default false) until that flag lands. Fleet-of-one renders as the single picker. - HubFleetRosterView(cloudEscalationEnabled: { settings.fullPotentialGate.isUnlocked(.fleetCloud) }, - activeChatID: chatModelSelection, - enabledCapabilityModelIDs: capabilityModelSelection, - onDownloadCapabilityModel: context.onDownloadCapabilityModel, - manager: models, - aiEnabled: settings.aiCommandsEnabled) - - // The existing per-model lifecycle surface (download / status / evict / delete) for the - // SELECTED model — preserved unchanged so the per-model status rule still holds. - ModelManagementView(manager: models, - descriptor: selectedModelDescriptor, - onDownload: context.onDownloadModel) - .disabled(!settings.aiCommandsEnabled) - - // Idle-TTL for the resident weights (`model-idle-ttl-and-memory-pressure`): after this - // long fully idle (no turn, no open chat, nothing scheduled) the loaded model is freed - // from memory; the next command reloads it on demand. Memory-pressure eviction is - // always armed and not a setting. "Never" = the pre-change keep-forever behavior. - Picker("Free model memory after", selection: $settings.aiIdleEvictMinutes) { - Text("Never").tag(0) - Text("15 minutes").tag(15) - Text("30 minutes").tag(30) - Text("1 hour").tag(60) - Text("2 hours").tag(120) - Text("4 hours").tag(240) - } - .disabled(!settings.aiCommandsEnabled) - .help("When the AI has been idle this long, the loaded model is freed from memory. The next command reloads it automatically.") - } - // Release Full Potential: the master gate + five cost-disclosing sub-toggles for the heavy - // fleet capabilities (`ai-full-potential-toggle`, addendum §D1). - fullPotentialSection - HubSection("Reasoning", - footnote: "Let the model think before answering for higher-quality results (a bit slower). Thinking is never shown or pasted — only the final result.") { - Toggle("Reasoning", isOn: $settings.aiReasoningEnabled) - .disabled(!settings.aiCommandsEnabled) - } - // Voice + computer use (`add-voice-computer-use-agent`): two separate opt-ins with honest - // cost disclosure. Voice needs macOS 26 (SpeechAnalyzer) + the microphone permission on - // first press; computer use reuses the existing Accessibility grant — no new permission. - HubSection("Voice conversation", - footnote: voiceFootnote) { - Toggle("Talk with the assistant (push-to-talk)", isOn: $settings.voiceConversationEnabled) - .disabled(!settings.aiCommandsEnabled || !voiceOSSupported) - Text("Double-tap Right Option and hold the second press to talk; release to send. The same double-tap-and-hold interrupts it mid-reply. Any trackpad touch stops it. Single presses and shortcuts like ⌥⌫ are never affected.") - .font(.caption).foregroundStyle(.secondary) - } - HubSection("Computer use", - footnote: "The assistant can read windows, focus them, and — with your approval — click and type in them. Uses the Accessibility permission you already granted; no new permission. Every action needs approval unless you turn on auto mode for a conversation, and any trackpad touch instantly stops it.") { - Toggle("Let the assistant use windows", isOn: $settings.computerUseEnabled) - .disabled(!settings.aiCommandsEnabled) - } - contextSection - parkedSessionsSection - // Background autonomy (`ai-background-autonomy`, §7): the user-editable trust boundary (the - // whitelist) + the append-only "what your agents did while you were away" ledger. - HubWhitelistEditor(trustedPaths: $settings.agentWhitelistPaths, - trustedCommands: $settings.agentWhitelistCommands, - isEnabled: settings.aiCommandsEnabled) - HubAuditLogViewer(records: context.recentAuditRecords, - persistError: context.auditStorePersistError) - } - // Keep the status row tied to the SELECTED model: re-settle the manager's displayed state on - // appear, when the picked model changes, and when AI is turned on — otherwise the single shared - // status would keep showing whichever model was last active. - .onAppear { models.showStatus(for: selectedModelDescriptor) } - .onChange(of: settings.aiSelectedModelID) { models.showStatus(for: selectedModelDescriptor) } - .onChange(of: settings.aiCommandsEnabled) { - if settings.aiCommandsEnabled { models.showStatus(for: selectedModelDescriptor) } - } - } -} - // MARK: - Keyboard Language struct KeyboardLanguagePage: View { @@ -945,13 +526,11 @@ struct GeneralPage: View { LazyVGrid(columns: [GridItem(.flexible(), spacing: 12), GridItem(.flexible(), spacing: 12)], alignment: .leading, spacing: 12) { ToggleCard("App data & settings", isOn: $wipeAppData, - caption: "Preferences, bands, AI commands, keyboard-language memory, clipboard history, project outputs, first-run state.") + caption: "Preferences, bands, keyboard-language memory, clipboard history, project outputs, first-run state.") ToggleCard("Caches", isOn: $wipeCaches, caption: "The app's cache and HTTP storage directories.") - ToggleCard("AI models", isOn: $wipeAIModels, - caption: "The downloaded on-device model weights (multi-GB, re-downloadable). Turns the AI opt-in off.") ToggleCard("Permissions", isOn: $wipePermissions, - caption: "Resets every permission the app can hold (Accessibility, Screen Recording, Input Monitoring, Automation, Calendar, Reminders, Contacts) — macOS will prompt again.") + caption: "Resets every permission the app can hold (Accessibility, Screen Recording, Input Monitoring, Automation) — macOS will prompt again.") } Divider() HStack { @@ -967,14 +546,12 @@ struct GeneralPage: View { @State private var wipeAppData = false @State private var wipeCaches = false - @State private var wipeAIModels = false @State private var wipePermissions = false private var dangerSelection: DangerZoneSelection { var selection: DangerZoneSelection = [] if wipeAppData { selection.insert(.appData) } if wipeCaches { selection.insert(.caches) } - if wipeAIModels { selection.insert(.aiModels) } if wipePermissions { selection.insert(.permissions) } return selection } @@ -982,7 +559,7 @@ struct GeneralPage: View { // MARK: - §13 Launcher demo holder (the user's REAL launcher, stepped by the sync drive) -/// The §13 holder behind the Launcher / Clipboard / Files / AI previews: it owns the **real** `LauncherModel` +/// The §13 holder behind the Launcher / Clipboard previews: it owns the **real** `LauncherModel` /// (rendered by a real `LauncherView`), seeded once with the user's real bands so the preview shows the actual /// launcher. Like `HubSwitcherDemo`, the model follows the ghost hand through the preview's **sync seam** /// (`drive(_:script:)`): the four-finger open pops the panel in at the activation beat, the two-finger @@ -992,7 +569,7 @@ struct GeneralPage: View { /// owns no clock — frames arrive only from the preview's visibility-gated `TimelineView`, and every mutation /// is state-guarded (idempotent per frame). /// -/// Band pages (Clipboard / Files / AI) seed with `landOnLastBand: true` so the preview's resting/static frame +/// Band pages (Clipboard) seed with `landOnLastBand: true` so the preview's resting/static frame /// *shows* their band (the last band) — the driven loop then replays the journey from the home band toward it. @MainActor final class HubLauncherDemo: ObservableObject { @@ -1024,7 +601,7 @@ final class HubLauncherDemo: ObservableObject { private var hoverPresented = false /// Seed the model from a `HubPreviewModels`-built launcher (the user's real bands). `landOnLastBand` - /// lands the selection on the last band (the Clipboard / Files / AI band) so a band page's resting + /// lands the selection on the last band (the Clipboard band) so a band page's resting /// preview shows that band; otherwise it rests on the home band, exactly as the real launcher opens. func seed(from source: LauncherModel, landOnLastBand: Bool = false) { model.dwell = source.dwell @@ -1072,15 +649,12 @@ extension HubLauncherDemo { return GesturePose.DemoGesture(strokes: [open, band, items], liftGap: 0.6) } - /// The band-journey teaching gesture (Clipboard / Files / AI): a **four-finger** open, then — CONNECTED — + /// The band-journey teaching gesture (Clipboard): a **four-finger** open, then — CONNECTED — /// a long two-finger DOWN stroke that traverses the band list toward the last band, then a settle + lift. /// The traverse is target-based in the holder, so the exact stroke extent need only read as "down the /// bands"; the open length still tracks the activation distance. Coordinates are y-UP, so "down the band - /// list" descends from `topY` to the smaller `botY`. An optional `resolve` appends the AI canvas's - /// resolve tail: the traverse's lift *fires* the armed command, then a FRESH two-finger directed swipe - /// (the real canvas grammar — a stray re-lift is a no-op; resolution is a new excursion) commits it. - static func bandJourneyGesture(openLength: CGFloat, - resolve: GesturePose.BandInSurfaceGesture? = nil) -> GesturePose.DemoGesture { + /// list" descends from `topY` to the smaller `botY`. + static func bandJourneyGesture(openLength: CGFloat) -> GesturePose.DemoGesture { let openL = max(0.10, min(0.46, openLength)) let xL: CGFloat = 0.34, topY: CGFloat = 0.66, botY: CGFloat = 0.20 let open = GesturePose.Stroke(fingers: 4, @@ -1088,12 +662,7 @@ extension HubLauncherDemo { to: CGPoint(x: xL, y: topY), gapAfter: 0) let traverse = GesturePose.Stroke(fingers: 2, from: CGPoint(x: xL, y: topY), to: CGPoint(x: xL, y: botY), hold: 0.22) - var strokes = [open, traverse] - if let resolve, resolve != .lift { - // A fresh, centered two-finger resolve after the firing lift (the canvas-resolve vocabulary). - strokes.append(contentsOf: GesturePose.canvasResolve(resolve).strokes) - } - return GesturePose.DemoGesture(strokes: strokes, liftGap: 0.6) + return GesturePose.DemoGesture(strokes: [open, traverse], liftGap: 0.6) } /// Map the configurable activation threshold (`0.01…0.15`, the real trigger distance) to the demo's @@ -1115,11 +684,8 @@ extension HubLauncherDemo { /// The Launcher page's attract loop: open → one band down → scrub into the grid and across the /// items (arming the last one) → lift. case teaching - /// A band page's journey (Clipboard / Files): open → traverse the band list to the LAST band → lift. + /// A band page's journey (Clipboard): open → traverse the band list to the LAST band → lift. case bandJourney - /// The AI page's journey: open → traverse to the AI band → the firing lift → a fresh two-finger - /// canvas-resolve swipe (the tail drives nothing — the miniature never shows the canvas). - case bandJourneyResolve } /// The band-step / grid-step pace, matching the real launcher's snappy selection moves. @@ -1130,7 +696,7 @@ extension HubLauncherDemo { /// real launcher stays up to show the canvas. private static func finalStrokeIndex(_ script: SyncScript) -> Int { switch script { - case .teaching, .bandJourneyResolve: return 2 + case .teaching: return 2 case .bandJourney: return 1 } } @@ -1164,10 +730,9 @@ extension HubLauncherDemo { if pose.fraction >= 0.5 { walkBands(to: min(1, model.bandCount - 1)) } case (.teaching, 2): scrubItems(fraction: pose.fraction) - case (.bandJourney, 1), (.bandJourneyResolve, 1): + case (.bandJourney, 1): walkBands(to: model.bandCount - 1, fraction: pose.fraction) default: - // The AI resolve tail: the canvas isn't rendered in the miniature — nothing to drive. break } } @@ -1260,7 +825,7 @@ private struct LauncherDemoMiniature: View { var body: some View { let n = launcherNaturalSize(model) let h = min(n.height, 320) // a compact preview slot - LauncherView(model: model, executor: nil, availability: nil) + LauncherView(model: model) .frame(width: n.width, height: h) .scaleEffect(scale * (demo.overlayShown ? 1.0 : 0.92)) .opacity(demo.overlayShown ? 1 : 0) diff --git a/Sources/ThreeFingerSwitcher/Hub/HubFilesPage.swift b/Sources/ThreeFingerSwitcher/Hub/HubFilesPage.swift deleted file mode 100644 index d606442..0000000 --- a/Sources/ThreeFingerSwitcher/Hub/HubFilesPage.swift +++ /dev/null @@ -1,631 +0,0 @@ -import SwiftUI -import AppKit - -// The Files feature page — the configuration surface for the launcher's Files band (a local-only -// Finder-mimic column navigator). Re-homed onto a Hub page and bound to the same `AppSettings` -// properties (same keys/defaults/reset). Like the other feature pages it leads with its master -// enable toggle; every control persists live via `AppSettings`' `didSet`, so there is no Apply step. -// -// Sections, top to bottom: opt-in · Roots (the entry column) · Appearance · Behavior. - -struct FilesPage: View { - let context: HubContext - @ObservedObject private var settings: AppSettings - - /// §11.5 — the REAL launcher showing its FILES band: a `LauncherView` over the user's actual bands with a - /// synthetic Files band appended (the last band, like Clipboard / AI), seeded once and landed on it (the - /// resting frame shows the Files band). The ghost-hand autoplay plays `filesJourney` — 4-finger open → - /// 2-finger traverse down the band list → land/lift — and the preview's `sync` seam replays it on the - /// model (pop in, band walk to the Files band, pop out); clockless, visibility-gated, never grows (see - /// `docs/postmortem-idle-cpu-spin.md`). The Files band itself stays a STATIC seeded band (the user's - /// configured roots, no live `FilesColumnController` / drill controller). - @StateObject private var demo = HubLauncherDemo() - @State private var seeded = false - /// The hover-demo override pushed into the preview by the drill-resolution binding rows: hovering a row - /// demos that action's currently-bound excursion as a directed candidate gesture. `nil` ⇒ the base - /// open→band→lift journey. (Mirrors the AI page's `hoverGesture` bridge.) - @State private var hoverGesture: GesturePose.DemoGesture? - /// The excursion the hovered binding row maps to — stashed by the picker's `demoAxis` closure (an - /// event-handler call) so the `demo` closure can build the matching directed candidate gesture. The - /// shared `HubBindingPicker` speaks `GesturePose.Axis`; this bridges its hover signal to the preview's - /// `DemoGesture` candidate without changing the component (the AI page's exact idiom). - @State private var hoveredExcursion: GestureBindings.FilesExcursion? - - init(context: HubContext) { - self.context = context - _settings = ObservedObject(wrappedValue: context.settings) - } - - /// The preview's attract journey: open the four-finger launcher → traverse DOWN the band list to the - /// Files band (the real band-rail grammar, so the driven miniature can follow stroke for stroke) → land - /// and lift to open. The hover-demo override (`hoverGesture`) plays a candidate drill excursion instead. - private static let filesJourney = HubLauncherDemo.bandJourneyGesture(openLength: 0.30) - - /// The coarse axis a Files-drill excursion sweeps along — the `GesturePose.Axis` the shared - /// `HubBindingPicker` component expects from `demoAxis`. The two lift excursions land-and-open (the - /// journey's in-surface tail is a dwell-and-lift, no directional travel → `nil` = no axis), while the - /// four-finger horizontal discard demos a sideways move. - private func axis(for excursion: GestureBindings.FilesExcursion) -> GesturePose.Axis? { - switch excursion { - case .lift, .plusOneFingerLift: return nil - case .fourFingerHorizontal: return .horizontal - } - } - - /// Map a Files-drill excursion to the directed candidate gesture its hover-demo should play. The two - /// lift excursions replay the full land-and-open journey (their in-surface tail is a dwell-and-lift); - /// the four-finger horizontal discard demos a decisive four-finger sideways stroke (the dismiss - /// finger-count grammar — four fingers discard an open surface). - private func candidate(for excursion: GestureBindings.FilesExcursion) -> GesturePose.DemoGesture { - switch excursion { - case .lift, .plusOneFingerLift: - return Self.filesJourney - case .fourFingerHorizontal: - // A standalone four-finger leftward discard stroke (carries the hand angle/bow + a lift + loop). - let mid: CGFloat = 0.5 - return GesturePose.DemoGesture(strokes: [ - GesturePose.Stroke(fingers: 4, - from: CGPoint(x: 0.78, y: mid), - to: CGPoint(x: 0.22, y: mid)) - ], liftGap: 0.6) - } - } - - /// Seed once with the user's real bands + a synthetic Files band appended as the LAST band, then point - /// the holder's scrub at the last band (the Files band) so the traverse stroke lands on it. Degrades - /// gracefully through `HubPreviewModels` (the real bands always; sample roots when none are configured). - private func seedIfNeeded() { - guard !seeded else { return } - seeded = true - let models = HubPreviewModels(realWindowRows: context.realWindowRows, - seedThumbnails: context.seedThumbnails, - launcherBands: context.launcherBands) - // The user's real bands (clipboard / AI off here — this page demos the Files band), then a static - // Files band appended last, built from the configured roots (no live drill controller). Land the - // static preview on the Files band (the last band). - let base = models.makeLauncherModel(clipboardOn: false, aiOn: false, - dwell: settings.dwellToArmDuration) - demo.seed(from: appendingFilesBand(to: base), landOnLastBand: true) - } - - /// Build a fresh `LauncherModel` that is `base` plus a synthetic **Files band** appended as the last - /// band — a STATIC seeded band (no `FilesColumnController`), its column the user's configured roots (or - /// sample folders when none are set), so the demo's "traverse to the Files band" lands on a real-looking - /// Files band. The Files band's sentinel id / tint / icon come from `FilesBandBuilder`, so it reads - /// exactly like the live band; only the live drill is omitted (this is presentation-only). - private func appendingFilesBand(to base: LauncherModel) -> LauncherModel { - let filesBand = FilesBandBuilder.build(currentColumn: sampleFilesColumn()) - var bands = base.bands - var names = base.bandNames - var colors = base.bandColors - var icons = base.bandIcons - bands.append(filesBand.items) - names.append(filesBand.name) - colors.append(filesBand.color) - icons.append(filesBand.resolvedIcon) - let filesIndex = bands.count - 1 - - let model = LauncherModel() - model.dwell = base.dwell - model.setBands(bands, names: names, colors: colors, icons: icons, - startBand: 0, column: 0, - clipboardBandIndex: base.clipboardBandIndex, - filesBandIndex: filesIndex) - return model - } - - /// The Files band's current-column entries for the demo: each configured root as a folder `FileEntry` - /// (the band's first column is its roots). Falls back to a few common folders so the band reads alive - /// when the user has not configured any roots yet — degrading gracefully like the rest of the Hub demos. - private func sampleFilesColumn() -> [FileEntry] { - let paths = settings.filesRoots.isEmpty - ? ["~/Desktop", "~/Documents", "~/Downloads", "~/Pictures"].map { ($0 as NSString).expandingTildeInPath } - : settings.filesRoots - return paths.map { path in - let url = URL(fileURLWithPath: path) - let name = url.lastPathComponent.isEmpty ? path : url.lastPathComponent - return FileEntry(url: url, name: name, isDirectory: true, - modificationDate: nil, kind: .folder) - } - } - - var body: some View { - HubPage(HubDestination.files.title, - subtitle: "A four-finger Files band — pilot your local folders, preview, and open by trackpad.") { - HubSection(footnote: "Adds a local-only column navigator as a band in the four-finger launcher: drill into your folders horizontally, highlight vertically, lift to deliver the item to the app you came from — or open it (your choice) — and add a finger for the action menu. Reads the local filesystem on demand — no new permission, no logout, nothing copied off this Mac. Off by default.") { - HubFeatureHeader( - preview: HubGesturePreview( - gesture: Self.filesJourney, - hoverGesture: hoverGesture, - // The sync seam: the miniature replays the journey — pop in on the open beat, - // traverse the band list to the Files band, pop out on the lift-to-open. - sync: { demo.drive($0, script: .bandJourney) } - ) { - FilesDemoMiniature(demo: demo) - }, - icon: HubDestination.files.systemImage, - title: HubDestination.files.title, - subtitle: "Pilot your local folders, preview, and open them by trackpad.", - isOn: $settings.filesBandEnabled - ) - .onAppear { seedIfNeeded() } - } - - drillBindingSection - actionMenuSection - rootsSection - appearanceSection - behaviorSection - } - } - - // MARK: - Action menu (the +1-finger menu — what it offers, per type) - - /// The configuration for the action menu a `+1`-finger lift opens over the highlighted item: what a lift - /// itself does (deliver vs open), the per-type item lists (add / remove / reorder from the catalog), and - /// which detected terminals/editors appear under "Open in". Persists live via `AppSettings`. - private var actionMenuSection: some View { - HubSection("Action menu", - footnote: "A +1-finger lift opens an action menu over the highlighted item. Choose what a plain lift does, customize the menu for files and folders, and pick which terminals/editors it can open folders in. Defaults match the built-in menus.") { - Picker("When you lift on an item", selection: $settings.filesLiftAction) { - ForEach(FilesLiftAction.allCases) { Text(liftActionLabel($0)).tag($0) } - } - Text(settings.filesLiftAction == .deliver - ? "Lifting delivers the item to the app you came from — its path into a text field, the file into Finder. Add a finger for the action menu." - : "Lifting opens the item (file → default app, folder → Finder). Add a finger for the action menu.") - .font(.caption).foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - - Divider() - menuEditor(title: "File menu", isFolder: false) - Divider() - menuEditor(title: "Folder menu", isFolder: true) - Divider() - toolsCuration - } - .disabled(!settings.filesBandEnabled) - } - - /// One per-type menu editor: the ordered catalog items (reorder / remove), plus an "Add item" menu of the - /// catalog actions not already present. Mirrors `rootsSection`'s row affordances. - @ViewBuilder - private func menuEditor(title: String, isFolder: Bool) -> some View { - let items = isFolder ? settings.filesActionMenu.folderItems : settings.filesActionMenu.fileItems - VStack(alignment: .leading, spacing: 6) { - Text(title).font(.callout) - ForEach(Array(items.enumerated()), id: \.element) { index, action in - HStack(spacing: 8) { - Image(systemName: FilesBandView.menuRowGlyph(.action(action))) - .foregroundStyle(.secondary).frame(width: 18) - Text(FilesBandView.menuRowLabel(.action(action))).font(.callout) - Spacer(minLength: 8) - Button { moveMenuItem(isFolder: isFolder, index: index, by: -1) } label: { Image(systemName: "chevron.up") } - .buttonStyle(.borderless).foregroundStyle(.secondary).disabled(index == 0).help("Move up") - Button { moveMenuItem(isFolder: isFolder, index: index, by: 1) } label: { Image(systemName: "chevron.down") } - .buttonStyle(.borderless).foregroundStyle(.secondary).disabled(index == items.count - 1).help("Move down") - Button { removeMenuItem(isFolder: isFolder, action: action) } label: { Image(systemName: "minus.circle.fill") } - .buttonStyle(.borderless).foregroundStyle(.secondary).help("Remove from this menu") - } - .contentShape(Rectangle()) - } - let available = FilesMenuAction.allCases.filter { !items.contains($0) } - if !available.isEmpty { - Menu { - ForEach(available) { action in - Button { addMenuItem(isFolder: isFolder, action: action) } label: { - Label(FilesBandView.menuRowLabel(.action(action)), - systemImage: FilesBandView.menuRowGlyph(.action(action))) - } - } - } label: { Label("Add item", systemImage: "plus") } - .menuStyle(.borderlessButton).fixedSize() - } - } - } - - /// The detected terminals/editors, each toggleable — disabling one drops it from the folder menu's tool - /// rows and the folder "Open in" grid (curation = "all detected, minus the disabled"). - private var toolsCuration: some View { - let tools = installedTools(FilesToolCatalog.terminals, role: .terminal) - + installedTools(FilesToolCatalog.editors, role: .editor) - return VStack(alignment: .leading, spacing: 6) { - Text("Open folders in").font(.callout) - if tools.isEmpty { - Text("No supported terminals or editors detected on this Mac.") - .font(.caption).foregroundStyle(.secondary).fixedSize(horizontal: false, vertical: true) - } - ForEach(tools) { tool in - Toggle(isOn: toolEnabledBinding(tool.bundleID)) { - Label(tool.name, systemImage: tool.role == .editor - ? "chevron.left.forward.slash.chevron.right" : "terminal") - } - } - } - } - - /// The catalog tools of `role` that are actually installed (a bundle-id probe), with the user's - /// enable state applied — used only to render the curation toggles. - private func installedTools(_ seeds: [(bundleID: String, name: String)], role: FilesTool.Role) -> [FilesTool] { - seeds.compactMap { seed in - guard NSWorkspace.shared.urlForApplication(withBundleIdentifier: seed.bundleID) != nil else { return nil } - return FilesTool(bundleID: seed.bundleID, name: seed.name, role: role, - enabled: !settings.filesToolsDisabled.contains(seed.bundleID)) - } - } - - private func toolEnabledBinding(_ bundleID: String) -> Binding { - Binding(get: { !settings.filesToolsDisabled.contains(bundleID) }, - set: { on in - if on { settings.filesToolsDisabled.removeAll { $0 == bundleID } } - else if !settings.filesToolsDisabled.contains(bundleID) { settings.filesToolsDisabled.append(bundleID) } - }) - } - - private func addMenuItem(isFolder: Bool, action: FilesMenuAction) { - var menu = settings.filesActionMenu - if isFolder { menu.folderItems.append(action) } else { menu.fileItems.append(action) } - settings.filesActionMenu = menu - } - - private func removeMenuItem(isFolder: Bool, action: FilesMenuAction) { - var menu = settings.filesActionMenu - if isFolder { menu.folderItems.removeAll { $0 == action } } else { menu.fileItems.removeAll { $0 == action } } - settings.filesActionMenu = menu - } - - private func moveMenuItem(isFolder: Bool, index: Int, by delta: Int) { - var menu = settings.filesActionMenu - var list = isFolder ? menu.folderItems : menu.fileItems - let target = index + delta - guard list.indices.contains(index), list.indices.contains(target) else { return } - list.swapAt(index, target) - if isFolder { menu.folderItems = list } else { menu.fileItems = list } - settings.filesActionMenu = menu - } - - private func liftActionLabel(_ a: FilesLiftAction) -> String { - switch a { - case .deliver: return "Deliver it to the front app" - case .open: return "Open it" - } - } - - // MARK: - Drill resolution bindings (7.2) - - /// Map the Files drill's three resolution actions — open / Open-With / discard — onto its excursion - /// vocabulary (lift · +1-finger lift · four-finger sideways), via the shared `HubBindingPicker`. - /// Choosing routes through the pure `FilesDrillBinding.assigning(_:to:)`, which keeps the map one-to-one - /// (picking a taken move swaps it). Defaults to today's behavior; hovering a row demos it in the preview. - private var drillBindingSection: some View { - HubSection("Resolve gestures", - footnote: "Choose which trackpad move runs the lift action, opens the action menu, or discards the highlighted item while the Files band is open. Each move maps to one action — picking a taken move swaps it. A discard never closes a running app. Hover a row to preview the move above.") { - HubBindingPicker( - actions: GestureBindings.FilesAction.allCases, - excursions: GestureBindings.FilesExcursion.allCases, - actionLabel: HubBindingLabels.filesAction, - excursionLabel: HubBindingLabels.files, - current: { settings.gestureBindings.filesDrill.excursion(for: $0) }, - assign: { excursion, action in - settings.gestureBindings.filesDrill = settings.gestureBindings.filesDrill.assigning(excursion, to: action) - }, - demoAxis: { excursion in - // Stash the hovered excursion (event-handler context) so `demo` can build the matching - // directed candidate; return the coarse axis the component expects (nil for a lift). - hoveredExcursion = excursion - return axis(for: excursion) - }, - demo: { _ in - // The component signals enter (it just called `demoAxis`) / exit (nil). On enter, build a - // directed candidate for the hovered excursion; on exit, clear the override. Lift - // excursions have a nil axis but a real candidate (the land-and-open journey), so key the - // enter/exit off `hoveredExcursion` being set rather than the axis being non-nil. - hoverGesture = hoveredExcursion.map { candidate(for: $0) } - hoveredExcursion = nil - } - ) - .disabled(!settings.filesBandEnabled) - } - } - - // MARK: - Roots (12.2) - - /// The entry column: the local root folders the band opens onto. Add (a directories-only open - /// panel), remove, and reorder, persisting to `settings.filesRoots` (absolute paths). The app is - /// unsandboxed, so plain paths suffice — no security-scoped bookmarks. Non-local (network / iCloud) - /// selections are rejected with an inline note. - private var rootsSection: some View { - HubSection("Root folders", - footnote: "These are the band's first column — where each drill-in starts. Local folders only; network shares and iCloud-only locations are rejected. The band restores where you last left off in each root.") { - if settings.filesRoots.isEmpty { - Text("No roots yet. Add folders like Home, Desktop, Downloads, or any project folder.") - .font(.caption).foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - } else { - ForEach(Array(settings.filesRoots.enumerated()), id: \.element) { index, path in - rootRow(index: index, path: path) - if index < settings.filesRoots.count - 1 { Divider() } - } - } - - if let rejection { - Label(rejection, systemImage: "exclamationmark.triangle.fill") - .font(.caption) - .foregroundStyle(.orange) - .fixedSize(horizontal: false, vertical: true) - } - - HStack { - Button { addRoots() } label: { Label("Add folder…", systemImage: "plus") } - Spacer() - } - } - .disabled(!settings.filesBandEnabled) - } - - private func rootRow(index: Int, path: String) -> some View { - let url = URL(fileURLWithPath: path) - return HStack(spacing: 10) { - Image(nsImage: NSWorkspace.shared.icon(forFile: path)) - .resizable().frame(width: 18, height: 18) - VStack(alignment: .leading, spacing: 1) { - Text(url.lastPathComponent.isEmpty ? path : url.lastPathComponent).font(.callout) - Text(abbreviate(path)).font(.caption).foregroundStyle(.secondary) - .lineLimit(1).truncationMode(.middle) - } - Spacer(minLength: 8) - // Reorder: a root higher in the list is offered higher in the band's entry column. - Button { move(index, by: -1) } label: { Image(systemName: "chevron.up") } - .buttonStyle(.borderless).foregroundStyle(.secondary) - .disabled(index == 0) - .help("Move up") - Button { move(index, by: 1) } label: { Image(systemName: "chevron.down") } - .buttonStyle(.borderless).foregroundStyle(.secondary) - .disabled(index == settings.filesRoots.count - 1) - .help("Move down") - Button { settings.filesRoots.removeAll { $0 == path } } label: { - Image(systemName: "minus.circle.fill") - } - .buttonStyle(.borderless).foregroundStyle(.secondary) - .help("Remove this root") - } - .contentShape(Rectangle()) - } - - // MARK: - Appearance (12.3) - - /// Column width / density / tint / icon-vs-preview. Width and tint are the bounded-overlay knobs; - /// density packs the current-list rows; the tint is the band's accent (stored as a `#RRGGBB` hex). - private var appearanceSection: some View { - HubSection("Appearance") { - LabeledSlider(title: "Column width", value: $settings.filesColumnWidth, - range: 180...420, format: "%.0f pt", - help: "Width of the current-folder column. The overlay stays bounded at any depth — ancestors collapse to a thin icon rail.") - - Picker("Row density", selection: $settings.filesDensity) { - ForEach(FilesDensity.allCases) { Text(densityLabel($0)).tag($0) } - } - - Picker("Row leading glyph", selection: $settings.filesIconStyle) { - ForEach(FilesIconStyle.allCases) { Text(iconStyleLabel($0)).tag($0) } - } - Text(settings.filesIconStyle == .preview - ? "Rows lead with a QuickLook thumbnail when one is available (icon fallback)." - : "Rows lead with the cheap file/folder type icon — no QuickLook churn while scrubbing.") - .font(.caption).foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - - ColorPicker("Accent tint", selection: tintBinding, supportsOpacity: false) - } - .disabled(!settings.filesBandEnabled) - } - - // MARK: - Behavior (12.4) - - /// Sort field + direction · default-open action · which metadata a row shows. The metadata is an - /// `OptionSet`, so it's a set of independent toggles (any/all may show at once). - private var behaviorSection: some View { - HubSection("Behavior") { - Picker("Sort folders by", selection: $settings.filesSortField) { - ForEach(FilesSortField.allCases) { Text(sortFieldLabel($0)).tag($0) } - } - Picker("Order", selection: $settings.filesSortDirection) { - ForEach(FilesSortDirection.allCases) { Text(sortDirectionLabel($0)).tag($0) } - } - .pickerStyle(.segmented) - - Divider() - - Picker("When you open a file", selection: $settings.filesDefaultOpen) { - ForEach(FilesDefaultOpen.allCases) { Text(defaultOpenLabel($0)).tag($0) } - } - Text("Applies when the lift action above is set to Open. " - + (settings.filesDefaultOpen == .openWith - ? "Opening a file shows the Open-With chooser." - : "Opening a file launches its default app.") - + " (Folders always open as a Finder window; the action menu’s “Open in ▸” is always available.)") - .font(.caption).foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - - Divider() - - VStack(alignment: .leading, spacing: 6) { - Text("Show beside each name").font(.callout) - Toggle("Modified date", isOn: metadataBinding(.date)) - Toggle("Kind", isOn: metadataBinding(.kind)) - Toggle("Size", isOn: metadataBinding(.size)) - } - - Divider() - - // Restore-last-folder (refinement 2): when on (default), opening the Files band lands on AND - // displays the last folder visited in that root — restored AT OPEN, so the column already shows - // it while the highlight is still on the band icon, and crossing horizontally lands there with no - // jump. Off opens fresh on the configured roots list. - VStack(alignment: .leading, spacing: 6) { - Toggle("Remember and reopen the last folder", isOn: $settings.filesRememberLocation) - Text(settings.filesRememberLocation - ? "Opening the Files band lands on the last folder you visited in that root — already shown before you cross in." - : "The Files band opens fresh on your root folders each time.") - .font(.caption).foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - } - } - .disabled(!settings.filesBandEnabled) - } - - // MARK: - Roots actions - - @State private var rejection: String? - - /// Present a directories-only open panel and append every accepted **local** selection. Network / - /// iCloud selections are dropped with an inline note. Unsandboxed ⇒ plain absolute paths, no bookmarks. - private func addRoots() { - rejection = nil - let panel = NSOpenPanel() - panel.canChooseDirectories = true - panel.canChooseFiles = false - panel.allowsMultipleSelection = true - panel.prompt = "Add" - panel.message = "Choose local folders to use as Files-band roots." - guard panel.runModal() == .OK else { return } - - var rejected: [String] = [] - for url in panel.urls { - let path = url.standardizedFileURL.path - guard Self.isLocalFolder(url) else { rejected.append(url.lastPathComponent); continue } - if !settings.filesRoots.contains(path) { settings.filesRoots.append(path) } - } - if !rejected.isEmpty { - rejection = "Skipped \(rejected.joined(separator: ", ")): the Files band is local-only (network shares and iCloud-only folders aren't supported)." - } - } - - /// Move the root at `index` by `delta` (clamped), nudging the band's entry-column order. - private func move(_ index: Int, by delta: Int) { - let target = index + delta - guard settings.filesRoots.indices.contains(index), - settings.filesRoots.indices.contains(target) else { return } - settings.filesRoots.swapAt(index, target) - } - - /// Local-only gate for a chosen root: reject iCloud placeholders and non-local (network) volumes. - /// Mirrors the lister's `.isUbiquitousItemKey` skip, adding `.volumeIsLocalKey` for network shares. - private static func isLocalFolder(_ url: URL) -> Bool { - let values = try? url.resourceValues(forKeys: [.isUbiquitousItemKey, .volumeIsLocalKey]) - if values?.isUbiquitousItem == true { return false } - if let isLocal = values?.volumeIsLocal { return isLocal } - return true // unknown volume locality ⇒ allow (e.g. the boot volume reports nil on some setups) - } - - /// Collapse a long home-relative path to `~/…` for the row's secondary line. - private func abbreviate(_ path: String) -> String { - (path as NSString).abbreviatingWithTildeInPath - } - - // MARK: - Bindings - - /// Bridge the persisted `#RRGGBB` hex string to/from a SwiftUI `Color` for the `ColorPicker`. The - /// app keeps the tint as a single plist-native hex property (no nested model), so the conversion - /// lives here rather than on `AppSettings`. - private var tintBinding: Binding { - Binding(get: { Self.color(fromHex: settings.filesBandTint) }, - set: { settings.filesBandTint = Self.hex(from: $0) }) - } - - /// A single member of the `filesRowMetadata` `OptionSet` as a Bool toggle. - private func metadataBinding(_ option: FilesRowMetadata) -> Binding { - Binding(get: { settings.filesRowMetadata.contains(option) }, - set: { on in - var set = settings.filesRowMetadata - if on { set.insert(option) } else { set.remove(option) } - settings.filesRowMetadata = set - }) - } - - // MARK: - Hex ⇄ Color (via NSColor sRGB, matching the app's ItemColor bridge) - - private static func color(fromHex hex: String) -> Color { - var s = hex.trimmingCharacters(in: .whitespacesAndNewlines) - if s.hasPrefix("#") { s.removeFirst() } - guard s.count == 6, let value = UInt32(s, radix: 16) else { return .accentColor } - let r = Double((value >> 16) & 0xFF) / 255 - let g = Double((value >> 8) & 0xFF) / 255 - let b = Double(value & 0xFF) / 255 - return Color(.sRGB, red: r, green: g, blue: b, opacity: 1) - } - - private static func hex(from color: Color) -> String { - let ns = (NSColor(color).usingColorSpace(.sRGB)) ?? .gray - let r = Int((ns.redComponent * 255).rounded()) - let g = Int((ns.greenComponent * 255).rounded()) - let b = Int((ns.blueComponent * 255).rounded()) - return String(format: "#%02X%02X%02X", r, g, b) - } - - // MARK: - Enum labels - - private func densityLabel(_ d: FilesDensity) -> String { - switch d { - case .compact: return "Compact" - case .comfortable: return "Comfortable" - case .spacious: return "Spacious" - } - } - - private func iconStyleLabel(_ s: FilesIconStyle) -> String { - switch s { - case .icon: return "Type icon" - case .preview: return "Preview thumbnail" - } - } - - private func sortFieldLabel(_ f: FilesSortField) -> String { - switch f { - case .name: return "Name" - case .date: return "Date modified" - case .kind: return "Kind" - } - } - - private func sortDirectionLabel(_ d: FilesSortDirection) -> String { - switch d { - case .ascending: return "Ascending" - case .descending: return "Descending" - } - } - - private func defaultOpenLabel(_ o: FilesDefaultOpen) -> String { - switch o { - case .defaultApp: return "Open in the default app" - case .openWith: return "Show the Open-With chooser" - } - } -} - -// MARK: - §14 Files demo miniature (the real launcher, showing the Files band) - -/// The §14 Files-page miniature: the **real** `LauncherView` over the holder's seeded model (the user's bands -/// + a synthetic Files band appended last), shown small (preview scale) — the preview "playing its part." It -/// is **static** (the ghost-hand autoplay plays over it; the model is not driven) and takes no hits. -private struct FilesDemoMiniature: View { - @ObservedObject var demo: HubLauncherDemo - @ObservedObject private var model: LauncherModel - - init(demo: HubLauncherDemo) { - self.demo = demo - _model = ObservedObject(wrappedValue: demo.model) - } - - private let scale: CGFloat = 0.46 - - var body: some View { - let n = launcherNaturalSize(model) - let h = min(n.height, 320) - LauncherView(model: model, executor: nil, availability: nil) - .frame(width: n.width, height: h) - .scaleEffect(scale * (demo.overlayShown ? 1.0 : 0.92)) - .opacity(demo.overlayShown ? 1 : 0) - .frame(width: n.width * scale, height: h * scale) - .allowsHitTesting(false) - } -} diff --git a/Sources/ThreeFingerSwitcher/Hub/HubFleetRosterView.swift b/Sources/ThreeFingerSwitcher/Hub/HubFleetRosterView.swift deleted file mode 100644 index 5de744e..0000000 --- a/Sources/ThreeFingerSwitcher/Hub/HubFleetRosterView.swift +++ /dev/null @@ -1,354 +0,0 @@ -import SwiftUI -import AppKit - -/// The Hub AI-page FLEET ROSTER (design D6, tasks 7.1–7.4). Evolves the single model picker into a list -/// of fleet members — each row shows its ROLE (Chat / Ternary / Image / Video / Cloud), LANE (GPU / CPU / -/// Cloud), PROVIDER, the existing per-model on-disk/resident status, and its HONEST residency cost -/// (`residencyBytes` in GB) in the same breath it offers selection. -/// -/// FLAGGED: user xcodebuild + stable-signed build. This is native SwiftUI in the app target — the agent -/// compile-verifies it under `swift build`; the LIVE roster rendering, real per-model status, the -/// disclosure copy, and the gated cloud rows are verified only in the user's stable-signed build (§8.3). -/// -/// Honest-surface invariants it upholds: -/// - The **evict-chat disclosure** is computed from the `ResidencyPlanner` PLAN (task 7.2), NOT hard-coded -/// — a member whose admission evicts chat shows "selecting ‹Role› pauses the chat model; it reloads -/// when generation finishes." It therefore shows for Video / FP16-image and NOT for Q4-image. -/// - **Cloud members** show a Cloud badge + escalation cost and are DISABLED with an explanatory caption -/// until `fleetCloudEscalationEnabled` (task 7.3). A fleet-of-one renders as today's single picker. -/// - Any admission/escalation failure is a **bounded, non-blocking** row (clean headline via -/// `AIError.message(for:)` + opt-in copyable details + Retry) — never an `NSAlert`, never raw error -/// text in a headline (task 7.4, D7). -struct HubFleetRosterView: View { - /// The fleet roster (descriptors + the resident view). Defaults to the standard fleet. - var roster: FleetRoster = .standard - /// The pure planner driving the evict-chat disclosure (task 7.2). - var planner: ResidencyPlanner = ResidencyPlanner() - /// The unified-memory budget the planner spends against. - var budgetBytes: UInt64 = FleetRoster.unifiedBudget48GB - /// Live free-memory probe (injected; pure value in previews/tests). - var freeBytes: () -> UInt64 = { FleetRoster.unifiedBudget48GB } - /// `fleetCloudEscalationEnabled` (OWNED by `ai-full-potential-toggle`; consumed here, default false). - var cloudEscalationEnabled: () -> Bool = { false } - - /// The pinned ACTIVE CHAT model id — a radio AMONG chat-role members (nil → the chat default). A chat - /// model is the single resident conversational brain, so it is a mutually-exclusive choice. - @Binding var activeChatID: String? - - /// The set of ENABLED capability-model ids (image / ternary / video) — INDEPENDENT toggles, NOT a - /// radio: these co-reside with (or evict around) chat rather than replacing it. On-enable the row - /// inserts the id AND triggers a download if the weights are not yet on disk. - @Binding var enabledCapabilityModelIDs: Set - - /// Trigger a download for a capability model the user just enabled (wired to the existing - /// `ModelManager.download/downloadAndVerify` path in `AppCoordinator`). A side effect that didn't land - /// surfaces through the manager's `.failed` state on that row — never a false "Done". - var onDownloadCapabilityModel: (ModelDescriptor) -> Void = { _ in } - - /// The manager whose per-model lifecycle status each row reflects. - @ObservedObject var manager: ModelManager - - /// Whether AI commands are enabled at all (the whole section is disabled otherwise). - var aiEnabled: Bool - - /// A bounded, non-blocking failure surfaced inline (task 7.4) — never a modal. - @State private var failure: AIPresentedError? - @State private var showingFailureDetails = false - - private var members: [ModelDescriptor] { roster.descriptors() } - - /// A fleet-of-one renders as today's single picker (task 7.3). - private var isFleetOfOne: Bool { members.count <= 1 } - - var body: some View { - VStack(alignment: .leading, spacing: 10) { - if isFleetOfOne { - singlePicker - } else { - ForEach(members, id: \.id) { member in - memberRow(member) - Divider() - } - } - if let failure { failureRow(failure) } - } - .disabled(!aiEnabled) - } - - // MARK: - Fleet-of-one (today's single picker) - - @ViewBuilder private var singlePicker: some View { - if let only = members.first { - HStack(spacing: 8) { - Image(systemName: "brain.head.profile").foregroundStyle(.secondary) - VStack(alignment: .leading, spacing: 1) { - Text(only.displayName).font(.headline) - Text(costLabel(only)).font(.caption).foregroundStyle(.secondary) - } - Spacer() - } - } - } - - // MARK: - One member row - - @ViewBuilder private func memberRow(_ member: ModelDescriptor) -> some View { - let isCloud = member.provider == .cloud - let cloudOn = cloudEscalationEnabled() - let disabled = isCloud && !cloudOn - let isChat = member.role == .chat - - VStack(alignment: .leading, spacing: 4) { - HStack(spacing: 8) { - Image(systemName: roleIcon(member.role)) - .foregroundStyle(disabled ? Color.secondary : .primary) - VStack(alignment: .leading, spacing: 2) { - Text(member.displayName) - .font(.subheadline.weight(.medium)) - .foregroundStyle(disabled ? Color.secondary : .primary) - HStack(spacing: 6) { - badge(roleTitle(member.role), color: .accentColor) - badge(laneTitle(member.lane), color: .secondary) - if isCloud { badge("Cloud", color: .purple) } - Text(costLabel(member)).font(.caption2).foregroundStyle(.secondary) - } - // Per-row lifecycle status (on-disk / downloading% + progress / resident), reusing the - // manager's pure per-descriptor probe — each row reflects its OWN model. - statusLine(member) - } - Spacer() - if isCloud { - // Cloud rows keep the radio-style Select (it pins the active chat target only when the - // gate is on — escalation routing is owned elsewhere); gated off otherwise. - if activeChatID == member.id { - Image(systemName: "checkmark.circle.fill").foregroundStyle(.green) - } else { - Button("Select") { selectChat(member) } - .controlSize(.small) - .disabled(disabled) - } - } else if isChat { - // Chat members are a RADIO — selecting one pins it as the active conversational brain. - if isActiveChat(member) { - Image(systemName: "checkmark.circle.fill").foregroundStyle(.green) - } else { - Button("Select") { selectChat(member) } - .controlSize(.small) - } - } else { - // Capability members (image / ternary / video) are INDEPENDENT toggles. - Toggle("", isOn: capabilityBinding(member)) - .labelsHidden() - .toggleStyle(.switch) - .controlSize(.small) - } - } - - // The honest-cost disclosure (task 7.2): computed from the PLAN, never hard-coded. - if !isCloud, evictsChat(member) { - disclosure("Selecting \(roleTitle(member.role)) pauses the chat model; it reloads when generation finishes.", - systemImage: "pause.circle") - } - // Cloud caption (task 7.3): gated off → explain why it's disabled. - if isCloud { - if cloudOn { - disclosure("Cloud escalation — routes this turn off-device (confirm-by-default, per-day budget cap).", - systemImage: "cloud") - } else { - disclosure("Cloud model. Turn on cloud escalation to use it (off by default — no silent spend).", - systemImage: "lock") - } - } - } - .padding(.vertical, 2) - } - - /// One line of per-model lifecycle status, read non-mutatingly from the manager (`status(for:)`): - /// on-disk, downloading with a live percentage + progress bar, verifying/loading, resident, or a - /// bounded failure headline. Each fleet row shows its OWN model's status (the manager's single - /// displayed `state` describes only one model at a time). - @ViewBuilder private func statusLine(_ member: ModelDescriptor) -> some View { - switch manager.status(for: member) { - case .notDownloaded: - EmptyView() - case let .downloading(progress): - VStack(alignment: .leading, spacing: 2) { - Text("Downloading… \(Int(progress * 100))%") - .font(.caption2).foregroundStyle(Color.accentColor) - ProgressView(value: progress).frame(maxWidth: 160) - } - case .verifying: - Text("Verifying…").font(.caption2).foregroundStyle(Color.accentColor) - case .ready: - Label("On disk", systemImage: "internaldrive").font(.caption2).foregroundStyle(.secondary) - case .loading: - Text("Loading…").font(.caption2).foregroundStyle(Color.accentColor) - case .loaded: - Label("Resident", systemImage: "checkmark.circle.fill").font(.caption2).foregroundStyle(.green) - case let .failed(reason, _): - Label(reason, systemImage: "exclamationmark.triangle.fill") - .font(.caption2).foregroundStyle(.red) - .lineLimit(2).truncationMode(.middle) - } - } - - // MARK: - Selection / toggles - - /// Whether `member` is the active chat radio selection (an explicit pin, or the chat default when none - /// is pinned and this is a chat member). - private func isActiveChat(_ member: ModelDescriptor) -> Bool { - if let id = activeChatID { return id == member.id } - return member.role == .chat && member.id == defaultChatID - } - - /// The id treated as the chat default when nothing is pinned (the first chat-role member). - private var defaultChatID: String? { members.first(where: { $0.role == .chat })?.id } - - /// Pin a chat (or gated cloud) member as the active conversational brain. A cloud selection while the - /// gate is off is a refusal surfaced through the ONE translator, bounded + inline. - private func selectChat(_ member: ModelDescriptor) { - failure = nil - if member.provider == .cloud && !cloudEscalationEnabled() { - failure = AIError.message(for: FleetError.cloudDisabled(modelName: member.displayName)) - return - } - activeChatID = member.id - } - - /// The independent on/off binding for a capability member. On-enable it inserts the id AND triggers a - /// download if the weights are not already on disk (residency itself stays lazy until a command runs). - private func capabilityBinding(_ member: ModelDescriptor) -> Binding { - Binding( - get: { enabledCapabilityModelIDs.contains(member.id) }, - set: { isOn in - failure = nil - if isOn { - enabledCapabilityModelIDs.insert(member.id) - // Trigger a download only when the weights aren't already present — re-enabling an - // on-disk model is a no-op fetch. - if !isOnDisk(member) { onDownloadCapabilityModel(member) } - } else { - enabledCapabilityModelIDs.remove(member.id) - } - } - ) - } - - /// Whether a capability model's weights are already on disk (so re-enabling needn't re-download). - private func isOnDisk(_ member: ModelDescriptor) -> Bool { - switch manager.status(for: member) { - case .ready, .loaded: return true - default: return false - } - } - - /// The bounded, non-blocking failure row (task 7.4): clean headline + opt-in details + Retry. NEVER a - /// modal `NSAlert`, never raw error text in the headline. - @ViewBuilder private func failureRow(_ presented: AIPresentedError) -> some View { - VStack(alignment: .leading, spacing: 6) { - Label { - Text(presented.headline) - .font(.caption) - .lineLimit(3) - .truncationMode(.middle) - .fixedSize(horizontal: false, vertical: true) - .textSelection(.enabled) - } icon: { - Image(systemName: "exclamationmark.triangle.fill").foregroundStyle(.red) - } - if let details = presented.details, !details.isEmpty { - DisclosureGroup(isExpanded: $showingFailureDetails) { - ScrollView { - Text(details) - .font(.system(.caption2, design: .monospaced)) - .foregroundStyle(.secondary) - .textSelection(.enabled) - .frame(maxWidth: .infinity, alignment: .leading) - } - .frame(maxHeight: 100) - Button { - let pb = NSPasteboard.general - pb.clearContents() - pb.setString(details, forType: .string) - } label: { Label("Copy details", systemImage: "doc.on.doc") } - .controlSize(.small) - } label: { - Text("Show details").font(.caption).foregroundStyle(.secondary) - } - } - Button("Dismiss") { failure = nil }.controlSize(.small) - } - .padding(8) - .background(RoundedRectangle(cornerRadius: 6).fill(Color.red.opacity(0.06))) - } - - // MARK: - Plan-driven disclosure - - /// Whether admitting `member` would evict chat — straight from the planner (task 7.2). - private func evictsChat(_ member: ModelDescriptor) -> Bool { - // Assume chat resident (the resting state) so the disclosure reflects the real trade-off. - let chatIDs = members.filter { $0.role == .chat }.map(\.id) - let ternaryIDs = members.filter { $0.role == .ternaryChat }.map(\.id) - return planner.admissionEvictsChat(targetID: member.id, - descriptors: members, - budgetBytes: budgetBytes, - freeBytes: freeBytes(), - currentlyResident: chatIDs + ternaryIDs) - } - - // MARK: - Labels - - private func roleTitle(_ role: ModelRole) -> String { - switch role { - case .chat: return "Chat" - case .ternaryChat: return "Ternary" - case .image: return "Image" - case .video: return "Video" - case .cloudEscalation: return "Cloud" - } - } - - private func roleIcon(_ role: ModelRole) -> String { - switch role { - case .chat: return "brain.head.profile" - case .ternaryChat: return "cpu" - case .image: return "photo" - case .video: return "film" - case .cloudEscalation: return "cloud" - } - } - - private func laneTitle(_ lane: ComputeLane?) -> String { - switch lane { - case .gpu: return "GPU" - case .cpuTernary: return "CPU" - case .none: return "Cloud" - } - } - - /// The honest residency cost: resident footprint in GB (0 for cloud → "no local memory"). - private func costLabel(_ member: ModelDescriptor) -> String { - if member.provider == .cloud { return "Cloud · no local memory" } - let formatter = ByteCountFormatter() - formatter.allowedUnits = [.useGB, .useMB] - formatter.countStyle = .memory - return "Resident: \(formatter.string(fromByteCount: Int64(member.residencyBytes)))" - } - - private func badge(_ text: String, color: Color) -> some View { - Text(text) - .font(.caption2.weight(.semibold)) - .padding(.horizontal, 5).padding(.vertical, 1) - .background(RoundedRectangle(cornerRadius: 4).fill(color.opacity(0.15))) - .foregroundStyle(color) - } - - private func disclosure(_ text: String, systemImage: String) -> some View { - Label { - Text(text).font(.caption2).foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - } icon: { - Image(systemName: systemImage).foregroundStyle(.secondary) - } - } -} diff --git a/Sources/ThreeFingerSwitcher/Hub/HubGesturePreview.swift b/Sources/ThreeFingerSwitcher/Hub/HubGesturePreview.swift index 16e100d..f8a2061 100644 --- a/Sources/ThreeFingerSwitcher/Hub/HubGesturePreview.swift +++ b/Sources/ThreeFingerSwitcher/Hub/HubGesturePreview.swift @@ -10,7 +10,7 @@ import CoreGraphics /// /// ## Autoplay-only (deliberately simple) /// This is **pure autoplay**: a `TimelineView(.periodic)` advances a continuous `phase` through the shared, -/// MLX-free directed-stroke pose driver (`GesturePose.pose(phase:gesture:)`) and renders the resulting ghost +/// pure directed-stroke pose driver (`GesturePose.pose(phase:gesture:)`) and renders the resulting ghost /// fingertips on the pad. The miniature is whatever the caller draws; the preview itself touches no overlay /// model — a page that wants its miniature to follow the hand observes the optional `sync` seam (quantized /// pose frames, emitted only while the clock runs) and steps its own model. Two autoplay states layer over diff --git a/Sources/ThreeFingerSwitcher/Hub/HubOverviewPage.swift b/Sources/ThreeFingerSwitcher/Hub/HubOverviewPage.swift index cb03806..88a53d6 100644 --- a/Sources/ThreeFingerSwitcher/Hub/HubOverviewPage.swift +++ b/Sources/ThreeFingerSwitcher/Hub/HubOverviewPage.swift @@ -28,12 +28,6 @@ struct OverviewPage: View { featureRow(.clipboard, isOn: $settings.keepClipboardHistory, subtitle: "Keep a history of what you copy, in the launcher's Clipboard band.") Divider() - featureRow(.files, isOn: $settings.filesBandEnabled, - subtitle: "Pilot your local folders, preview, and open files — a Files band in the launcher.") - Divider() - featureRow(.ai, isOn: $settings.aiCommandsEnabled, - subtitle: "Run on-device AI commands on your selection, clipboard, or screen.") - Divider() featureRow(.keyboardLanguage, isOn: $settings.keyboardLanguageEnabled, subtitle: "Remember and auto-switch the keyboard language per app.") } diff --git a/Sources/ThreeFingerSwitcher/Hub/HubPreviewModels.swift b/Sources/ThreeFingerSwitcher/Hub/HubPreviewModels.swift index 181494b..b0832ba 100644 --- a/Sources/ThreeFingerSwitcher/Hub/HubPreviewModels.swift +++ b/Sources/ThreeFingerSwitcher/Hub/HubPreviewModels.swift @@ -16,11 +16,11 @@ import AppKit struct HubPreviewModels { private let realWindowRows: () -> [[WindowInfo]] private let seedThumbnails: (SwitcherModel) -> Void - private let launcherBands: (_ clipboardOn: Bool, _ aiOn: Bool) -> [ContextBand] + private let launcherBands: (_ clipboardOn: Bool) -> [ContextBand] init(realWindowRows: @escaping () -> [[WindowInfo]], seedThumbnails: @escaping (SwitcherModel) -> Void, - launcherBands: @escaping (_ clipboardOn: Bool, _ aiOn: Bool) -> [ContextBand]) { + launcherBands: @escaping (_ clipboardOn: Bool) -> [ContextBand]) { self.realWindowRows = realWindowRows self.seedThumbnails = seedThumbnails self.launcherBands = launcherBands @@ -59,10 +59,10 @@ struct HubPreviewModels { /// Build a `LauncherModel` seeded from the user's bands for the given toggles. `dwell` comes from /// the caller's settings (`dwellToArmDuration`). Lands on the band list at the home band (nothing /// armed), exactly as the real launcher / the wizard tour do. - func makeLauncherModel(clipboardOn: Bool, aiOn: Bool, dwell: Double) -> LauncherModel { + func makeLauncherModel(clipboardOn: Bool, dwell: Double) -> LauncherModel { let model = LauncherModel() model.dwell = dwell - let bands = launcherBands(clipboardOn, aiOn) + let bands = launcherBands(clipboardOn) guard !bands.isEmpty else { return model } model.setBands(bands.map(\.items), names: bands.map(\.name), diff --git a/Sources/ThreeFingerSwitcher/Hub/HubView.swift b/Sources/ThreeFingerSwitcher/Hub/HubView.swift index 507fe86..b63a6d9 100644 --- a/Sources/ThreeFingerSwitcher/Hub/HubView.swift +++ b/Sources/ThreeFingerSwitcher/Hub/HubView.swift @@ -2,12 +2,12 @@ import SwiftUI import AppKit /// The pages of the configuration Hub, as sidebar destinations. One window, grouped navigation: -/// Overview · Content(Bands) · Features(Switcher/Launcher/Clipboard/AI) · System(Setup/General). +/// Overview · Content(Bands) · Features(Switcher/Launcher/Clipboard) · System(Setup/General). /// Space-row switching is a sub-feature of the Switcher and lives on the Switcher page (no own destination). enum HubDestination: Hashable, CaseIterable { case overview case bands - case switcher, launcher, clipboard, files, ai, keyboardLanguage + case switcher, launcher, clipboard, keyboardLanguage case devices case setup, general @@ -18,8 +18,6 @@ enum HubDestination: Hashable, CaseIterable { case .switcher: return "Window Switcher" case .launcher: return "Launcher" case .clipboard: return "Clipboard" - case .files: return "Files" - case .ai: return "AI Commands" case .keyboardLanguage: return "Keyboard Language" case .devices: return "Devices" case .setup: return "Setup & Permissions" @@ -31,7 +29,6 @@ enum HubDestination: Hashable, CaseIterable { var sidebarTitle: String { switch self { case .switcher: return "Switcher" - case .ai: return "AI" case .keyboardLanguage: return "Language" case .setup: return "Setup" default: return title @@ -45,8 +42,6 @@ enum HubDestination: Hashable, CaseIterable { case .switcher: return "arrow.left.arrow.right" case .launcher: return "square.grid.3x3.fill" case .clipboard: return "doc.on.clipboard" - case .files: return "folder" - case .ai: return "sparkles" case .keyboardLanguage: return "globe" case .devices: return "iphone.and.arrow.forward" case .setup: return "gearshape.2" @@ -71,7 +66,6 @@ final class HubContext { let settings: AppSettings let favorites: FavoritesStore let clipboard: ClipboardStore - let models: ModelManager let permissions: PermissionsService /// The gesture-preview VISIBILITY gate. `AppCoordinator` flips `isActive` false the instant the Hub @@ -92,29 +86,12 @@ final class HubContext { var inspectWindows: () -> [WindowInspectorEntry] = { [] } /// Seed/prefetch live thumbnails into the passed demo model (post-Screen-Recording reveal). var seedThumbnails: (SwitcherModel) -> Void = { _ in } - /// The launcher's bands for the current toggles (favorites + AI/Clipboard when on). - var launcherBands: (_ clipboardOn: Bool, _ aiOn: Bool) -> [ContextBand] = { _, _ in [] } + /// The launcher's bands for the current toggles (favorites + Clipboard when on). + var launcherBands: (_ clipboardOn: Bool) -> [ContextBand] = { _ in [] } // Clipboard feature page. var onClearClipboard: (_ includingPinned: Bool) -> Void = { _ in } - // AI feature page. - var onDownloadModel: () -> Void = {} - /// Download a SPECIFIC capability fleet model (image / ternary / video) the user just enabled in the - /// roster. Routes to the EXISTING `ModelManager` download path in the coordinator — no new seam. A - /// failure surfaces through the manager's per-row `.failed` state, never a false "Done". - var onDownloadCapabilityModel: (ModelDescriptor) -> Void = { _ in } - - // AI feature page — Background autonomy (`ai-background-autonomy`, §7). The whitelist editor binds - // directly to `settings.agentWhitelistPaths` / `agentWhitelistCommands` (no provider needed). The - // audit viewer reads recent records synchronously through this seam (the Core `AuditLog`), and the - // store-persist failure surfaces as a bounded, non-blocking banner via the optional headline. - /// The most-recent audit records, reverse-chronological, capped at `limit`. - var recentAuditRecords: (_ limit: Int) -> [AuditRecord] = { _ in [] } - /// A clean, bounded headline if the durable audit store last failed to persist (else `nil`). Routed - /// through `AIError.message(for:)` so it never carries raw OS text. - var auditStorePersistError: () -> AIPresentedError? = { nil } - // Keyboard Language feature page — the picker's source list (read fresh on each render). Provided by // the coordinator so the page never imports Carbon directly (the list comes from the service's // `InputSourceController`). @@ -171,12 +148,10 @@ final class HubContext { init(settings: AppSettings, favorites: FavoritesStore, clipboard: ClipboardStore, - models: ModelManager, permissions: PermissionsService) { self.settings = settings self.favorites = favorites self.clipboard = clipboard - self.models = models self.permissions = permissions } } @@ -235,7 +210,7 @@ struct HubView: View { railDivider railButton(.bands) railDivider - railButton(.switcher); railButton(.launcher); railButton(.clipboard); railButton(.files); railButton(.ai); railButton(.keyboardLanguage); railButton(.devices) + railButton(.switcher); railButton(.launcher); railButton(.clipboard); railButton(.keyboardLanguage); railButton(.devices) railDivider railButton(.setup); railButton(.general) } @@ -287,8 +262,6 @@ struct HubView: View { case .switcher: SwitcherPage(context: context) case .launcher: LauncherPage(context: context) case .clipboard: ClipboardPage(context: context) - case .files: FilesPage(context: context) - case .ai: AIPage(context: context) case .keyboardLanguage: KeyboardLanguagePage(context: context) case .devices: DevicesPage(context: context) case .setup: SetupPage(context: context) diff --git a/Sources/ThreeFingerSwitcher/KeyboardLanguage/BrowserRegistry.swift b/Sources/ThreeFingerSwitcher/KeyboardLanguage/BrowserRegistry.swift index 63ddfd9..6ff7ce7 100644 --- a/Sources/ThreeFingerSwitcher/KeyboardLanguage/BrowserRegistry.swift +++ b/Sources/ThreeFingerSwitcher/KeyboardLanguage/BrowserRegistry.swift @@ -16,7 +16,7 @@ enum BrowserFamily { /// behavior unchanged (Firefox has no scriptable URL and is deliberately omitted — best-effort/unsupported). /// /// This is intentionally a small static table with no AppKit/Carbon/AX dependency, so it compiles and -/// unit-tests under `swift build` / `swift test` in the MLX-free Core. `ContextResolver` consults it to +/// unit-tests under `swift build` / `swift test` in Core. `ContextResolver` consults it to /// decide whether to resolve a host at all; the host readers consult `family(for:)` to pick the right /// AX layout / Apple Events vocabulary. enum BrowserRegistry { diff --git a/Sources/ThreeFingerSwitcher/KeyboardLanguage/KeyboardLanguagePolicy.swift b/Sources/ThreeFingerSwitcher/KeyboardLanguage/KeyboardLanguagePolicy.swift index 3b63b0a..0adb18f 100644 --- a/Sources/ThreeFingerSwitcher/KeyboardLanguage/KeyboardLanguagePolicy.swift +++ b/Sources/ThreeFingerSwitcher/KeyboardLanguage/KeyboardLanguagePolicy.swift @@ -2,7 +2,7 @@ import Foundation /// The two pure decision rules for the per-app keyboard-language feature (design D4). These are the /// "brains": no AppKit, no Carbon, no I/O — just dictionary math — so they compile and unit-test -/// under `swift build` / `swift test` in the MLX-free Core. All side effects (reading/selecting the +/// under `swift build` / `swift test` in Core. All side effects (reading/selecting the /// input source) live behind `InputSourceController`; all persistence lives in `KeyboardLanguageStore`. enum KeyboardLanguagePolicy { /// What to select when `bundleID` becomes frontmost: its remembered source if we have one, else diff --git a/Sources/ThreeFingerSwitcher/Launcher/ClaudeLaunch.swift b/Sources/ThreeFingerSwitcher/Launcher/ClaudeLaunch.swift index 3bd0b46..946979e 100644 --- a/Sources/ThreeFingerSwitcher/Launcher/ClaudeLaunch.swift +++ b/Sources/ThreeFingerSwitcher/Launcher/ClaudeLaunch.swift @@ -11,7 +11,7 @@ import Foundation /// user's shell PATH), which the app's headless `.script` runner (`/bin/zsh -c`) can't provide and /// which Claude (an interactive TUI) requires. -/// A small Core error taxonomy for the Open-Claude-Here flow, parallel to `FileActionError`: +/// A small Core error taxonomy for the Open-Claude-Here flow: /// `LocalizedError` with a clean, per-case, user-facing headline for every case (never a reflected /// enum dump or raw OS text). Vendor/OS errors are stringified into the opt-in `copyableDetails` /// payload at the boundary (a `String?`, kept `Equatable`) — surfaced only as a "Show details / Copy" diff --git a/Sources/ThreeFingerSwitcher/Launcher/FavoritesStore.swift b/Sources/ThreeFingerSwitcher/Launcher/FavoritesStore.swift index 964120c..531b9c9 100644 --- a/Sources/ThreeFingerSwitcher/Launcher/FavoritesStore.swift +++ b/Sources/ThreeFingerSwitcher/Launcher/FavoritesStore.swift @@ -19,28 +19,19 @@ final class FavoritesStore: ObservableObject { private convenience init() { self.init(defaults: .standard) } /// Test/seam initializer: inject an isolated `UserDefaults`. Loads the stored record (migrating - /// older schema versions forward, and folding any legacy AI commands into a normal "AI" band on the - /// first upgrade) or seeds the starter bands (including the "AI" band) on first run. + /// older schema versions forward) or seeds the starter bands on first run. init(defaults: UserDefaults) { self.defaults = defaults if let data = defaults.data(forKey: key), let decoded = try? JSONDecoder().decode(Favorites.self, from: data) { let storedVersion = decoded.schemaVersion - var record = Self.migrate(decoded) // stamps forward only when upgrading; identity otherwise - let didFold = Self.foldInLegacyAICommands(into: &record, storedVersion: storedVersion, defaults: defaults) - self.favorites = record - // Persist once if the load did one-time work (an upgrade and/or the AI fold-in). The legacy - // "aiCommands" key is retired ONLY after the new record is durably written — a failed save - // leaves it intact so the fold-in retries next launch (never lose data; spec/design D4). - // A downgrade (storedVersion > current) is NOT saved, so a future record is never clobbered. - if didFold || storedVersion < Favorites.currentSchemaVersion { - if save(), storedVersion < Favorites.aiCommandsFoldedSchemaVersion { - defaults.removeObject(forKey: Self.legacyAICommandsKey) - } - } + self.favorites = Self.migrate(decoded) // stamps forward only when upgrading; identity otherwise + // Persist once if the load did one-time upgrade work. A downgrade (storedVersion > current) + // is NOT saved, so a future record is never clobbered. + if storedVersion < Favorites.currentSchemaVersion { save() } } else { self.favorites = Self.seeded() - // Persist the seed so its ids (notably the seeded AI commands') are stable across relaunch. + // Persist the seed so its ids are stable across relaunch. save() } } @@ -108,8 +99,8 @@ final class FavoritesStore: ObservableObject { updateBand(bandID) { $0.items.removeAll { $0.id == itemID } } } - /// Move an item to a different band (any kind, including `.aiCommand`), appending it to the - /// destination. No-op when the item/bands can't be resolved or source == destination. + /// Move an item to a different band, appending it to the destination. No-op when the item/bands + /// can't be resolved or source == destination. func moveItem(_ itemID: UUID, fromBand: UUID, toBand: UUID) { guard fromBand != toBand else { return } mutate { fav in @@ -138,11 +129,22 @@ final class FavoritesStore: ObservableObject { // MARK: - Load / migrate - /// Forward-migrate an older record to the current schema **content** (identity today) and stamp the - /// current version. The AI fold-in is handled separately (`foldInLegacyAICommands`) because it needs - /// `UserDefaults` access to read+retire the legacy `aiCommands` key. + /// The retired "AI" band's sentinel id (the former `AIBand.bandID`). Kept only so the v3 + /// migration can recognize and remove the seeded band; users' own bands are never touched. + private static let legacyAIBandID = UUID(uuidString: "A1C0AAAA-0000-4000-8000-000000000001")! + + /// Forward-migrate an older record to the current schema content and stamp the current version. + /// v3 (the local-AI + Files-band removal): retired item kinds were already dropped per-item by + /// the lossy band decode (`ContextBand.FailableItem`); here the seeded "AI" band is removed once + /// it holds nothing else (a renamed/repurposed band with surviving items is the user's — kept). static func migrate(_ record: Favorites) -> Favorites { var record = record + if record.schemaVersion < 3 { + record.bands.removeAll { $0.id == legacyAIBandID && $0.items.isEmpty } + if let home = record.homeBandID, !record.bands.contains(where: { $0.id == home }) { + record.homeBandID = record.bands.first?.id + } + } // Only stamp FORWARD when upgrading; never down-stamp a future record. (init won't persist a // non-upgrade, so a newer-schema record written by a future build isn't clobbered on launch.) if record.schemaVersion < Favorites.currentSchemaVersion { @@ -151,49 +153,14 @@ final class FavoritesStore: ObservableObject { return record } - // MARK: - AI fold-in migration (one-time, idempotent) - - /// The legacy key the former `AICommandStore` persisted its commands under. - private static let legacyAICommandsKey = "aiCommands" - - /// One-time AI fold-in (configuration-hub): when upgrading from a record predating the fold-in - /// (`storedVersion < aiCommandsFoldedSchemaVersion`), append an "AI" band to `record`. It does NOT - /// touch the legacy key — the caller retires `aiCommands` only after a successful save, so a failed - /// write never loses commands. Idempotent: never appends a second "AI" band. Cases: - /// • legacy `aiCommands` present with commands → import them, preserving id + order; - /// • legacy key present but empty → opted in then cleared: import nothing (respect the empty choice); - /// • legacy key ABSENT → never opted in → seed the default "AI" band for discoverability (design D4). - /// Returns whether it changed `record`. - @discardableResult - static func foldInLegacyAICommands(into record: inout Favorites, storedVersion: Int, - defaults: UserDefaults) -> Bool { - guard storedVersion < Favorites.aiCommandsFoldedSchemaVersion else { return false } - guard !record.bands.contains(where: { AIBand.isAIBand($0) }) else { return false } - if let data = defaults.data(forKey: legacyAICommandsKey) { - // Opted in before: import their commands (an empty record imports nothing). - guard let commands = decodeLegacyAICommands(data), !commands.isEmpty else { return false } - record.bands.append(AIBand.band(from: commands)) - return true - } - // Never opted in (no legacy key): seed the default "AI" band so the feature is discoverable. - record.bands.append(AIBand.seededBand()) - return true - } - - /// Decode the legacy `AICommandStore` on-disk record (`{ schemaVersion, commands }`). - private static func decodeLegacyAICommands(_ data: Data) -> [AICommand]? { - struct LegacyStored: Codable { var schemaVersion: Int; var commands: [AICommand] } - return (try? JSONDecoder().decode(LegacyStored.self, from: data))?.commands - } - // MARK: - Seed /// Starter bands shown on first run — the SAME composition the First Touch wizard's tour /// teaches with, so what the user learns in onboarding is exactly what the launcher holds: - /// **Apps** (flame — the stock apps in one row), **Windows** (display — the twelve + /// **Apps** (flame — the stock apps in one row) and **Windows** (display — the twelve /// window-management actions, two exact grid rows; built by `WizardTourBands.windowsBand()` - /// so tour and seed cannot drift), and the **AI** band. The user re-arranges everything from - /// the editor; empty bands are also valid. Home cell points at the Apps band, column 0. + /// so tour and seed cannot drift). The user re-arranges everything from the editor; empty + /// bands are also valid. Home cell points at the Apps band, column 0. static func seeded() -> Favorites { func app(_ name: String, _ path: String) -> LaunchItem? { guard FileManager.default.fileExists(atPath: path) else { return nil } @@ -211,9 +178,6 @@ final class FavoritesStore: ObservableObject { app("Finder", "/System/Library/CoreServices/Finder.app"), app("System Settings", "/System/Applications/System Settings.app")].compactMap { $0 }) let windows = WizardTourBands.windowsBand() - // Fresh installs also get the "AI" band (a normal, editable band of seeded AI commands). Its - // items only act once AI is enabled; firing one while AI is off opens the enable/download canvas. - return Favorites(bands: [apps, windows, AIBand.seededBand()], - homeBandID: apps.id, homeColumn: 0) + return Favorites(bands: [apps, windows], homeBandID: apps.id, homeColumn: 0) } } diff --git a/Sources/ThreeFingerSwitcher/Launcher/LaunchItem.swift b/Sources/ThreeFingerSwitcher/Launcher/LaunchItem.swift index 1c7abcf..00cf36f 100644 --- a/Sources/ThreeFingerSwitcher/Launcher/LaunchItem.swift +++ b/Sources/ThreeFingerSwitcher/Launcher/LaunchItem.swift @@ -62,9 +62,6 @@ enum SystemAction: String, Codable, Equatable, CaseIterable, Identifiable { case missionControl, appExpose, showDesktop, nextSpace, previousSpace case lockScreen, screenSaver, sleepDisplay, emptyTrash case screenshotSelection, screenshotFullScreen, screenshotTools - /// Read the front window's last assistant reply aloud (`add-speak-last-response-launcher-action` - /// — the trackpad-native trigger for the menu-bar item; both route to the same coordinator verb). - case speakLastResponse // Media & display case playPause, nextTrack, previousTrack, volumeUp, volumeDown, mute, brightnessUp, brightnessDown @@ -142,7 +139,6 @@ enum SystemAction: String, Codable, Equatable, CaseIterable, Identifiable { case .screenshotSelection: return ("Screenshot — Selection", "camera.viewfinder", "Capture a selected area (system shortcut).", .system) case .screenshotFullScreen:return ("Screenshot — Full Screen", "camera.fill", "Capture the whole screen (system shortcut).", .system) case .screenshotTools: return ("Screenshot — Tools", "camera.on.rectangle", "Open the screenshot toolbar (system shortcut).", .system) - case .speakLastResponse: return ("Speak Last Response", "speaker.wave.2.bubble", "Read the front window's last assistant reply aloud (great over a Claude terminal).", .system) case .playPause: return ("Play / Pause", "playpause.fill", "Toggle media playback.", .media) case .nextTrack: return ("Next Track", "forward.fill", "Skip to the next track.", .media) case .previousTrack: return ("Previous Track", "backward.fill", "Go to the previous track.", .media) @@ -246,22 +242,8 @@ enum LaunchItemKind: Codable, Equatable { /// built at launcher-open from `ClipboardStore`, never created in the editor and never written /// into the persisted `Favorites` record. Firing it pastes the entry into the captured front app. case clipboardEntry(ClipboardEntry) - /// A filesystem entry shown in the synthetic Files band. **Synthetic and ephemeral** (like - /// `.clipboardEntry`, not like the persisted `.aiCommand`): built at launcher-open by - /// `FilesBandBuilder` from a live directory listing, never created in the editor and never written - /// into the persisted `Favorites` record. The Files band resolves a chosen entry through its own - /// drill-down / open path (folders descend, files open in their default handler) rather than the - /// generic `LaunchService.fire`, so firing one here is a no-op. - case fileEntry(FileEntry) - /// A configured AI command. **Persisted, first-class band item** (configuration-hub fold-in): - /// authored on the Hub's Bands page like any other item and stored inside the `Favorites` record, - /// so it can live in ANY band and move between bands. Its display fields (`title`/`icon`/`tint`) - /// mirror the embedded command. Firing it hands off to the `AICommandExecutor` and opens the - /// streaming preview canvas — it does NOT dismiss the overlay or complete on the lift. (Unlike - /// `.clipboardEntry`, which remains synthetic/ephemeral — clipboard entries are captured data.) - case aiCommand(AICommand) - /// A folder-bound "Open Claude Here" item. **Persisted, first-class band item** (like `.aiCommand`, - /// NOT synthetic like `.fileEntry`): authored on the Hub's Bands page and stored in the `Favorites` + /// A folder-bound "Open Claude Here" item. **Persisted, first-class band item** (NOT synthetic + /// like `.clipboardEntry`): authored on the Hub's Bands page and stored in the `Favorites` /// record, so it can live in any band and move between bands. Firing it opens the user's default /// terminal at `folder` and runs `command` — fire-and-forget, no mid-gesture picker. /// `command` is the shell command(s) to run after `cd`-ing into the folder; **nil/empty means the @@ -325,11 +307,8 @@ struct LaunchItem: Codable, Equatable, Identifiable { var isConsequential: Bool { switch kind { case .script, .preset, .claudeProject, .terminalCommand, .claudeProjectPrompt, .terminalCommandPrompt: return true - // `.aiCommand` reports its own success/failure through the executor's canvas state, not the - // launcher's fire notification, so it is not "consequential" in this sense. `.fileEntry` - // never flows through `fire` (the Files band resolves it via its own open path). `.automation` - // toggles a mode whose feedback IS the dimming/undimming, not a fire notification. - case .app, .path, .url, .shortcut, .action, .clipboardEntry, .aiCommand, .fileEntry, .automation: return false + // `.automation` toggles a mode whose feedback IS the dimming/undimming, not a fire notification. + case .app, .path, .url, .shortcut, .action, .clipboardEntry, .automation: return false } } } @@ -356,6 +335,29 @@ struct ContextBand: Codable, Equatable, Identifiable { /// The band's launcher icon, resolving the nil/legacy case to a neutral default symbol. var resolvedIcon: ItemIcon { icon ?? .sfSymbol("square.grid.2x2.fill") } + + // MARK: Lossy item decoding + + /// Wraps one item so a single undecodable element never fails the whole band. This is the + /// v3 migration path for retired kinds (`.aiCommand`, `.fileEntry`, `.action(.speakLastResponse)`): + /// a stored record that still contains one decodes with that item dropped, instead of the whole + /// `Favorites` record failing to decode and being reseeded (which would wipe the user's bands). + private struct FailableItem: Decodable { + let item: LaunchItem? + init(from decoder: Decoder) throws { item = try? LaunchItem(from: decoder) } + } + + private enum CodingKeys: String, CodingKey { case id, name, color, icon, defaultAppStrategy, items } + + init(from decoder: Decoder) throws { + let c = try decoder.container(keyedBy: CodingKeys.self) + id = try c.decode(UUID.self, forKey: .id) + name = try c.decode(String.self, forKey: .name) + color = try c.decode(ItemColor.self, forKey: .color) + icon = try c.decodeIfPresent(ItemIcon.self, forKey: .icon) + defaultAppStrategy = try c.decode(AppStrategy.self, forKey: .defaultAppStrategy) + items = try c.decode([FailableItem].self, forKey: .items).compactMap(\.item) + } } /// The root favorites record — the single persisted value (see `FavoritesStore`). @@ -375,14 +377,11 @@ struct Favorites: Codable, Equatable { self.homeColumn = homeColumn } - /// v2 folds AI commands into the band model (configuration-hub): `.aiCommand` items are now - /// persisted inside bands. A record at v1 (or older) triggers the one-time AI fold-in migration in - /// `FavoritesStore`, which imports any legacy `AICommandStore` record into a normal "AI" band. - static let currentSchemaVersion = 2 - - /// The schema version at which AI commands became persisted band items. A stored record older than - /// this is folded in once (legacy `aiCommands` → an "AI" band). - static let aiCommandsFoldedSchemaVersion = 2 + /// v3 removes the retired kinds (`.aiCommand`, `.fileEntry`, `.action(.speakLastResponse)` — the + /// local-AI and Files-band removal): stored items of those kinds are dropped by the lossy band + /// decode (`ContextBand.FailableItem`), and `FavoritesStore.migrate` removes the seeded "AI" band + /// once it is empty. v2 had folded AI commands into the band model. + static let currentSchemaVersion = 3 // MARK: Resolved, deterministic accessors (never recency-ordered) diff --git a/Sources/ThreeFingerSwitcher/Launcher/LaunchService.swift b/Sources/ThreeFingerSwitcher/Launcher/LaunchService.swift index 088ecc6..70919f7 100644 --- a/Sources/ThreeFingerSwitcher/Launcher/LaunchService.swift +++ b/Sources/ThreeFingerSwitcher/Launcher/LaunchService.swift @@ -28,27 +28,18 @@ final class LaunchService { /// window of the destination Space once the switch settles — macOS leaves it visually front but /// not key, exactly like the native shortcut. No-op by default / in tests. private let onSpaceSwitch: () -> Void - /// Called when an `.aiCommand` item is fired. Wired by the coordinator to hand the command off to - /// the `AICommandExecutor` (which streams into the overlay's preview canvas). Injected so - /// `LaunchService` stays decoupled from the AI layer; no-op by default / in tests. Firing an AI - /// command does NOT dismiss the overlay (the order-out-before-fire rule does not apply) — the - /// overlay handles that exception itself (see `LauncherOverlayController.end`). - private let onAICommand: (AICommand) -> Void /// Called after the user picks a folder for a choose-folder-at-launch item (`.claudeProjectPrompt` / /// `.terminalCommandPrompt`) — `(itemID, bandID, chosenFolder)`. Wired by the coordinator to persist - /// the folder back onto the item (its remembered last folder). Injected like `onAICommand` so - /// `LaunchService` stays decoupled/testable; no-op by default / in tests. + /// the folder back onto the item (its remembered last folder). Injected so `LaunchService` stays + /// decoupled/testable; no-op by default / in tests. private let onPromptedFolderChosen: (UUID, UUID, URL) -> Void /// Called when an `.automation` item is fired — `(kind, settings)`. Wired by the coordinator to /// TOGGLE the automation's stateful owner (`KeepAwakeController`) — start if inactive, stop if /// active — passing the item's resolved `AutomationSettings` (dim level, keyboard dim, guard lock). - /// Injected like `onAICommand` so `LaunchService` stays decoupled from the automation's state; - /// no-op by default / in tests. Unlike every other kind, firing an automation is a toggle, not a - /// one-shot completion. + /// Injected like `onPromptedFolderChosen` so `LaunchService` stays decoupled from the automation's + /// state; no-op by default / in tests. Unlike every other kind, firing an automation is a toggle, + /// not a one-shot completion. private let onAutomation: (AutomationKind, AutomationSettings) -> Void - /// Speak-last-response dispatch (`add-speak-last-response-launcher-action`): injected like - /// `onAICommand` so the launcher never references the AI stack; no-op by default / in tests. - private let onSpeakLastResponse: () -> Void /// Resolves a clipboard entry id to its **fully-materialized** entry at fire time. The band carries /// only bounded/light preview entries (image bytes dropped, large text truncated — see /// `ClipboardStore.bandWindow`), so paste must fetch the complete representations by id to restore the @@ -61,20 +52,16 @@ final class LaunchService { goToWindow: @escaping (pid_t) -> Bool = { _ in false }, frontAppProvider: @escaping () -> NSRunningApplication? = { NSWorkspace.shared.frontmostApplication }, onSpaceSwitch: @escaping () -> Void = {}, - onAICommand: @escaping (AICommand) -> Void = { _ in }, onPromptedFolderChosen: @escaping (UUID, UUID, URL) -> Void = { _, _, _ in }, onAutomation: @escaping (AutomationKind, AutomationSettings) -> Void = { _, _ in }, - onSpeakLastResponse: @escaping () -> Void = {}, clipboardResolver: @escaping (UUID) -> ClipboardEntry? = { _ in nil }) { self.favoritesProvider = favoritesProvider self.mover = mover ?? NullWindowMover() self.goToWindow = goToWindow self.frontAppProvider = frontAppProvider self.onSpaceSwitch = onSpaceSwitch - self.onAICommand = onAICommand self.onPromptedFolderChosen = onPromptedFolderChosen self.onAutomation = onAutomation - self.onSpeakLastResponse = onSpeakLastResponse self.clipboardResolver = clipboardResolver } @@ -98,25 +85,15 @@ final class LaunchService { case .preset: firePreset(item, inBand: band) case .automation(let kind, let dimPercent, let dimKeyboard, let lockOnStop): - // Toggle the automation's stateful owner (start if inactive, stop if active). Like - // `.aiCommand`, this is NOT a one-shot that completes on the lift — it enters/leaves a - // persistent mode owned outside the launcher (see `onAutomation`). The item's authored - // optionals resolve here (nil = default) into one `AutomationSettings` for the owner. + // Toggle the automation's stateful owner (start if inactive, stop if active). This is + // NOT a one-shot that completes on the lift — it enters/leaves a persistent mode owned + // outside the launcher (see `onAutomation`). The item's authored optionals resolve here + // (nil = default) into one `AutomationSettings` for the owner. onAutomation(kind, AutomationSettings(dimPercent: dimPercent, dimKeyboard: dimKeyboard ?? false, lockOnStop: lockOnStop ?? false)) case .clipboardEntry(let entry): pasteEntry(entry) - case .fileEntry: - // No-op: the synthetic Files band resolves a chosen entry through its OWN drill-down / - // open path (folders descend in place; files open via `FileOpenService`), not the generic - // fire — so a `.fileEntry` item never lands here in practice, and firing one is harmless. - break - case .aiCommand(let command): - // Hand off to the executor (which streams into the overlay's preview canvas). Unlike every - // other kind, this does NOT complete or dismiss on the lift — a fresh four-finger DOWN swipe - // (commit) / horizontal swipe (discard) resolve it (see `LauncherOverlayController`). - onAICommand(command) case .claudeProject(let folder, let command, let claudePath): launchClaude(folder: folder, command: command, claudePath: claudePath, title: item.title) case .terminalCommand(let folder, let command): @@ -388,7 +365,6 @@ final class LaunchService { case .mute: postMediaKey(7) case .brightnessUp: adjustBrightness(up: true, adjustment) case .brightnessDown: adjustBrightness(up: false, adjustment) - case .speakLastResponse: onSpeakLastResponse() } } diff --git a/Sources/ThreeFingerSwitcher/Onboarding/FirstTouchWizardModel.swift b/Sources/ThreeFingerSwitcher/Onboarding/FirstTouchWizardModel.swift index f88e7cd..0964385 100644 --- a/Sources/ThreeFingerSwitcher/Onboarding/FirstTouchWizardModel.swift +++ b/Sources/ThreeFingerSwitcher/Onboarding/FirstTouchWizardModel.swift @@ -500,10 +500,9 @@ final class FirstTouchWizardModel: ObservableObject { /// Seed (or re-seed) the tour from the context's band composition. Toggle parameters carry the /// EMITTED values from the optional-feature switches — `@Published` fires on willSet, so a /// property re-read in those observers would still see the old value. - private func seedLauncherDemo(clipboardOn: Bool? = nil, aiOn: Bool? = nil) { + private func seedLauncherDemo(clipboardOn: Bool? = nil) { let clipboard = clipboardOn ?? context.settings.keepClipboardHistory - let ai = aiOn ?? context.settings.aiCommandsEnabled - let bands = context.launcherBands(clipboard, ai) + let bands = context.launcherBands(clipboard) guard !bands.isEmpty else { return } launcherDemo.dwell = context.settings.dwellToArmDuration launcherDemo.setBands(bands.map(\.items), @@ -516,7 +515,7 @@ final class FirstTouchWizardModel: ObservableObject { } /// Re-seed the tour live when the optional-feature toggles change on the playground act, so - /// flipping Clipboard / AI immediately shows (or removes) the band it controls. + /// flipping Clipboard immediately shows (or removes) the band it controls. private func observeTourToggles() { context.settings.$keepClipboardHistory .dropFirst() @@ -528,15 +527,6 @@ final class FirstTouchWizardModel: ObservableObject { } } .store(in: &cancellables) - context.settings.$aiCommandsEnabled - .dropFirst() - .sink { [weak self] on in - MainActor.assumeIsolated { - guard let self, self.stage == .playground else { return } - withAnimation(.easeInOut(duration: 0.3)) { self.seedLauncherDemo(aiOn: on) } - } - } - .store(in: &cancellables) } // MARK: - Act IV: the launcher lane toggle diff --git a/Sources/ThreeFingerSwitcher/Onboarding/WizardActs.swift b/Sources/ThreeFingerSwitcher/Onboarding/WizardActs.swift index 1181a1d..9fee112 100644 --- a/Sources/ThreeFingerSwitcher/Onboarding/WizardActs.swift +++ b/Sources/ThreeFingerSwitcher/Onboarding/WizardActs.swift @@ -451,7 +451,7 @@ struct PlaygroundAct: View { // the moment four fingers land it MORPHS to (near-)actual size, floating // over the act while the hand plays the real thing, and settles back into // its slot on the lift. The slot never changes size, so nothing reflows. - LauncherView(model: launcherDemo, executor: nil, availability: nil) + LauncherView(model: launcherDemo) .frame(width: tourSize.width, height: tourSize.height) // Completing the contract washes light across the tour — the scene's // applause for the hand that just learned it. @@ -477,14 +477,10 @@ struct PlaygroundAct: View { symbol: "doc.on.clipboard", title: "Clipboard history", cost: "Records what you copy. Stays on this Mac. No permission, instant.", isOn: $settings.keepClipboardHistory, index: 1) - OptionalFeatureCard( - symbol: "sparkles", title: "AI commands", - cost: "On-device Gemma model — a one-time multi-gigabyte download. Apple Silicon only.", - isOn: $settings.aiCommandsEnabled, index: 2) OptionalFeatureCard( symbol: "globe", title: "Keyboard language", cost: "Remembers your input source per app. No permission, no re-login.", - isOn: $settings.keyboardLanguageEnabled, index: 3) + isOn: $settings.keyboardLanguageEnabled, index: 2) } Text("Tip: add favorites from any app via the menu-bar icon — “Add Front App to Band”.") .font(.caption) diff --git a/Sources/ThreeFingerSwitcher/Onboarding/WizardContext.swift b/Sources/ThreeFingerSwitcher/Onboarding/WizardContext.swift index e832edf..19d6f3a 100644 --- a/Sources/ThreeFingerSwitcher/Onboarding/WizardContext.swift +++ b/Sources/ThreeFingerSwitcher/Onboarding/WizardContext.swift @@ -60,7 +60,7 @@ final class WizardContext { // toggles: the favorites bands, the seeded AI band when AI is on but no AI command survives in // the favorites, and the Clipboard band when history is on (with example entries while the // store is still empty, so the user sees what to expect). - var launcherBands: (_ clipboardOn: Bool, _ aiOn: Bool) -> [ContextBand] = { _, _ in [] } + var launcherBands: (_ clipboardOn: Bool) -> [ContextBand] = { _ in [] } /// Whether the four-finger lanes are already effective — the recognizer then drives the tour; /// until they are, the wizard's raw touch feed does. var launcherLive: () -> Bool = { false } diff --git a/Sources/ThreeFingerSwitcher/Onboarding/WizardMotion.swift b/Sources/ThreeFingerSwitcher/Onboarding/WizardMotion.swift index 98c1fa4..ced2cda 100644 --- a/Sources/ThreeFingerSwitcher/Onboarding/WizardMotion.swift +++ b/Sources/ThreeFingerSwitcher/Onboarding/WizardMotion.swift @@ -111,7 +111,7 @@ struct RippleRing: View { // MARK: - Breathing halo -/// A soft breathing halo — the AI canvas's sparkle-pulse idiom generalized: `TimelineView`-driven +/// A soft breathing halo: `TimelineView`-driven /// (no bound state), calm enough to wait on. Used behind the brand mark, behind waiting states /// (the re-login door), and under the demo strip while the user's hand drives it. struct PulseHalo: View { diff --git a/Sources/ThreeFingerSwitcher/Onboarding/WizardTourBands.swift b/Sources/ThreeFingerSwitcher/Onboarding/WizardTourBands.swift index 7c7431b..1cbb750 100644 --- a/Sources/ThreeFingerSwitcher/Onboarding/WizardTourBands.swift +++ b/Sources/ThreeFingerSwitcher/Onboarding/WizardTourBands.swift @@ -6,8 +6,6 @@ import Foundation /// - **flame** — every app currently scattered across the user's bands, gathered into one row. /// - **display** — the twelve window-management actions (halves, quarters, maximize, center, /// minimize, full screen): two exact rows of six. -/// - **sparkles** — only when AI is on: the user's AI commands (or the seeded set when they have -/// none yet). /// - **clipboard** — only when history is on: the real clipboard band (sample entries while empty). /// /// Nothing more. Pure value composition — unit-tested; the coordinator supplies the inputs. @@ -22,13 +20,8 @@ enum WizardTourBands { ] static func compose(userBands: [ContextBand], - aiOn: Bool, - seededAIBand: () -> ContextBand, clipboardBand: ContextBand?) -> [ContextBand] { var result: [ContextBand] = [appsBand(from: userBands), windowsBand()] - if aiOn { - result.append(aiBand(from: userBands, seeded: seededAIBand)) - } if let clipboardBand { result.append(clipboardBand) } @@ -65,16 +58,4 @@ enum WizardTourBands { }) } - /// sparkles: the user's own AI commands gathered from their bands; the seeded band when they - /// have none yet (fresh install, or AI just switched on in the wizard). - private static func aiBand(from userBands: [ContextBand], seeded: () -> ContextBand) -> ContextBand { - let owned = userBands.flatMap(\.items).filter { - if case .aiCommand = $0.kind { return true } else { return false } - } - guard !owned.isEmpty else { return seeded() } - return ContextBand(name: "AI", - color: ItemColor(red: 0.66, green: 0.36, blue: 0.86), - icon: .sfSymbol("sparkles"), - items: owned) - } } diff --git a/Sources/ThreeFingerSwitcher/Overlay/AICommandCanvasView.swift b/Sources/ThreeFingerSwitcher/Overlay/AICommandCanvasView.swift deleted file mode 100644 index 877cc41..0000000 --- a/Sources/ThreeFingerSwitcher/Overlay/AICommandCanvasView.swift +++ /dev/null @@ -1,591 +0,0 @@ -import SwiftUI -import AppKit - -/// The AI command **streaming preview canvas** (spec launcher-overlay: "AI command streaming preview -/// canvas" + "Swipe-to-resolve (commit / discard)" + "Armed-confirmation state"). It replaces the launcher grid while an -/// AI command is in flight, binding to the injected `AICommandExecutor`'s observable `state` so the -/// model's result fills in **incrementally** as it streams — the same live-render pattern the -/// `ClipboardBandView` value preview uses, here driven by `@Published state` rather than `.task(id:)`. -/// -/// The overlay panel stays non-activating throughout (the captured front app remains key); this view -/// is pure presentation. The gesture wiring (a fresh four-finger DOWN swipe commits, a horizontal -/// swipe discards; a stray re-lift is a no-op) lives in `LauncherOverlayController`; this view only -/// reflects the executor's state and shows the matching commit/discard affordance hint per state. -struct AICommandCanvasView: View { - @ObservedObject var executor: AICommandExecutor - /// The command being run (its name titles the canvas). - let command: AICommand - /// The command's tint (falls back to the band color) for the header + accents. - let tint: Color - /// Enable/download wiring for the `.unavailable` state (configuration-hub). Optional so the canvas - /// still renders without it (a defensive fallback message); wired by the launcher overlay. - var availability: AICanvasAvailability? = nil - - /// Whether the collapsible Thinking section is expanded. COLLAPSED by default: the user sees only a - /// pulsing "Thinking…" label + a live elapsed timer until they tap to expand the full reasoning. - @State private var thinkingExpanded = false - /// When the model's reasoning FIRST became non-empty — anchors the live elapsed timer. Set once per - /// fire (cleared when thinking goes empty on a re-run/discard), so the timer counts from first token. - @State private var thinkingStart: Date? - /// When reasoning FINISHED — the moment the response began (or generation otherwise completed). Once - /// set, the elapsed readout FREEZES at this value instead of ticking forever (the timer must stop once - /// the model is done thinking + answering). Cleared with `thinkingStart` on a re-run/discard. - @State private var thinkingEnd: Date? - - var body: some View { - VStack(alignment: .leading, spacing: 0) { - // The language picker rides at the top-middle of the canvas as a centered Liquid-Glass pill, - // above the header, only for translate-style commands (`showsLanguagePicker`). - if showsLanguagePicker { - HStack { Spacer(); languagePill; Spacer() } - .padding(.bottom, 6) - } - header - Divider().opacity(0.35).padding(.vertical, 8) - // The model's reasoning ("show the model's thinking"): a collapsible, scrollable section - // under the header and above the response content. Rendered only while there's thinking to - // show; the RESPONSE below is always the committed text (never the thinking). - if !executor.thinking.isEmpty { - thinkingSection - .padding(.bottom, 8) - } - content - .frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .topLeading) - Divider().opacity(0.3).padding(.top, 8) - footerHint - } - .padding(.top, 6) - // Anchor the elapsed timer to the moment thinking first appears; clear it (and the frozen end) - // when thinking resets (a fresh fire / discard empties `executor.thinking`), so the next run times - // from scratch. Also collapse the Thinking section on reset so a previously-expanded run doesn't - // leak its expanded state into the next fire (keeps the collapsed-by-default behavior). - .onChange(of: executor.thinking.isEmpty) { _, isEmpty in - thinkingStart = isEmpty ? nil : (thinkingStart ?? Date()) - if isEmpty { thinkingExpanded = false; thinkingEnd = nil } - } - // Freeze the elapsed timer the moment reasoning finishes (the response begins, or generation - // otherwise completes) — set once, so it stops ticking instead of counting forever. - .onChange(of: reasoningFinished) { _, finished in - if finished, thinkingStart != nil, thinkingEnd == nil { thinkingEnd = Date() } - } - // Update the resolve gate: a fresh DOWN swipe applies only when every scrollable region is at its - // top (so scrolling the response/thinking never inserts). The reduce ANDs all reporters. - .onPreferenceChange(CanvasAtTopKey.self) { atTop in - executor.canvasAtTop = atTop - } - .onAppear { - if !executor.thinking.isEmpty, thinkingStart == nil { thinkingStart = Date() } - if reasoningFinished, thinkingStart != nil, thinkingEnd == nil { thinkingEnd = Date() } - } - } - - /// Whether the model has finished REASONING: false while loading or while only thinking is streaming - /// (the response is still empty); true once the response begins streaming or any terminal state is - /// reached. Drives the one-shot freeze of the elapsed timer. - private var reasoningFinished: Bool { - switch executor.state { - case .idle, .loadingModel: return false - case let .streaming(partial): return !partial.isEmpty // response started → thinking is done - default: return true // ready / review / declined / failed / … - } - } - - // MARK: Thinking (collapsible reasoning) - - /// The collapsible "show the model's thinking" section. COLLAPSED (default) = a pulsing "✦ Thinking…" - /// label + a live elapsed timer (so the user sees it's alive, not stuck) and a `chevron.right`. - /// EXPANDED = a bounded, scrollable `BidiText` of the full streamed reasoning (capped at 160pt so it - /// never sprawls) + a `chevron.down`. Tapping the row toggles — the canvas is interactive for its - /// whole life (see `setCanvasInteractive`), so the tap lands. - private var thinkingSection: some View { - VStack(alignment: .leading, spacing: 8) { - Button { - thinkingExpanded.toggle() - } label: { - thinkingHeaderRow - } - .buttonStyle(.plain) - - if thinkingExpanded { - thinkingExpandedBody - } - } - .padding(10) - .background(RoundedRectangle(cornerRadius: 10).fill(Color.primary.opacity(0.04))) - .overlay(RoundedRectangle(cornerRadius: 10).strokeBorder(.white.opacity(0.08))) - } - - /// The always-visible header row: a pulsing sparkle + "Thinking…", a live elapsed readout, and a - /// chevron reflecting the expanded/collapsed state. - private var thinkingHeaderRow: some View { - HStack(spacing: 8) { - // A static sparkle marks the reasoning header. The earlier TimelineView "breathing" pulse - // was removed: it kept animating after thinking finished AND while the canvas was hidden, - // driving a perpetual SwiftUI relayout that pinned the main thread (see - // docs/postmortem-idle-cpu-spin.md). A better liveness cue may return later. - Image(systemName: "sparkles") - .font(.system(size: 13, weight: .semibold)) - .foregroundStyle(tint) - Text(reasoningFinished ? "Thought" : "Thinking…") - .font(.system(size: 13, weight: .medium)) - .foregroundStyle(.secondary) - elapsedLabel - Spacer() - Image(systemName: thinkingExpanded ? "chevron.down" : "chevron.right") - .font(.system(size: 11, weight: .semibold)) - .foregroundStyle(.secondary) - } - .contentShape(Rectangle()) // the whole row is tappable, not just the glyphs - } - - /// A live elapsed readout. While reasoning is in flight it recomputes ~10×/s off a `TimelineView` so - /// the user sees it ticking; once `thinkingEnd` is set it FREEZES at the final duration (stops - /// counting). Shows "3.2s" under a minute, "mm:ss" beyond, anchored at `thinkingStart`. - @ViewBuilder - private var elapsedLabel: some View { - if let start = thinkingStart { - if let end = thinkingEnd { - // Frozen: reasoning finished — show the final duration, no longer ticking. - Text(Self.elapsedString(end.timeIntervalSince(start))) - .font(.system(size: 11, weight: .regular).monospacedDigit()) - .foregroundStyle(.secondary) - } else { - TimelineView(.periodic(from: .now, by: 0.1)) { ctx in - Text(Self.elapsedString(ctx.date.timeIntervalSince(start))) - .font(.system(size: 11, weight: .regular).monospacedDigit()) - .foregroundStyle(.secondary) - } - } - } - } - - /// Format an elapsed interval: "3.2s" under a minute, "m:ss" at/over a minute. Pure (testable). - static func elapsedString(_ seconds: TimeInterval) -> String { - let s = max(0, seconds) - if s < 60 { return String(format: "%.1fs", s) } - let total = Int(s) - return String(format: "%d:%02d", total / 60, total % 60) - } - - /// The expanded reasoning: a bounded, scrollable `BidiText` of the full streamed thinking, auto- - /// scrolling to the tail as it grows (a ScrollViewReader, best-effort). Capped at 160pt so streaming - /// reasoning never pushes the response off the canvas. - private var thinkingExpandedBody: some View { - ScrollViewReader { proxy in - ScrollView { - VStack(spacing: 0) { - Color.clear.frame(height: 1).id(Self.thinkingHeadID) // top anchor - BidiText(text: executor.thinking, fontSize: 12, color: .secondaryLabelColor) - .frame(maxWidth: .infinity, alignment: .topLeading) - Color.clear.frame(height: 1).id(Self.thinkingTailID) // bottom anchor - } - // The Thought panel does NOT report into `CanvasAtTopKey`: it auto-scrolls to its tail as - // reasoning streams (below), so a long reasoning leaves it not-at-top — which, under the - // AND-reduce, would wrongly drag `canvasAtTop` false and BLOCK the down-to-commit gesture on - // the result. Commit is gated only by the RESULT / THREAD scrolls (which have their own - // reporters); the secondary, self-scrolling reasoning must never block applying the result. - } - .frame(maxHeight: 160) - .coordinateSpace(name: Self.thinkingScrollSpace) - .onChange(of: executor.thinking) { - // Follow the tail only WHILE reasoning is still streaming, so the user watches it grow. - guard !reasoningFinished else { return } - withAnimation(.easeOut(duration: 0.15)) { - proxy.scrollTo(Self.thinkingTailID, anchor: .bottom) - } - } - .onChange(of: reasoningFinished) { _, finished in - // When reasoning ends, rest at the TOP (read from the start, and so the box reports at-top - // and never blocks the commit gate from a stale tail position). - if finished { - withAnimation(.easeOut(duration: 0.15)) { proxy.scrollTo(Self.thinkingHeadID, anchor: .top) } - } - } - } - } - - /// Stable ids for the top / bottom scroll anchors of the expanded reasoning. - private static let thinkingHeadID = "ai-thinking-head" - private static let thinkingTailID = "ai-thinking-tail" - - // MARK: Header - - private var header: some View { - HStack(spacing: 10) { - Image(systemName: "sparkles") - .font(.system(size: 18, weight: .semibold)) - .foregroundStyle(tint) - Text(command.name) - .font(.system(size: 18, weight: .semibold)) - Spacer() - statusBadge - } - } - - @ViewBuilder - private var statusBadge: some View { - switch executor.state { - case .loadingModel: - badge("Loading…", systemImage: "hourglass", color: .secondary) - case .streaming: - badge("Generating…", systemImage: "dot.radiowaves.left.and.right", color: tint) - case .ready: - badge("Ready", systemImage: "checkmark.circle.fill", color: .green) - case .reviewingAction: - badge("Review", systemImage: "exclamationmark.shield.fill", color: .orange) - case .noInput: - badge("No input", systemImage: "text.cursor", color: .secondary) - case .declined: - badge("Declined", systemImage: "hand.raised.fill", color: .secondary) - case .failed: - badge("Failed", systemImage: "exclamationmark.triangle.fill", color: .red) - case .unavailable: - badge("Unavailable", systemImage: "exclamationmark.circle.fill", color: .orange) - case .committed: - badge("Done", systemImage: "checkmark.seal.fill", color: .green) - case .idle: - EmptyView() - } - } - - // MARK: Runtime language picker - - /// Whether the in-canvas language dropdown is shown: only when the active command declares a - /// language runtime parameter (`executor.activeLanguage != nil`) AND the state still has a result - /// worth re-translating. It rides across the in-flight/result states (loadingModel, streaming, - /// ready, reviewingAction, …) but is hidden where re-running makes no sense — `.idle` (nothing in - /// flight), `.unavailable` (the enable/download canvas owns the surface), and `.committed` (done). - private var showsLanguagePicker: Bool { - guard executor.activeLanguage != nil else { return false } - switch executor.state { - case .idle, .unavailable, .committed: return false - default: return true - } - } - - /// A centered Liquid-Glass pill at the top-middle of the canvas reading as the "Detect language → - /// dropdown" flow: a leading globe glyph + a subtle "Auto-detect" label, an arrow "→", then the - /// language `Menu`. Picking one calls `executor.setLanguage`, which cancels + re-runs the command in - /// place and persists the choice (the view only wires the control). The options come from the ACTIVE - /// command's declared parameter — human languages for Translate, programming languages for "Rewrite - /// in Language" — falling back to `AILanguages.all`. The current `executor.activeLanguage` is always - /// included (prepended if missing, mirroring `AILanguages.including`) so a persisted/declared default - /// off the canonical list stays selectable. The glass treatment matches `HubGlass` / `LauncherView` - /// (macOS 26+ `glassEffect`, `.ultraThinMaterial` fallback). - private var languagePill: some View { - let opts = command.runtimeParameter?.options ?? AILanguages.all - let active = executor.activeLanguage ?? opts.first ?? "English" - let options = opts.contains(active) ? opts : [active] + opts - return HStack(spacing: 8) { - Image(systemName: "globe") - .font(.system(size: 12, weight: .semibold)) - .foregroundStyle(tint) - Text("Auto-detect") - .font(.system(size: 12)) - .foregroundStyle(.secondary) - Image(systemName: "arrow.right") - .font(.system(size: 11, weight: .semibold)) - .foregroundStyle(.secondary) - Menu { - ForEach(options, id: \.self) { language in - Button(language) { executor.setLanguage(language) } - } - } label: { - HStack(spacing: 4) { - Text(executor.activeLanguage ?? "English") - .font(.system(size: 12, weight: .medium)) - Image(systemName: "chevron.down") - .font(.system(size: 9, weight: .semibold)) - } - .foregroundStyle(tint) - } - .menuStyle(.borderlessButton) - .menuIndicator(.hidden) - .fixedSize() - } - .padding(.horizontal, 12).padding(.vertical, 6) - .background { - if #available(macOS 26.0, *) { Color.clear.glassEffect(.regular, in: Capsule()) } - else { Capsule().fill(.ultraThinMaterial) } - } - .overlay(Capsule().strokeBorder(.white.opacity(0.12))) - } - - // MARK: Content (the live preview surface) - - @ViewBuilder - private var content: some View { - switch executor.state { - case .idle, .loadingModel: - centered { - ProgressView().controlSize(.large) - Text("Loading the model…").font(.system(size: 14)).foregroundStyle(.secondary) - } - case .noInput: - centered { - Image(systemName: "text.cursor").font(.system(size: 40)).foregroundStyle(.secondary) - Text("No input").font(.system(size: 16, weight: .medium)) - Text("Select some text (or copy something) and try again.") - .font(.system(size: 12)).foregroundStyle(.secondary).multilineTextAlignment(.center) - } - case let .streaming(partial): - resultScroll(text: partial.isEmpty ? "…" : partial) - case let .ready(result): - resultScroll(text: result) - case let .reviewingAction(review): - reviewFields(review) // `review` is the TaskReview carried by the state - case let .declined(reason): - centered { - Image(systemName: "hand.raised.fill").font(.system(size: 36)).foregroundStyle(.secondary) - Text("The model declined this command").font(.system(size: 15, weight: .medium)) - // A decline reason can be a full Hebrew/Arabic sentence (multi-paragraph), so route it - // through BidiText for true PER-PARAGRAPH base direction, not the single-direction helper. - ScrollView { - BidiText(text: reason, fontSize: 12, color: .secondaryLabelColor) - .frame(maxWidth: .infinity, alignment: .topLeading) - } - .frame(maxHeight: 120) // bounded so a long reason can't overflow the panel - } - case let .failed(message): - centered { - Image(systemName: "exclamationmark.triangle.fill").font(.system(size: 36)).foregroundStyle(.red) - Text("Something went wrong").font(.system(size: 15, weight: .medium)) - // A failure message can be a full Hebrew/Arabic sentence too — route it through BidiText - // for true per-paragraph base direction (mixed LTR+RTL resolves cleanly). - ScrollView { - BidiText(text: message, fontSize: 12, color: .secondaryLabelColor) - .frame(maxWidth: .infinity, alignment: .topLeading) - } - .frame(maxHeight: 120) // bounded for symmetry with the Settings row cap - } - case .unavailable: - if let availability { - AIUnavailableCanvas(settings: availability.settings, - models: availability.models, - tint: tint, - onDownload: availability.onDownload) - } else { - centered { - Image(systemName: "sparkles").font(.system(size: 40)).foregroundStyle(.secondary) - Text("AI is unavailable").font(.system(size: 16, weight: .medium)) - Text("Enable AI commands and download the model in the Hub.") - .font(.system(size: 12)).foregroundStyle(.secondary).multilineTextAlignment(.center) - } - } - case .committed: - centered { - Image(systemName: "checkmark.seal.fill").font(.system(size: 40)).foregroundStyle(.green) - Text("Done").font(.system(size: 16, weight: .medium)) - } - } - } - - /// The streamed/ready text, scrollable, in the canvas's large value pane. Rendered through - /// `BidiText` (a natural-base-direction NSTextView) so Hebrew/Arabic output starts on the right and - /// mixed LTR+RTL resolves cleanly, recomputed per paragraph as tokens stream (design D6). - private func resultScroll(text: String) -> some View { - ScrollView { - BidiText(text: text, fontSize: 14) - .frame(maxWidth: .infinity, alignment: .topLeading) - .background(atTopReporter(space: Self.resultScrollSpace)) - } - .coordinateSpace(name: Self.resultScrollSpace) - } - - /// A 0-impact background probe that reports whether `space`'s scroll content is at its TOP, via - /// `CanvasAtTopKey`. Placed as the content's background so its `minY` in the scroll's coordinate space - /// is the content top's offset: 0 at the top, negative once scrolled down (small epsilon for rounding). - private func atTopReporter(space: String) -> some View { - GeometryReader { geo in - Color.clear.preference(key: CanvasAtTopKey.self, - value: geo.frame(in: .named(space)).minY >= -2) - } - } - - private static let resultScrollSpace = "ai-canvas-result-scroll" - private static let thinkingScrollSpace = "ai-canvas-thinking-scroll" - - /// The armed-confirmation review: the parsed action's concrete fields, before the commit fires the - /// side effect (spec: "displays the parsed action's concrete fields before it can be committed"). - @ViewBuilder - private func reviewFields(_ taskReview: TaskReview) -> some View { - if case let .action(title, fields, _) = taskReview { - VStack(alignment: .leading, spacing: 12) { - Text(title).font(.system(size: 15, weight: .semibold)) - ForEach(Array(fields.enumerated()), id: \.offset) { _, field in - VStack(alignment: .leading, spacing: 2) { - Text(field.label).font(.system(size: 11, weight: .semibold)) - .foregroundStyle(.secondary).textCase(.uppercase) - // Review-field values (Title/Start/Email/…) are SHORT and single-paragraph by - // construction, so the lightweight first-strong SwiftUI helper suffices here — - // no BidiText / NSTextView needed for these single-line values. - Text(field.value).font(.system(size: 14)) - .frame(maxWidth: .infinity, alignment: .leading) - .naturalTextDirection(for: field.value) // RTL value starts on the right - } - } - Text("Swipe down to confirm the action, or swipe aside to cancel.") - .font(.system(size: 12)).foregroundStyle(.secondary).padding(.top, 4) - } - } else { - Color.clear - } - } - - // MARK: Footer hint - - /// A short hint that mirrors the available gestures for the current state, so a signed-build user - /// knows that a fresh four-finger DOWN swipe commits/applies and a HORIZONTAL swipe discards. The - /// gestures themselves are in the controller; a stray re-lift is a no-op, so it is never advertised. - @ViewBuilder - private var footerHint: some View { - switch executor.state { - case .ready: - hint("Swipe down to apply", "Swipe aside to discard") - case .reviewingAction: - hint("Swipe down to confirm", "Swipe aside to cancel") - case .streaming: - hint(nil, "Swipe aside to discard") - case .noInput, .declined, .failed, .unavailable, .committed: - hint(nil, "Swipe aside to dismiss") - case .idle, .loadingModel: - hint(nil, "Swipe aside to cancel") - } - } - - @ViewBuilder - private func hint(_ commit: String?, _ discard: String?) -> some View { - HStack(spacing: 16) { - if let commit { - Label(commit, systemImage: "arrow.down.circle.fill") // down swipe = bring it into the document - .font(.system(size: 12)).foregroundStyle(tint) - } - Spacer() - if let discard { - Label(discard, systemImage: "arrow.left.and.right.circle.fill") // sideways swipe = discard - .font(.system(size: 12)).foregroundStyle(.secondary) - } - } - } - - // MARK: Helpers - - private func badge(_ text: String, systemImage: String, color: Color) -> some View { - Label(text, systemImage: systemImage) - .font(.system(size: 12, weight: .medium)) - .foregroundStyle(color) - .padding(.horizontal, 10).padding(.vertical, 4) - .background(Capsule().fill(Color.primary.opacity(0.06))) - } - - private func centered(@ViewBuilder _ content: () -> C) -> some View { - VStack(spacing: 10) { content() } - .frame(maxWidth: .infinity, maxHeight: .infinity) - } -} - -/// Reports whether the canvas's scrollable content is at its TOP. Each scrollable region contributes a -/// boolean; `reduce` ANDs them, so the combined value is true only when EVERY region is at its top — the -/// condition under which a fresh down-swipe applies the result (otherwise the down-swipe is a scroll). -private struct CanvasAtTopKey: PreferenceKey { - static let defaultValue = true - static func reduce(value: inout Bool, nextValue: () -> Bool) { - value = value && nextValue() - } -} - -/// Enable/download wiring the canvas needs for its `.unavailable` state (configuration-hub). Holds the -/// observable `AppSettings` + `ModelManager` (the unavailable subview observes them) and the download -/// action. A value type so the overlay can set it on every `show` without retaining cycles. -struct AICanvasAvailability { - let settings: AppSettings - let models: ModelManager - /// Begin (or retry) the on-device model download (honors the selected model). Continues in the - /// background after the canvas is dismissed. - let onDownload: () -> Void -} - -/// The canvas body for `.unavailable`: a clean message plus Enable / Download actions and a model -/// picker, reflecting live opt-in + model state. Dismissable like any canvas (a horizontal swipe); -/// any download it starts keeps running in the background. -private struct AIUnavailableCanvas: View { - @ObservedObject var settings: AppSettings - @ObservedObject var models: ModelManager - let tint: Color - let onDownload: () -> Void - - var body: some View { - ScrollView { - VStack(alignment: .leading, spacing: 14) { - header - if !settings.aiCommandsEnabled { - Button { settings.aiCommandsEnabled = true } label: { - Label("Turn on AI commands", systemImage: "power") - } - .buttonStyle(.borderedProminent) - } - modelPicker - statusRow - Text("Swipe aside to dismiss — any download keeps running in the background.") - .font(.system(size: 11)).foregroundStyle(.secondary) - } - .frame(maxWidth: .infinity, alignment: .leading) - .padding(.vertical, 4) - } - } - - private var header: some View { - VStack(alignment: .leading, spacing: 4) { - Text(settings.aiCommandsEnabled ? "Model not downloaded yet" : "AI commands are turned off") - .font(.system(size: 16, weight: .semibold)) - Text(settings.aiCommandsEnabled - ? "Download the on-device model to run this command. It runs entirely on your Mac." - : "Turn AI on and download the on-device model to run this command.") - .font(.system(size: 12)).foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - } - } - - @ViewBuilder - private var modelPicker: some View { - let registry = ModelCatalog.standard - Picker("Model", selection: Binding( - get: { settings.aiSelectedModelID }, - set: { settings.aiSelectedModelID = $0 })) { - Text("Default (\(registry.defaultDescriptor?.displayName ?? "registry"))").tag(String?.none) - ForEach(registry.models, id: \.id) { model in - Text(model.displayName).tag(String?.some(model.id)) - } - } - .pickerStyle(.menu) - .disabled(!settings.aiCommandsEnabled) - } - - @ViewBuilder - private var statusRow: some View { - switch models.state { - case .notDownloaded: - Button { onDownload() } label: { Label("Download model", systemImage: "arrow.down.circle") } - .buttonStyle(.bordered) - .disabled(!settings.aiCommandsEnabled) - case let .downloading(progress): - HStack(spacing: 8) { - ProgressView(value: progress).frame(width: 140) - Text("Downloading… \(Int(progress * 100))%").font(.system(size: 12)).foregroundStyle(.secondary) - } - case .verifying: - Label("Verifying…", systemImage: "checkmark.shield").font(.system(size: 12)).foregroundStyle(.secondary) - case .ready, .loading, .loaded: - Label("Model ready — fire the command again.", systemImage: "checkmark.circle.fill") - .font(.system(size: 12)).foregroundStyle(.green) - case let .failed(reason, _): - VStack(alignment: .leading, spacing: 6) { - Text(reason).font(.system(size: 12)).foregroundStyle(.red) - .lineLimit(3).truncationMode(.middle) - Button { onDownload() } label: { Label("Retry download", systemImage: "arrow.clockwise") } - .buttonStyle(.bordered) - .disabled(!settings.aiCommandsEnabled) - } - } - } -} diff --git a/Sources/ThreeFingerSwitcher/Overlay/BidiText.swift b/Sources/ThreeFingerSwitcher/Overlay/BidiText.swift deleted file mode 100644 index 16417cd..0000000 --- a/Sources/ThreeFingerSwitcher/Overlay/BidiText.swift +++ /dev/null @@ -1,197 +0,0 @@ -import SwiftUI -import AppKit - -/// A read-only, transparent text view that renders **bidirectional (RTL/LTR) text correctly** for the -/// AI command preview canvas (spec launcher-overlay: "Bidirectional text rendering" — Hebrew/Arabic -/// starts on the correct side; mixed LTR+RTL resolves cleanly; recomputed as tokens stream). -/// -/// Design D6: SwiftUI's `Text` doesn't expose a NATURAL base writing direction, so the robust path for -/// the streamed/result body is a tiny `NSTextView` wrapper. The base direction is a **paragraph-style** -/// attribute, not a view-level one: an `NSMutableParagraphStyle` with `baseWritingDirection = .natural` -/// makes TextKit apply the Unicode Bidi Algorithm's first-strong rule (P2/P3) **per paragraph**, and -/// `alignment = .natural` then follows the resolved direction (an RTL paragraph aligns right, an LTR -/// paragraph left) — so a Hebrew line aligns right, an English line aligns left, and a mixed paragraph -/// picks its side from its first strong character, with no per-character work here. -/// -/// IMPORTANT: the VIEW-LEVEL `textView.baseWritingDirection`/`alignment` must NOT be set to `.natural`. -/// View-level `.natural` resolves to the USER-INTERFACE layout direction (the app/system language — LTR -/// on an English-localized Mac), NOT per-paragraph first-strong, and a view-level `alignment` then -/// force-aligns every paragraph to that uniform side — which is the exact bug this file fixes (Hebrew -/// rendered left-aligned). We apply the paragraph style to `defaultParagraphStyle`, `typingAttributes`, -/// and the full text-storage range instead. We set the string in `updateNSView`, which re-runs on every -/// `@Published` stream update and re-asserts the paragraph style over the full range, so the base -/// direction is recomputed live as the model streams. Final RTL rendering is confirmed on a signed build. -/// -/// It is NOT editable and NOT selectable (matching the canvas's prior `.textSelection(.disabled)`), -/// draws no background (transparent over the overlay), and grows with its content inside the existing -/// `ScrollView` — it wraps to the available width and reports its fitted height. -struct BidiText: NSViewRepresentable { - /// The text to render. Set into the view on every update so base direction recomputes as it streams. - let text: String - /// The point size for the system font (14 to match the canvas body / `Text(.system(size: 14))`). - var fontSize: CGFloat = 14 - /// The text color. `labelColor` matches a default SwiftUI `Text`; pass `.secondaryLabelColor` for - /// the dimmer decline/failed message bodies. - var color: NSColor = .labelColor - - func makeNSView(context: Context) -> NSTextView { - // A plain NSTextView does NOT report a usable `intrinsicContentSize` height, so SwiftUI sizes it to - // the viewport and the enclosing ScrollView never overflows (it rubber-bands "as if at the - // bottom"). `FittingTextView` reports its laid-out text height, so a long body grows past the - // viewport and the ScrollView actually scrolls. - let textView = FittingTextView() - textView.isEditable = false - textView.isSelectable = false - textView.drawsBackground = false // transparent over the overlay - textView.backgroundColor = .clear - textView.textContainerInset = .zero - // NATURAL base direction is set as a PARAGRAPH STYLE in `apply(to:)` (NOT view-level): TextKit then - // resolves each paragraph's side by the Unicode first-strong rule. A view-level `.natural` here would - // instead resolve to the UI-language direction and override per-paragraph alignment — that was the bug. - // Wrap to the container's width and let height grow with content (vertical scroll lives outside): - // the container tracks the view's width but is UNBOUNDED in height, so a multi-line body can never - // be clipped to one line by a finite container as text wraps inside the SwiftUI ScrollView. - textView.isHorizontallyResizable = false - textView.isVerticallyResizable = true - if let container = textView.textContainer { - container.widthTracksTextView = true - container.lineFragmentPadding = 0 - container.size = NSSize(width: container.size.width, height: .greatestFiniteMagnitude) - } - // Don't fight the ScrollView for width (hug low horizontally) but report the full fitted height - // (hug high vertically) so the body grows to its wrapped height rather than collapsing to one line. - textView.setContentHuggingPriority(.defaultLow, for: .horizontal) - textView.setContentCompressionResistancePriority(.defaultLow, for: .horizontal) - textView.setContentHuggingPriority(.defaultHigh, for: .vertical) - textView.setContentCompressionResistancePriority(.defaultHigh, for: .vertical) - apply(to: textView) - return textView - } - - func updateNSView(_ textView: NSTextView, context: Context) { - // Re-set the string on every stream update; this is what recomputes the per-paragraph base - // direction (first-strong) as new tokens arrive. - if textView.string != text { textView.string = text } - apply(to: textView) - // Keep the height unbounded after a string change, then force layout + re-report the intrinsic - // size so the SwiftUI ScrollView sizes to the freshly-wrapped height (and never a stale 1-line one). - // NOTE: final visual sizing must be confirmed in a real signed build (manual test). - if let container = textView.textContainer, let layout = textView.layoutManager { - container.size = NSSize(width: container.size.width, height: .greatestFiniteMagnitude) - layout.ensureLayout(for: container) - } - textView.invalidateIntrinsicContentSize() - } - - /// Apply font/color and an EXPLICIT per-paragraph base direction + alignment after a string change - /// (setting `.string` can reset attributes, so we restore them each update). - /// - /// We do NOT use `.natural`: on `NSTextView` a `.natural` writing direction resolves to the - /// USER-INTERFACE / system locale direction (LTR on an English-localized Mac) — NOT the Unicode - /// first-strong rule — so Hebrew rendered left-aligned. Instead we choose each paragraph's base from - /// its **first strong directional character** (`firstStrongDirection`) — the first word/char of the - /// line decides its side — and PIN `baseWritingDirection` + `alignment` explicitly: a paragraph whose - /// first strong char is Hebrew/Arabic (or any RTL script) → `.rightToLeft` + `.right`, otherwise - /// `.leftToRight` + `.left`. Once that first strong char is present the side is STABLE (later - /// characters never change it); mixed runs within the paragraph still resolve via the system Bidi - /// algorithm. Re-run on every stream update only so a paragraph that begins with neutrals adopts its - /// side once the first strong char streams in — never to re-decide from later content. - private func apply(to textView: NSTextView) { - let font = NSFont.systemFont(ofSize: fontSize) - textView.font = font - textView.textColor = color - // Default (LTR) for the empty/typing case; real paragraphs are pinned per-paragraph below. - let ltrDefault = NSMutableParagraphStyle() - ltrDefault.baseWritingDirection = .leftToRight - ltrDefault.alignment = .left - textView.defaultParagraphStyle = ltrDefault - textView.typingAttributes = [.font: font, .foregroundColor: color, .paragraphStyle: ltrDefault] - - guard let storage = textView.textStorage, storage.length > 0 else { return } - let full = NSRange(location: 0, length: storage.length) - storage.addAttribute(.font, value: font, range: full) - storage.addAttribute(.foregroundColor, value: color, range: full) - // Explicit first-strong direction per paragraph. - let ns = storage.string as NSString - ns.enumerateSubstrings(in: full, options: .byParagraphs) { substring, _, enclosingRange, _ in - let rtl = firstStrongDirection(substring ?? "") == .rightToLeft - let para = NSMutableParagraphStyle() - para.baseWritingDirection = rtl ? .rightToLeft : .leftToRight - para.alignment = rtl ? .right : .left - storage.addAttribute(.paragraphStyle, value: para, range: enclosingRange) - } - } -} - -/// A non-scrolling `NSTextView` that reports its **laid-out text height** as the intrinsic content -/// height, so SwiftUI sizes it to the wrapped content (not the viewport) and the enclosing `ScrollView` -/// overflows + scrolls for long bodies. Height is computed at the current container width (which tracks -/// the view width), so it reflects the actual wrapping; width stays `noIntrinsicMetric` (the parent -/// decides width). `invalidateIntrinsicContentSize()` on each text update re-queries it as tokens stream. -private final class FittingTextView: NSTextView { - override var intrinsicContentSize: NSSize { - guard let layoutManager, let textContainer else { return super.intrinsicContentSize } - layoutManager.ensureLayout(for: textContainer) - let height = layoutManager.usedRect(for: textContainer).height - return NSSize(width: NSView.noIntrinsicMetric, height: ceil(height)) - } - - /// This text view is NOT inside an NSScrollView — the enclosing SwiftUI `ScrollView` owns scrolling. - /// A bare NSTextView otherwise SWALLOWS scroll-wheel events (it has no clip view to move), so the - /// SwiftUI ScrollView never sees them and the body appears unscrollable. Forward them up so the host - /// ScrollView scrolls. - override func scrollWheel(with event: NSEvent) { - nextResponder?.scrollWheel(with: event) - } -} - -/// The base writing direction of a paragraph by the Unicode **first-strong** rule, as a SwiftUI -/// `LayoutDirection`: the first strong RTL character (Hebrew/Arabic/… ) ⇒ `.rightToLeft`, the first strong -/// Latin/Greek/Cyrillic/… character ⇒ `.leftToRight`, and an empty/neutral-only string ⇒ `.leftToRight` -/// (the canvas default). The FIRST word/char of a line decides its side; leading neutrals (digits, -/// punctuation, whitespace, a URL) are skipped until the first strong char, and once that char is present -/// the side is STABLE — later characters never change it (that is the intended behavior: a line's -/// alignment is fixed by how it starts, not re-decided as more content streams in). -func firstStrongDirection(_ text: String) -> LayoutDirection { - for scalar in text.unicodeScalars { - if isStrongRTL(scalar.value) { return .rightToLeft } - // First strong character is NOT in an RTL block ⇒ treat it as left-to-right. We detect only the - // RTL ranges explicitly and default everything else to LTR, so a strong-LTR character outside the - // common Latin range (e.g. IPA Extensions 0x0250–0x02AF, Greek, Cyrillic) still resolves LTR - // rather than being mistaken for a neutral and scanned past. - if isStrongLTR(scalar.value) { return .leftToRight } - // Neutral (digits, punctuation, whitespace, symbols) ⇒ keep scanning for the first strong char. - } - return .leftToRight -} - -/// Whether a scalar is a strong RIGHT-TO-LEFT character. Covers **all** RTL scripts, not just Hebrew and -/// Arabic: the contiguous RTL block `U+0590–U+08FF` (Hebrew, Arabic, Syriac, Thaana, N'Ko, Samaritan, -/// Mandaic, Arabic Extended-A/B) plus the Hebrew/Arabic presentation-form blocks. -func isStrongRTL(_ value: UInt32) -> Bool { - (0x0590...0x08FF).contains(value) // Hebrew, Arabic, Syriac, Thaana, NKo, Samaritan, Mandaic, Arabic Ext - || (0xFB1D...0xFB4F).contains(value) // Hebrew presentation forms - || (0xFB50...0xFDFF).contains(value) // Arabic presentation forms-A - || (0xFE70...0xFEFF).contains(value) // Arabic presentation forms-B -} - -/// Whether a scalar is a strong LEFT-TO-RIGHT character: any letter that is not in an RTL block. We -/// approximate "letter" as "not a neutral" — digits, punctuation, whitespace, and symbols are neutral -/// and skipped — so a strong-LTR letter outside the Latin range still counts as LTR. -func isStrongLTR(_ value: UInt32) -> Bool { - guard let scalar = Unicode.Scalar(value), !isStrongRTL(value) else { return false } - let p = scalar.properties - return p.isAlphabetic || p.generalCategory == .modifierLetter || p.generalCategory == .otherLetter -} - -extension View { - /// Align a short SwiftUI `Text` by its content's first-strong base direction: a value that STARTS with - /// Hebrew/Arabic reads right-aligned, one that starts with Latin left-aligned. Sets both the multiline - /// alignment and the layout direction so the leading edge matches the text's opening script. - func naturalTextDirection(for text: String) -> some View { - let direction = firstStrongDirection(text) - return self - .multilineTextAlignment(direction == .rightToLeft ? .trailing : .leading) - .environment(\.layoutDirection, direction) - } -} diff --git a/Sources/ThreeFingerSwitcher/Overlay/BubbleMorph.swift b/Sources/ThreeFingerSwitcher/Overlay/BubbleMorph.swift deleted file mode 100644 index f1e698d..0000000 --- a/Sources/ThreeFingerSwitcher/Overlay/BubbleMorph.swift +++ /dev/null @@ -1,60 +0,0 @@ -import SwiftUI - -/// The app's **first spring** — a near-zero droplet that buds up into presence. A `ViewModifier` -/// (applied via `View.bubbleMorph(anchor:)`) that animates a view *in* by interpolating -/// `scaleEffect(0.02 → 1, anchor:)` + `opacity(0 → 1)` on a single `spring(response: 0.34, -/// dampingFraction: 0.72)`, flipped on `.onAppear`. It is the Files band's depth/preview/row entrance -/// motion and is deliberately confined to **containers / rows / preview / menus, never leaf glyphs**. -/// -/// It intentionally does **not** touch the existing motion vocabulary: the selection highlight keeps -/// its linear charge ramp (`.linear(dwell)`) and its `.easeOut` arm snap, and there are **no new -/// haptics** — the `.alignment` arm tick stays the only one (design D8). For SwiftUI membership -/// insert/remove (e.g. a depth `.id`-swap, where there's no stable view to drive `.onAppear` across the -/// change), use the matching ``bubbleTransition(anchor:)`` so the same droplet shape governs both the -/// budding-in *and* the receding-out side. -struct BubbleMorph: ViewModifier { - /// The point the droplet buds from — `.center` by default; columns may bud from their attachment - /// edge (`.leading` / `.trailing`) so the morph reads as growing *out of* the rail. - var anchor: UnitPoint = .center - - /// The bud spring, shared with ``bubbleTransition(anchor:)`` so a modifier-driven entrance and a - /// transition-driven one settle on the same clock. - static let spring: Animation = .spring(response: 0.34, dampingFraction: 0.72) - - /// The droplet's starting scale — small enough to read as a bud, non-zero so the anchor stays - /// well-defined (a literal `0` collapses the frame and the anchor with it). - static let seedScale: CGFloat = 0.02 - - @State private var shown = false - - func body(content: Content) -> some View { - content - .scaleEffect(shown ? 1 : Self.seedScale, anchor: anchor) - .opacity(shown ? 1 : 0) - .onAppear { - withAnimation(Self.spring) { shown = true } - } - } -} - -extension View { - /// Bud this view up into presence on appear (see ``BubbleMorph``). Use on **containers / rows / - /// preview / menus, never leaf glyphs**, and never on the single sliding selection highlight - /// (per-row morphs reintroduce the documented scrub strobe — design D8). - func bubbleMorph(anchor: UnitPoint = .center) -> some View { - modifier(BubbleMorph(anchor: anchor)) - } -} - -extension AnyTransition { - /// The membership-insert/remove counterpart to ``BubbleMorph``: the same `0.02 → 1` droplet scale - /// combined with opacity, for views that come and go inside a `ForEach` / `.id`-keyed swap (where - /// there's no persistent view to carry `.onAppear` across the change). Pair it with - /// `.animation(BubbleMorph.spring, value:)` (or wrap the mutation in `withAnimation(BubbleMorph - /// .spring)`) so insert and remove ride the bud spring — matching the `SwitcherView` `.id`/ - /// `.transition` idiom, but **scaling, not sliding**. - static func bubbleMorph(anchor: UnitPoint = .center) -> AnyTransition { - .scale(scale: BubbleMorph.seedScale, anchor: anchor) - .combined(with: .opacity) - } -} diff --git a/Sources/ThreeFingerSwitcher/Overlay/FilesBandView.swift b/Sources/ThreeFingerSwitcher/Overlay/FilesBandView.swift deleted file mode 100644 index 2789771..0000000 --- a/Sources/ThreeFingerSwitcher/Overlay/FilesBandView.swift +++ /dev/null @@ -1,977 +0,0 @@ -import SwiftUI -import AppKit -import Foundation - -/// The Files band's body: a **bounded column navigator** (design D6), rendered instead of the icon grid -/// when `model.currentBandIsFiles`. Three regions, left to right: -/// -/// 1. a thin **ancestor icon rail** — one collapsed folder icon per ancestor of the current column -/// (`model.filesColumn.navigation.ancestors`), deepest nearest the current list, so the path is -/// legible without ever widening the overlay; -/// 2. the **current-folder list** — the full vertical list of `model.filesColumn.visibleEntries`, each -/// row a kind glyph + name + the metadata `AppSettings.filesRowMetadata` selects; and -/// 3. a **live preview** pane — a QuickLook content preview for a highlighted file (reusing the Clipboard -/// band's `FilePreview`) or a peek of a highlighted folder's contents. -/// -/// At any depth exactly one current list + one preview are full-size; the rail stays a fixed width, so the -/// visible width is **bounded regardless of depth** (the whole point of D6). All sizes come from -/// `FilesBandLayout`, the single source `LauncherOverlayController` also reads to size the panel, so the -/// rendered surface and the `NSWindow` frame can never drift. -/// -/// **Motion (design D8).** Containers / rows / preview / the rail bud in with `BubbleMorph`; a **depth** -/// change scales the current list down into its new ancestor icon while the incoming list buds in (the -/// `SwitcherView` `.id`/`.transition` idiom, but *scaling, not sliding*). The selection highlight is a -/// **single sliding element** (cloned from `ClipboardBandView.RowHighlight`) tracking -/// `highlightedIndex` — never re-created per row (that would reintroduce the documented scrub strobe), and -/// never bubble-morphed. -/// -/// The overlay panel is **non-activating** (it never becomes key/main on its own); these animations are -/// display-only. The navigator is **purely gesture-driven** — there is no keyboard focus and the panel never -/// needs to become key (unlike the AI canvas), so every interaction is a trackpad intent routed in by the -/// recognizer's Files-drill sub-state. -struct FilesBandView: View { - @ObservedObject var model: LauncherModel - /// Live appearance/behaviour tunables (column width, density, tint, row metadata, icon style). The - /// overlay layer reads the shared singleton (matching `LauncherOverlayController`'s panel sizer and the - /// AI canvas's `@ObservedObject` settings), so a tweak in the Hub re-renders the open navigator. - @ObservedObject var settings: AppSettings = .shared - - /// Whether the failure row's raw "Show details" disclosure is expanded (collapsed by default — detail is - /// opt-in, never shown inline; mirrors `ModelDetailView.showingDetails`). The raw OS/workspace text lives - /// ONLY behind this disclosure, never in the headline. - @State private var showingFailureDetails = false - - private var controller: FilesColumnController? { model.filesColumn } - - var body: some View { - Group { - if let controller { - navigator(controller) - } else { - // Defensive: the Files band is only the current band when a controller is injected; an empty - // surface keeps the view total without ever crashing on a missing controller. - Color.clear - } - } - .background(glassFill) - // The Open-With picker buds in over the navigator when a relative +1-finger lift opens it, and - // recedes the same way on exit (choose / discard) — a bounded popup centered on the band. It is an - // overlay (not a replacement) so the column stays visible behind it, matching the user's "a popup - // list opened over what I was looking at" vision. - .overlay { openWithPicker } - // The +1-finger ACTION MENU buds in over the navigator the same way (files & folders). "Open in ▸" - // within it exits to the Open-With grid; the two popups are never both open. - .overlay { actionMenuPopup } - // A failed open surfaces as a BOUNDED, non-blocking row pinned to the bottom of the navigator — - // never an app-modal alert (spec: failures are observable, never silent; bounded + non-blocking). It - // buds in over the column (the navigator stays usable behind it) and recedes on Dismiss / a fresh open. - .overlay(alignment: .bottom) { failureRow } - } - - // MARK: - Layout: (rail | current list | preview) over a full-width breadcrumb bar - - /// The navigator is the three columns stacked OVER a full-width breadcrumb bar (refinement 4): the - /// `HStack` of rail | current list | preview, then a bottom strip spanning all three columns rendering - /// the path to the highlighted item. The whole thing is pinned to `FilesBandLayout`'s FIXED container - /// size (= the Clipboard band's exact dimensions, refinement 3) so crossing in / changing depth never - /// resizes or moves the panel; a folder taller than the fixed row area scrolls inside `currentList`. - @ViewBuilder - private func navigator(_ controller: FilesColumnController) -> some View { - VStack(spacing: 0) { - HStack(spacing: 0) { - ancestorRail(controller) - .frame(width: FilesBandLayout.ancestorRailWidth) - - Divider().opacity(0.25) - - currentList(controller) - .frame(width: FilesBandLayout.currentColumnWidth) - // A depth change scales the whole current list down toward its ancestor icon / buds the - // incoming list in — the `SwitcherView` `.id`/`.transition` idiom, scaling not sliding - // (design D8). The depth token is the current location, so descend/ascend swap the subtree; - // the driving `.animation(value:)` sits on the enclosing HStack (the `SwitcherView` shape). - .id(depthID(controller)) - .transition(.bubbleMorph(anchor: .leading)) - - Divider().opacity(0.25) - - preview(controller) - .frame(width: FilesBandLayout.previewWidth) - .frame(maxHeight: .infinity) - } - .frame(maxHeight: .infinity) - .animation(BubbleMorph.spring, value: depthID(controller)) - - Divider().opacity(0.25) - - breadcrumbBar(controller) - .frame(height: FilesBandLayout.breadcrumbBarHeight) - } - // FILL the FIXED panel (refinement 3) — exactly like `ClipboardBandView` fills its container — rather - // than imposing an outer frame here. `LauncherOverlayController` already sizes the panel to the - // CONSTANT `FilesBandLayout` dims (Clipboard-equal, no per-depth / per-density variation) and - // `LauncherView` insets it by the shared container padding; the three fixed-width columns - // (rail + current + preview + dividers) and the row-area/breadcrumb heights are computed from - // those same constants to sum to the available area, so the surface and the window frame can't drift. - // The container never resizes on crossing in or changing depth — the current list scrolls inside. - .frame(maxWidth: .infinity, maxHeight: .infinity) - } - - // MARK: - Ancestor icon rail (left) - - /// The collapsed path: one folder icon per ancestor, oldest at the top and the **deepest nearest the - /// current list** (the rail reads downward toward where you are). Leaf glyphs, so the *icons* are not - /// bubble-morphed; the rail container buds in once. Empty (but still reserved-width) at the roots list, - /// so the current column never jumps sideways on the first descend. - @ViewBuilder - private func ancestorRail(_ controller: FilesColumnController) -> some View { - let ancestors = controller.navigation.ancestors - VStack(spacing: FilesBandLayout.ancestorRowSpacing) { - ForEach(ancestors, id: \.self) { url in - ancestorIcon(url) - } - } - .frame(maxHeight: .infinity, alignment: .top) - .padding(.top, 6) - .bubbleMorph(anchor: .leading) - } - - @ViewBuilder - private func ancestorIcon(_ url: URL) -> some View { - Image(systemName: "folder.fill") - .font(.system(size: FilesBandLayout.ancestorIconSize * 0.62)) - .foregroundStyle(tint.opacity(0.85)) - .frame(width: FilesBandLayout.ancestorIconSize, height: FilesBandLayout.ancestorIconSize) - .help(url.lastPathComponent) - } - - // MARK: - Current folder list (centre) - - @ViewBuilder - private func currentList(_ controller: FilesColumnController) -> some View { - let entries = controller.visibleEntries - VStack(spacing: 0) { - // The current-folder rows SCROLL inside the fixed container (refinement 3): a `ScrollViewReader` - // + a vertical `ScrollView` (cloned from `ClipboardBandView.keyList`) so a folder taller than the - // fixed row area scrolls to keep the highlight visible, driven by the same vertical edge-auto- - // repeat that moves `highlightedIndex`. The single sliding highlight lives in the scrolled content - // (so it tracks its row through the scroll), drawn BEHIND the rows exactly like the Clipboard list's - // `.background(alignment: .top) { highlight }`, so the row text reads on top of the pill (design D8). - ScrollViewReader { proxy in - ScrollView(.vertical, showsIndicators: false) { - ZStack(alignment: .top) { - highlight(controller) - - VStack(spacing: 0) { - ForEach(entries) { entry in - row(entry, controller: controller) - .frame(height: rowHeight) - // Row CONTENT comes/goes with the droplet: rows that bud in when a late - // async listing lands (the depth-swap itself - // is governed by the whole list's `.id`/`.transition`, so this only fires - // for membership changes within a stable depth — design D8). Never on the - // sliding highlight. The row's `.id` is the scroll target the reader seeks. - .id(entry.id) - .transition(.bubbleMorph()) - } - } - .frame(maxWidth: .infinity, alignment: .top) - .animation(BubbleMorph.spring, value: entries.map(\.id)) - } - .frame(maxWidth: .infinity, alignment: .top) - } - // Follow the highlight as the vertical edge-auto-repeat steps it: scroll the highlighted row - // into view (centered, like the Clipboard key-list) whenever the index moves OR the listing - // changes underneath it (a late async landing can shift which row is highlighted). - .onChange(of: controller.highlightedIndex) { scroll(proxy, controller) } - .onChange(of: entries.map(\.id)) { scroll(proxy, controller) } - } - .frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .top) - } - } - - /// Scroll the highlighted row into view (centered), the Files-band clone of `ClipboardBandView.scroll`: - /// targets the entry at `highlightedIndex` within `visibleEntries` by its id, so the list follows the - /// sliding highlight when the folder is taller than the fixed container. A no-op on an empty column - /// (nothing highlighted to centre on). - private func scroll(_ proxy: ScrollViewProxy, _ controller: FilesColumnController) { - guard let id = controller.highlightedEntry?.id else { return } - withAnimation(.easeInOut(duration: 0.16)) { - proxy.scrollTo(id, anchor: .center) - } - } - - // MARK: - Breadcrumb bar (bottom, full-width across all three columns) - - /// The full-width breadcrumb bar pinned at the BOTTOM (refinement 4), spanning the rail + current list + - /// preview: `controller.breadcrumb` rendered root → chevron → … → the currently-HIGHLIGHTED item, compact - /// and middle-truncating when long, updating live as the highlight moves (each highlight step re-derives - /// the path in the model). Buds in with `BubbleMorph` like the band's other containers. A leading folder - /// glyph anchors it; an empty path (nothing highlighted) shows nothing but keeps the strip's height so the - /// container never reflows. - @ViewBuilder - private func breadcrumbBar(_ controller: FilesColumnController) -> some View { - let components = controller.breadcrumb - HStack(spacing: 4) { - Image(systemName: "folder") - .font(.system(size: 11)) - .foregroundStyle(tint.opacity(0.85)) - // Join the components with chevron separators on a SINGLE line, middle-truncating the whole crumb - // string when the path is long (rather than wrapping or growing the fixed-height strip). The - // highlighted leaf is the last component, so the tail (where you are) is what survives truncation. - Text(breadcrumbText(components)) - .font(.system(size: 11)) - .foregroundStyle(.secondary) - .lineLimit(1) - .truncationMode(.middle) - Spacer(minLength: 0) - } - .padding(.horizontal, 12) - .frame(maxWidth: .infinity, alignment: .leading) - .frame(height: FilesBandLayout.breadcrumbBarHeight) - .bubbleMorph() - } - - /// Render the breadcrumb components as `root › child › … › leaf` for the bottom bar. AppKit-free — the - /// model names each component by its last path component (the view "can prettify"); here that prettifying - /// is just joining them with a chevron so the single-line `Text` can middle-truncate the whole path. - private func breadcrumbText(_ components: [FilesBreadcrumbComponent]) -> String { - components.map(\.name).joined(separator: " › ") - } - - /// One current-list row: a kind glyph (or a QuickLook thumbnail when `filesIconStyle == .preview`), - /// the display name, and the metadata selected by `AppSettings.filesRowMetadata`. Row **content** is - /// allowed to bubble-morph (it's a container/row, not the sliding highlight), so a re-listed row buds - /// in. The selection backing is the single sliding `highlight`, never a per-row fill. - @ViewBuilder - private func row(_ entry: FileEntry, controller: FilesColumnController) -> some View { - let selected = entry.id == controller.highlightedEntry?.id - HStack(spacing: 8) { - rowGlyph(entry) - .frame(width: 20, height: 20) - Text(entry.name.isEmpty ? " " : entry.name) - .font(.system(size: 13)) - .lineLimit(1) - .truncationMode(.middle) - .foregroundStyle(selected ? .primary : .secondary) - Spacer(minLength: 4) - if let meta = metadataLabel(entry) { - Text(meta) - .font(.system(size: 11)) - .foregroundStyle(.secondary) - .lineLimit(1) - } - if entry.isDirectory { - Image(systemName: "chevron.right") - .font(.system(size: 9, weight: .semibold)) - .foregroundStyle(.tertiary) - } - } - .padding(.horizontal, 10) - .frame(height: rowHeight) - } - - /// The per-row leading glyph: a `FileKind` SF Symbol by default (`filesIconStyle == .icon`, cheap), or - /// a live QuickLook thumbnail when the user picked `.preview` (icon fallback while it loads / when none - /// exists). Reuses `FilePreview` for the thumbnail so there's one QuickLook surface. - @ViewBuilder - private func rowGlyph(_ entry: FileEntry) -> some View { - switch settings.filesIconStyle { - case .preview: - FilePreview(url: entry.url) - case .icon: - kindGlyph(entry.kind) - } - } - - @ViewBuilder - private func kindGlyph(_ kind: FileKind) -> some View { - let symbol: String = { - if case let .sfSymbol(name) = FilesBandBuilder.glyph(for: kind) { return name } - return "doc" - }() - Image(systemName: symbol) - .font(.system(size: 13)) - .foregroundStyle(kind == .folder ? tint : Color.secondary) - } - - // MARK: - The single sliding selection highlight - - /// One persistent Liquid Glass pill that **slides** to the highlighted row by offsetting on its index - /// (cloned from `ClipboardBandView.RowHighlight`). It is the Files band's analog of the grid's - /// `SelectionSquare` / the clipboard list's `RowHighlight`: never re-created per row (that strobes while - /// scrubbing — design D8), and never bubble-morphed. Hidden when the column is empty. - @ViewBuilder - private func highlight(_ controller: FilesColumnController) -> some View { - if controller.highlightedEntry != nil { - FilesRowHighlight(token: model.armingToken, armed: model.armed, dwell: model.dwell, color: tint) - .frame(height: rowHeight) - .frame(maxWidth: .infinity) - .padding(.horizontal, 4) - .offset(y: CGFloat(controller.highlightedIndex) * rowHeight) - .animation(.easeOut(duration: 0.14), value: controller.highlightedIndex) - } - } - - // MARK: - Open-With picker (the held +1-finger app list, budded over the navigator) - - /// The Open-With popup: a bounded, BubbleMorph-entrance list of the apps that can open the highlighted - /// file (`model.filesPicker`), centered over the navigator with the same tinted glass fill as the band. - /// The user scrubs it vertically (the recognizer's highlight steps are routed to `filesPickerMove`) and - /// lifts to choose. It uses the SAME single-sliding-highlight pattern as the folder list — one pill that - /// offsets by the picker index, never a per-row fill — so scrubbing never strobes (design D8). The popup - /// is a conditional member of this `ZStack`, so it buds in on insert and recedes on remove via the - /// `.bubbleMorph()` membership transition (both sides governed by the one droplet shape), the bud spring - /// driven from the enclosing `.animation(BubbleMorph.spring, value: model.filesPicker != nil)`. - @ViewBuilder - private var openWithPicker: some View { - ZStack { - if let picker = model.filesPicker { - pickerPanel(picker) - .transition(.bubbleMorph()) - } - } - .frame(maxWidth: .infinity, maxHeight: .infinity) - .animation(BubbleMorph.spring, value: model.filesPicker != nil) - } - - /// The popup body: a header, then the candidate rows behind a single sliding selection pill (the same - /// back-to-front layering the folder list uses). Bounded width/height so a long association list never - /// overruns the band — the list is gesture-scrubbed, so it does not need to show every row at once, but - /// the count here is small in practice (the apps that handle one file). - @ViewBuilder - private func pickerPanel(_ picker: LauncherModel.FilesPickerState) -> some View { - VStack(alignment: .leading, spacing: 6) { - HStack(spacing: 6) { - Image(systemName: "arrow.up.forward.app") - .font(.system(size: 12)) - .foregroundStyle(tint) - Text("Open With") - .font(.system(size: 13, weight: .semibold)) - Spacer(minLength: 4) - } - .padding(.horizontal, 4) - - ZStack(alignment: .top) { - // The single sliding selection pill BEHIND the rows (drawn first = at the back), offset by - // the picker index exactly like the folder list's highlight — never re-created per row. - if picker.highlighted != nil { - FilesRowHighlight(token: model.armingToken, armed: model.armed, dwell: model.dwell, color: tint) - .frame(height: pickerRowHeight) - .frame(maxWidth: .infinity) - .padding(.horizontal, 4) - .offset(y: CGFloat(picker.highlightedIndex) * pickerRowHeight) - .animation(.easeOut(duration: 0.14), value: picker.highlightedIndex) - } - - VStack(spacing: 0) { - ForEach(picker.candidates) { candidate in - pickerRow(candidate) - .frame(height: pickerRowHeight) - } - } - .frame(maxWidth: .infinity, alignment: .top) - } - } - .padding(10) - // Fit the grid to its content — width to the widest app name (a wider trailing allowance when a - // "Default" badge is present), height to its rows. - .frame(width: popupWidth(labels: picker.candidates.map(\.label), header: "Open With", - trailing: picker.candidates.contains(where: \.isDefault) ? 64 : 16), - height: popupHeight(rowCount: picker.candidates.count)) - .background(pickerGlassFill) - } - - /// One Open-With row: an external app (the real bundle icon, its name, and a "Default" marker on the - /// file's default app). A leaf content row (the selection backing is the single sliding pill, never a - /// per-row fill). - @ViewBuilder - private func pickerRow(_ entry: OpenWithEntry) -> some View { - HStack(spacing: 8) { - pickerRowIcon(entry) - .frame(width: 22, height: 22) - Text(entry.label) - .font(.system(size: 13)) - .lineLimit(1) - .truncationMode(.middle) - .foregroundStyle(.primary) - Spacer(minLength: 4) - if entry.isDefault { - Text("Default") - .font(.system(size: 10, weight: .semibold)) - .foregroundStyle(.secondary) - .padding(.horizontal, 6) - .padding(.vertical, 2) - .background( - Capsule().fill(tint.opacity(0.18)) - ) - } - } - .padding(.horizontal, 10) - .frame(height: pickerRowHeight) - } - - /// The leading glyph for a picker row: the real bundle icon for the external app. - @ViewBuilder - private func pickerRowIcon(_ entry: OpenWithEntry) -> some View { - switch entry { - case let .external(candidate): - Image(nsImage: NSWorkspace.shared.icon(forFile: candidate.app.url.path)) - .resizable() - .aspectRatio(contentMode: .fit) - } - } - - /// The popup's glass fill: the availability-gated `glassEffect` (macOS 26+) with the `.ultraThinMaterial` - /// fallback below it — a slightly more opaque, more rounded clone of `glassFill` so the popup reads as a - /// distinct surface floating over the navigator. - @ViewBuilder - private var pickerGlassFill: some View { - let shape = RoundedRectangle(cornerRadius: 18, style: .continuous) - if #available(macOS 26.0, *) { - Color.clear.glassEffect(.regular.tint(tint.opacity(0.16)), in: shape) - } else { - shape.fill(tint.opacity(0.12)).background(shape.fill(.ultraThinMaterial)) - } - } - - /// The popup's row metrics: a per-row height a touch taller than a folder row (it carries an app icon), - /// and a height cap so a pathologically long association list stays bounded inside the band. Width and - /// height are otherwise fit to content (`popupWidth` / `popupHeight`). - private var pickerRowHeight: CGFloat { 32 } - private var pickerMaxHeight: CGFloat { 360 } - - // MARK: - Action menu (the +1-finger menu of actions, budded over the navigator) - - /// The action-menu popup: a bounded, BubbleMorph-entrance list of the actions for the highlighted entry - /// (`model.filesActionMenu`), centered over the navigator — the SAME single-sliding-highlight pattern as - /// the Open-With picker, scrubbed vertically and resolved on lift. "Open in ▸" exits this and enters the - /// app grid; the menu and the picker are never both open. - @ViewBuilder - private var actionMenuPopup: some View { - ZStack { - if let menu = model.filesActionMenu { - actionMenuPanel(menu) - .transition(.bubbleMorph()) - } - } - .frame(maxWidth: .infinity, maxHeight: .infinity) - .animation(BubbleMorph.spring, value: model.filesActionMenu != nil) - } - - /// The menu body: a header naming the entry, then the action rows behind the single sliding pill (the - /// same layering as the picker / folder list — never a per-row fill, so scrubbing never strobes). - @ViewBuilder - private func actionMenuPanel(_ menu: LauncherModel.FilesActionMenuState) -> some View { - VStack(alignment: .leading, spacing: 6) { - HStack(spacing: 6) { - Image(systemName: menu.entry.isDirectory ? "folder" : "doc") - .font(.system(size: 12)) - .foregroundStyle(tint) - Text(menu.entry.name) - .font(.system(size: 13, weight: .semibold)) - .lineLimit(1) - .truncationMode(.middle) - Spacer(minLength: 4) - } - .padding(.horizontal, 4) - - ZStack(alignment: .top) { - if menu.highlighted != nil { - FilesRowHighlight(token: model.armingToken, armed: model.armed, dwell: model.dwell, color: tint) - .frame(height: pickerRowHeight) - .frame(maxWidth: .infinity) - .padding(.horizontal, 4) - .offset(y: CGFloat(menu.highlightedIndex) * pickerRowHeight) - .animation(.easeOut(duration: 0.14), value: menu.highlightedIndex) - } - VStack(spacing: 0) { - ForEach(menu.rows) { row in - actionMenuRow(row) - .frame(height: pickerRowHeight) - } - } - .frame(maxWidth: .infinity, alignment: .top) - } - } - .padding(10) - // Fit the menu to its content — width to the widest label, height to its rows (not a fixed box). - .frame(width: popupWidth(labels: menu.rows.map { Self.menuRowLabel($0) }, header: menu.entry.name, trailing: 26), - height: popupHeight(rowCount: menu.rows.count)) - .background(pickerGlassFill) - } - - /// Fit a popup's WIDTH to its content (the requested behavior, shared by the action menu and the Open-With - /// grid): the widest row `labels` rendered at the real 13pt menu font, plus the leading glyph/icon, the - /// inter-spacing, a `trailing` allowance (the action menu's chevron / the picker's "Default" badge), and - /// the paddings — measured against the `header` too, and clamped to a sensible range. A definite width - /// keeps the single sliding highlight (`maxWidth: .infinity`) spanning cleanly. - private func popupWidth(labels: [String], header: String, trailing: CGFloat) -> CGFloat { - func textWidth(_ s: String, _ font: NSFont) -> CGFloat { - (s as NSString).size(withAttributes: [.font: font]).width - } - let rowFont = NSFont.systemFont(ofSize: 13) - let headerFont = NSFont.systemFont(ofSize: 13, weight: .semibold) - // Row: glyph/icon(22) + spacing(8) + label + trailing + row h-padding(20). - let widestRow = labels.map { textWidth($0, rowFont) + 22 + 8 + trailing + 20 }.max() ?? 0 - // Header: icon(12) + spacing(6) + text + slack(16). - let headerW = textWidth(header, headerFont) + 12 + 6 + 16 - let content = max(widestRow, headerW) + 20 // panel h-padding (10 each side) - return min(max(content, Self.popupMinWidth), Self.popupMaxWidth) - } - - /// Fit a popup's HEIGHT to its `rowCount`: the header + the rows (each `pickerRowHeight`) + the VStack - /// spacing + the panel's vertical padding — so the panel is exactly as tall as its content, clamped to the - /// safety cap (`pickerMaxHeight`) for a pathologically long list. Shared by both popups. - private func popupHeight(rowCount: Int) -> CGFloat { - let header: CGFloat = 18, vstackSpacing: CGFloat = 6, vPadding: CGFloat = 20 - let content = header + vstackSpacing + CGFloat(max(rowCount, 1)) * pickerRowHeight + vPadding - return min(content, pickerMaxHeight) - } - - private static let popupMinWidth: CGFloat = 170 - private static let popupMaxWidth: CGFloat = 340 - - /// One action-menu row: a glyph + label, with a disclosure chevron on "Open in ▸" (which descends into - /// the app grid). A leaf row — the selection backing is the single sliding pill. - @ViewBuilder - private func actionMenuRow(_ row: FilesMenuRow) -> some View { - HStack(spacing: 8) { - Image(systemName: Self.menuRowGlyph(row)) - .font(.system(size: 13)) - .foregroundStyle(tint) - .frame(width: 22, height: 22) - Text(Self.menuRowLabel(row)) - .font(.system(size: 13)) - .lineLimit(1) - .truncationMode(.middle) - .foregroundStyle(.primary) - Spacer(minLength: 4) - if case .action(.openIn) = row { - Image(systemName: "chevron.right") - .font(.system(size: 10, weight: .semibold)) - .foregroundStyle(.secondary) - } - } - .padding(.horizontal, 10) - .frame(height: pickerRowHeight) - } - - /// The human label for a menu row (a tool row names the tool; "Open in…" is the app grid). - static func menuRowLabel(_ row: FilesMenuRow) -> String { - switch row { - case let .tool(_, tool): return "Open in \(tool.name)" - case let .action(action): - switch action { - case .copyAsPath: return "Copy as Path" - case .copy: return "Copy" - case .cut: return "Cut" - case .pasteInto: return "Paste" - case .openIn: return "Open in…" - case .delete: return "Delete" - case .openInTerminals: return "Open in Terminal" - case .openInEditor: return "Open in Editor" - case .revealInFinder: return "Reveal in Finder" - case .addToFavorites: return "Add to Favorites" - case .copyName: return "Copy Name" - } - } - } - - /// The SF Symbol for a menu row. - static func menuRowGlyph(_ row: FilesMenuRow) -> String { - switch row { - case let .tool(action, _): - return action == .openInEditor ? "chevron.left.forward.slash.chevron.right" : "terminal" - case let .action(action): - switch action { - case .copyAsPath: return "doc.on.clipboard" - case .copy: return "doc.on.doc" - case .cut: return "scissors" - case .pasteInto: return "arrow.down.doc" - case .openIn: return "arrow.up.forward.app" - case .delete: return "trash" - case .openInTerminals: return "terminal" - case .openInEditor: return "chevron.left.forward.slash.chevron.right" - case .revealInFinder: return "magnifyingglass" - case .addToFavorites: return "star" - case .copyName: return "textformat" - } - } - } - - // MARK: - Failure row (a failed open, surfaced bounded + non-blocking) - - /// The Files-band failure surface: a bounded card pinned to the bottom of the navigator when - /// `model.filesOpenFailure != nil`, with the clean headline (capped + middle-truncating so an - /// unexpectedly long message degrades instead of overrunning the band), an opt-in "Show details / Copy" - /// disclosure for the raw text, and **Retry** + **Dismiss** affordances. It is a conditional member of - /// this `ZStack`, so it buds in on insert and recedes on remove via the `.bubbleMorph()` membership - /// transition (both sides governed by the one droplet shape) — never an app-modal alert (spec: bounded + - /// non-blocking, never silent). The driving bud spring sits on the enclosing `.animation`. - @ViewBuilder - private var failureRow: some View { - ZStack { - if let failure = model.filesOpenFailure { - failureCard(failure) - .transition(.bubbleMorph(anchor: .bottom)) - } - } - .frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .bottom) - .animation(BubbleMorph.spring, value: model.filesOpenFailure) - } - - /// The failure card body: a warning glyph + the clean headline, the opt-in details disclosure (only when - /// there is raw text), then the action row. Bounded width/height; the headline never carries raw text. - @ViewBuilder - private func failureCard(_ failure: LauncherModel.FilesOpenFailure) -> some View { - VStack(alignment: .leading, spacing: 6) { - HStack(alignment: .firstTextBaseline, spacing: 6) { - Image(systemName: "exclamationmark.triangle.fill") - .font(.system(size: 12)) - .foregroundStyle(.orange) - // The concise headline is primary: capped + middle-truncating so a long message degrades - // gracefully instead of overflowing the fixed card (spec: bounded). Never raw error text. - Text(failure.headline) - .font(.system(size: 13)) - .foregroundStyle(.primary) - .lineLimit(2) - .truncationMode(.middle) - .fixedSize(horizontal: false, vertical: true) - .textSelection(.enabled) - Spacer(minLength: 4) - } - - // The raw OS/workspace text rides ONLY here, behind an opt-in disclosure — never inline. - if let details = failure.details, !details.isEmpty { - failureDetailsDisclosure(details) - } - - HStack(spacing: 8) { - Spacer(minLength: 0) - Button { - showingFailureDetails = false - model.onFilesRetryOpen?() - } label: { - Label("Retry", systemImage: "arrow.clockwise") - } - .controlSize(.small) - Button { - showingFailureDetails = false - model.filesOpenFailure = nil // Dismiss: clear the failure (a bounded, non-blocking exit) - } label: { - Label("Dismiss", systemImage: "xmark") - } - .controlSize(.small) - } - } - .padding(10) - .frame(width: failureCardWidth) - .frame(maxHeight: failureCardMaxHeight) - .background(failureGlassFill) - .padding(.bottom, 10) - } - - /// A collapsed "Show details" disclosure for the raw technical text behind the failure: bounded (the text - /// scrolls past ~120pt) so even a giant dump can't grow the card, with a "Copy details" action. Cloned - /// from `ModelDetailView.detailsDisclosure` so the same error reads / copies identically everywhere. - @ViewBuilder - private func failureDetailsDisclosure(_ details: String) -> some View { - DisclosureGroup(isExpanded: $showingFailureDetails) { - VStack(alignment: .leading, spacing: 6) { - ScrollView { - Text(details) - .font(.system(.caption2, design: .monospaced)) - .foregroundStyle(.secondary) - .textSelection(.enabled) - .frame(maxWidth: .infinity, alignment: .leading) - } - .frame(maxHeight: 120) - Button { - let pb = NSPasteboard.general - pb.clearContents() - pb.setString(details, forType: .string) - } label: { - Label("Copy details", systemImage: "doc.on.doc") - } - .controlSize(.small) - } - .padding(.top, 4) - } label: { - Text("Show details").font(.caption).foregroundStyle(.secondary) - } - } - - /// The failure card's glass fill: the availability-gated `glassEffect` (macOS 26+) with the - /// `.ultraThinMaterial` fallback below it — the same tinted-glass surface idiom as `pickerGlassFill`, so - /// the card reads as a distinct surface floating over the navigator. - @ViewBuilder - private var failureGlassFill: some View { - let shape = RoundedRectangle(cornerRadius: 16, style: .continuous) - if #available(macOS 26.0, *) { - Color.clear.glassEffect(.regular.tint(tint.opacity(0.16)), in: shape) - } else { - shape.fill(tint.opacity(0.12)).background(shape.fill(.ultraThinMaterial)) - } - } - - /// The failure card's fixed metrics: a comfortable bounded width and a capped height so an expanded - /// details disclosure stays scroll-safe inside the band. - private var failureCardWidth: CGFloat { 320 } - private var failureCardMaxHeight: CGFloat { 280 } - - // MARK: - Live preview (right) - - /// The preview pane for the current highlight: a QuickLook content preview for a **file** (reusing the - /// Clipboard band's `FilePreview`, with the file/app icon as a fallback), or a **peek** of a folder's - /// contents (the same listing a descend would promote, so the peek and a subsequent descend agree). The - /// pane buds in; switching highlight content re-buds via `.id` on the previewed entry. It is **not** - /// separately navigable (there is no horizontal crossing into it — horizontal is the depth axis). - @ViewBuilder - private func preview(_ controller: FilesColumnController) -> some View { - Group { - switch controller.previewTarget { - case let .file(entry): - FilePreview(url: entry.url) - case let .folder(entry, contents): - FolderPeek(folder: entry, contents: contents, tint: tint) - case nil: - Color.clear - } - } - // The previewed entry's id keys the swap, so moving the highlight tears down the old preview and - // buds the new one in (and the leaving one recedes). Use the membership *transition* (not the - // `.bubbleMorph()` modifier) across an `.id` swap so the same droplet shape governs BOTH sides, - // per `BubbleMorph`'s own guidance; the driving `.animation(value:)` sits on the container. - .id(previewID(controller)) - .transition(.bubbleMorph()) - .frame(maxWidth: .infinity, maxHeight: .infinity) - .padding(12) - .animation(BubbleMorph.spring, value: previewID(controller)) - } - - /// A `Hashable` identity for whatever the preview currently shows — the highlighted entry's path, or a - /// sentinel when the column is empty. Drives the preview's bud-in/recede on highlight changes. - private func previewID(_ controller: FilesColumnController) -> String { - controller.highlightedEntry?.id ?? "\u{2014}empty" - } - - // MARK: - Glass fill (tinted from AppSettings.filesBandTint) - - /// The band surface fill: the availability-gated `glassEffect(.regular.tint(...))` (macOS 26+) with the - /// `.ultraThinMaterial` fallback below it — cloned from `LauncherView.bandIconBackground` / `HubGlass`, - /// tinted from `AppSettings.filesBandTint`. Subtle so the rows/preview read on top of it. - @ViewBuilder - private var glassFill: some View { - let shape = RoundedRectangle(cornerRadius: 24, style: .continuous) - if #available(macOS 26.0, *) { - Color.clear.glassEffect(.regular.tint(tint.opacity(0.12)), in: shape) - } else { - shape.fill(tint.opacity(0.08)).background(shape.fill(.ultraThinMaterial)) - } - } - - // MARK: - Derived metrics & helpers - - /// The per-row height for the live density (`FilesBandLayout.rowHeight(for:)`) — also the highlight's - /// slide step, so the highlight and the rows can never drift. (The current-list *column width* is the - /// FIXED `FilesBandLayout.currentColumnWidth` now, not the live `filesColumnWidth` setting — refinement - /// 3 pins the in-launcher navigator to the Clipboard-sized container, so the panel never resizes.) - private var rowHeight: CGFloat { FilesBandLayout.rowHeight(for: settings.filesDensity) } - - /// The band tint as a SwiftUI `Color`, parsed from the `AppSettings.filesBandTint` hex (falling back to - /// the builder's default blue if the hex is malformed). The single place the band's accent is resolved. - private var tint: Color { - Color(hexString: settings.filesBandTint) ?? Color(FilesBandBuilder.color) - } - - /// A `Hashable` identity for the current depth (the `.id` the depth transition keys on): the current - /// folder's path, or a sentinel for the roots list. Descend/ascend change it, so SwiftUI swaps the - /// current-list subtree with the scaling bubble transition. - private func depthID(_ controller: FilesColumnController) -> String { - controller.current.folderURL?.path ?? "\u{1F4C1}roots" - } - - /// The secondary metadata string for a row, per `AppSettings.filesRowMetadata` (an `OptionSet`, so - /// several may show — joined with a middle dot). Folders show an item count for `.size`; files a byte - /// size. Returns nil when no metadata is selected (or none is available). - private func metadataLabel(_ entry: FileEntry) -> String? { - var parts: [String] = [] - let meta = settings.filesRowMetadata - if meta.contains(.date), let date = entry.modificationDate { - parts.append(Self.dateFormatter.string(from: date)) - } - if meta.contains(.kind) { - parts.append(Self.kindLabel(entry.kind)) - } - if meta.contains(.size) { - if entry.isDirectory { - if let count = controller?.cache[entry.url.standardizedFileURL.path]?.count { - parts.append("\(count) item\(count == 1 ? "" : "s")") - } - } else if let size = Self.fileSize(entry.url) { - parts.append(Self.byteFormatter.string(fromByteCount: size)) - } - } - return parts.isEmpty ? nil : parts.joined(separator: " · ") - } - - // MARK: - Static formatters / label maps - - private static let dateFormatter: DateFormatter = { - let f = DateFormatter() - f.dateStyle = .short - f.timeStyle = .none - return f - }() - - private static let byteFormatter: ByteCountFormatter = { - let f = ByteCountFormatter() - f.countStyle = .file - return f - }() - - private static func fileSize(_ url: URL) -> Int64? { - let values = try? url.resourceValues(forKeys: [.fileSizeKey]) - return values?.fileSize.map(Int64.init) - } - - /// A human label for a `FileKind` (the `.kind` row metadata). View-layer concern, kept here. - private static func kindLabel(_ kind: FileKind) -> String { - switch kind { - case .folder: return "Folder" - case .image: return "Image" - case .audio: return "Audio" - case .video: return "Video" - case .pdf: return "PDF" - case .archive: return "Archive" - case .sourceCode: return "Code" - case .text: return "Text" - case .application: return "App" - case .other: return "Document" - } - } -} - -// MARK: - Folder-contents peek (preview pane, folder highlight) - -/// A peek of a highlighted **folder's** contents in the preview pane: the same listing a descend would -/// promote to the current column (so the peek and the descend agree — `FilesNavigationModel.PreviewTarget -/// .folder(_, contents:)` carries it). A compact, NON-navigable list (horizontal is the depth axis, so the -/// preview is never crossed into) — a header, then a bounded run of rows, with an overflow count. Each row -/// buds in via the membership transition; the container buds via `BubbleMorph` at the call site. -private struct FolderPeek: View { - let folder: FileEntry - let contents: [FileEntry] - let tint: Color - - /// Rows shown before an overflow line — bounded so a large folder's peek never overruns the pane. - private let visibleLimit = 12 - - var body: some View { - VStack(alignment: .leading, spacing: 6) { - HStack(spacing: 6) { - Image(systemName: "folder.fill").foregroundStyle(tint) - Text(folder.name).font(.system(size: 13, weight: .medium)).lineLimit(1).truncationMode(.middle) - Spacer(minLength: 4) - Text("\(contents.count) item\(contents.count == 1 ? "" : "s")") - .font(.system(size: 11)).foregroundStyle(.secondary) - } - Divider().opacity(0.2) - if contents.isEmpty { - Text("Empty folder").font(.system(size: 12)).foregroundStyle(.secondary) - .frame(maxWidth: .infinity, alignment: .leading) - } else { - ForEach(contents.prefix(visibleLimit)) { entry in - HStack(spacing: 6) { - Image(systemName: peekSymbol(entry.kind)) - .font(.system(size: 11)) - .foregroundStyle(entry.isDirectory ? tint : Color.secondary) - .frame(width: 16) - Text(entry.name).font(.system(size: 12)).foregroundStyle(.secondary) - .lineLimit(1).truncationMode(.middle) - Spacer(minLength: 0) - } - } - if contents.count > visibleLimit { - Text("+ \(contents.count - visibleLimit) more") - .font(.system(size: 11)).foregroundStyle(.tertiary) - .padding(.top, 2) - } - } - Spacer(minLength: 0) - } - .frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .topLeading) - } - - private func peekSymbol(_ kind: FileKind) -> String { - if case let .sfSymbol(name) = FilesBandBuilder.glyph(for: kind) { return name } - return "doc" - } -} - -// MARK: - The sliding row highlight (Files band) - -/// The Files band's dwell highlight, shared by the folder list **and** the sub-column popups (the action menu -/// and the Open-With / app grid): a Liquid Glass pill that starts nearly clear and tints over the dwell, then -/// locks when armed — a clone of `ClipboardBandView.RowHighlight` (which is `private` to that file). A single -/// persistent view that **slides** between rows (the caller offsets it by index), so scrubbing never strobes -/// (design D8). `token` re-charges the pill per selection; it carries the existing linear-charge / -/// ease-out-arm motion vocabulary unchanged (no bubble-morph; the arm haptic lives in the controller). The -/// charge is the always-present arm signal now that a Files lift fires only when armed -/// (add-files-band-dwell-arm). -private struct FilesRowHighlight: View { - let token: Int - let armed: Bool - let dwell: Double - let color: Color - @State private var intensity: CGFloat = 0 - - var body: some View { - let shape = RoundedRectangle(cornerRadius: 10, style: .continuous) - Group { - if #available(macOS 26.0, *) { - Color.clear - .glassEffect(.regular.tint(color.opacity(0.10 + 0.42 * intensity)), in: shape) - } else { - shape.fill(color.opacity(0.08 + 0.46 * intensity)) - .background(shape.fill(.regularMaterial)) - } - } - .onAppear { restart() } - .onChange(of: token) { restart() } - .onChange(of: armed) { if armed { withAnimation(.easeOut(duration: 0.10)) { intensity = 1 } } } - } - - private func restart() { - intensity = 0 - withAnimation(.linear(duration: dwell)) { intensity = 1 } - } -} - -// MARK: - Hex → Color - -extension Color { - /// Build a `Color` from a `#RRGGBB` / `#RRGGBBAA` (or un-prefixed) hex string — the form - /// `AppSettings.filesBandTint` persists. Returns nil for a malformed string so the caller can fall back - /// to a default. 3-digit shorthand is not supported (the settings always write 6/8 digits). - init?(hexString: String) { - var hex = hexString.trimmingCharacters(in: .whitespacesAndNewlines) - if hex.hasPrefix("#") { hex.removeFirst() } - guard hex.count == 6 || hex.count == 8, let value = UInt64(hex, radix: 16) else { return nil } - let r, g, b, a: Double - if hex.count == 8 { - r = Double((value >> 24) & 0xFF) / 255 - g = Double((value >> 16) & 0xFF) / 255 - b = Double((value >> 8) & 0xFF) / 255 - a = Double(value & 0xFF) / 255 - } else { - r = Double((value >> 16) & 0xFF) / 255 - g = Double((value >> 8) & 0xFF) / 255 - b = Double(value & 0xFF) / 255 - a = 1 - } - self.init(.sRGB, red: r, green: g, blue: b, opacity: a) - } -} - diff --git a/Sources/ThreeFingerSwitcher/Overlay/FilesDwellArming.swift b/Sources/ThreeFingerSwitcher/Overlay/FilesDwellArming.swift deleted file mode 100644 index aba1dd1..0000000 --- a/Sources/ThreeFingerSwitcher/Overlay/FilesDwellArming.swift +++ /dev/null @@ -1,40 +0,0 @@ -/// The pure decision behind the Files-band dwell-to-arm: given the identity of the currently-highlighted -/// Files "thing" — a folder row (its standardized path) or, when a sub-column is open, that popup's -/// highlighted-row key — and the last identity that began charging, decide whether to **restart** the dwell, -/// **keep** the running one, or **disarm**. -/// -/// It is identity-keyed on purpose. The `+1`-finger morph that opens the action menu moves no highlight (the -/// recognizer re-baselines the contact-count change without emitting a step), so the identity is unchanged and -/// the decision is `.keep` — the arm the user charged on that row survives into the menu-open gate. Any real -/// move (a highlight step, a depth descend/ascend, a sub-column scrub, an async re-list that shifts the row, -/// entering/leaving a sub-column) changes the identity → `.restart`. An empty column (`identity == nil`) -/// after having been on something → `.disarm`. -/// -/// Pure (no timer, no haptic, no `@MainActor`) so the restart logic is unit-tested directly; the controller -/// owns the `DwellArmDriver` timer + the arm haptic and just acts on the decision. -struct FilesDwellArming { - enum Decision: Equatable { - /// The highlighted thing changed and is real → cancel any running charge and begin a fresh dwell. - case restart - /// The same thing is still highlighted → leave the running charge (or settled arm) untouched. - case keep - /// The highlight moved onto nothing (an empty column) → cancel and disarm. - case disarm - } - - private(set) var lastIdentity: String? - - /// Feed the current identity (nil = empty column / nothing to arm); returns the decision and records the - /// new identity. Calling repeatedly with the same identity yields `.keep` — so it is safe to call after - /// every Files move and on async landings. - mutating func update(identity: String?) -> Decision { - guard identity != lastIdentity else { return .keep } - lastIdentity = identity - return identity == nil ? .disarm : .restart - } - - /// Forget the last identity so the next `update` treats even an unchanged highlight as a fresh charge — - /// for the re-arm seams where the drill is restarted in place (a delivery that failed but kept the - /// navigator open) and the user must re-dwell before another committing lift fires. - mutating func reset() { lastIdentity = nil } -} diff --git a/Sources/ThreeFingerSwitcher/Overlay/LauncherGridLayout.swift b/Sources/ThreeFingerSwitcher/Overlay/LauncherGridLayout.swift index b873ccf..7159856 100644 --- a/Sources/ThreeFingerSwitcher/Overlay/LauncherGridLayout.swift +++ b/Sources/ThreeFingerSwitcher/Overlay/LauncherGridLayout.swift @@ -104,95 +104,3 @@ enum ClipboardBandLayout { static let padding = LauncherGridLayout.containerPadding } -/// Metrics for the Files band's bounded column-navigator (design D6): a thin **ancestor icon rail**, -/// the **current list** column, one **preview** pane, and a full-width **breadcrumb bar** at the bottom. -/// Both `FilesBandView` (which lays the regions out) and the panel sizer in `LauncherOverlayController` -/// read THIS one enum, so the rendered surface and the `NSWindow` frame can never drift (the same -/// contract `ClipboardBandLayout` gives the Clipboard band). -/// -/// **Fixed container (refinement 3).** The container is the EXACT size of the Clipboard band's container -/// (`ClipboardBandLayout.containerWidth` × `containerHeight`) — a constant, with **no per-depth and no -/// per-density variation**. Crossing into the band or changing depth never resizes or moves the panel on -/// screen; instead the current-folder list **scrolls** (a `ScrollViewReader`, driven by the vertical -/// edge-auto-repeat) when it is taller than the fixed row area. The three columns lay out *within* that -/// fixed width — the rail and current list are fixed widths, and the preview FILLS the remainder (the same -/// fixed-key-column-then-fill split `ClipboardBandLayout` uses), so they always sum to `containerWidth`. -enum FilesBandLayout { - /// Width of the collapsed ancestor **icon rail** on the left — just wide enough for one tile + - /// breathing room (the Hub-sidebar / launcher-band-strip idiom, narrowed because it only holds the - /// path's folder icons, never labels). Zero ancestors (the roots list) still reserves it so the - /// current column doesn't jump sideways on the first descend. - static let ancestorRailWidth: CGFloat = 56 - /// Fixed tile size for an ancestor rail icon (the rail's glyphs are leaf icons, so they are NOT - /// bubble-morphed; the rail itself can be). - static let ancestorIconSize: CGFloat = 30 - /// Vertical gap between ancestor rail icons. - static let ancestorRowSpacing: CGFloat = 10 - - /// Width of the **current list** column inside the FIXED container — seeded from - /// `AppSettings.Defaults.filesColumnWidth` (~260pt). Within the fixed Clipboard-sized launcher container - /// this is a CONSTANT (the preview fills the remainder); the per-user `AppSettings.filesColumnWidth` - /// "live column width" tuning is the *panel-width* variant that refinement 3 drops — so the in-launcher - /// navigator uses THIS constant, not the live setting, keeping the three-pane split summing to - /// `containerWidth` exactly at any column setting. - static let currentColumnWidth: CGFloat = CGFloat(AppSettings.Defaults.filesColumnWidth) - - /// Width of the **preview** pane on the right (file QuickLook / folder-contents peek): the remainder of - /// the fixed container after the rail, the current list, the two dividers, and the outer padding — the - /// same fixed-column-then-fill split `ClipboardBandLayout` gives its value preview. Wider than the - /// current column (the container is the roomy Clipboard width), so a document preview reads comfortably. - /// Computed from the constants, so rail + current + preview + dividers + padding == `containerWidth`. - static var previewWidth: CGFloat { - containerWidth - ancestorRailWidth - currentColumnWidth - 2 * dividerWidth - 2 * padding - } - - /// Internal padding around the whole navigator (shared with the grid/clipboard container padding so - /// the three band surfaces frame identically). - static let padding = LauncherGridLayout.containerPadding - /// Width consumed by each `Divider()` between the three regions (rail | current | preview). - static let dividerWidth: CGFloat = 1 - - /// Height of the full-width **breadcrumb bar** pinned at the BOTTOM, spanning all three columns - /// (refinement 4): it shows the path to the currently-highlighted item and updates live as the highlight - /// moves. A fixed strip subtracted from `containerHeight` so it never eats the scroll area. - static let breadcrumbBarHeight: CGFloat = 30 - - /// Per-row height for the current list at the given density. The concrete point metrics for each - /// `FilesDensity` case live here (the enum's doc-comment defers them to "the view layer"): a tight - /// pack, a default, and a roomier row. - static func rowHeight(for density: FilesDensity) -> CGFloat { - switch density { - case .compact: return 32 - case .comfortable: return 40 - case .spacious: return 48 - } - } - - /// The container width — the **exact** Clipboard container width (refinement 3): a fixed constant, with - /// no per-depth and no per-column variation, so crossing in / changing depth never resizes or moves the - /// panel. The interior split (rail + current list + preview + dividers + padding) sums to exactly this. - static var containerWidth: CGFloat { ClipboardBandLayout.containerWidth } - - /// The container height — the **exact** Clipboard container height (refinement 3): a fixed constant, with - /// no per-density variation. A folder taller than the fixed `rowAreaHeight` scrolls inside; the frame - /// never grows for it. - static var containerHeight: CGFloat { ClipboardBandLayout.containerHeight } - - /// Height of the **scrollable current-folder row area** (refinement 3): the fixed container height minus - /// the bottom breadcrumb bar and the outer top/bottom padding — so the view knows how tall the scroll - /// region is. The view scrolls the current list within this region (a `ScrollViewReader` following the - /// highlight) when the folder has more rows than fit; the container itself stays the fixed - /// `containerHeight`. Density-independent (it's container minus chrome); the number of rows that fit is - /// `floor(rowAreaHeight / rowHeight(for: density))`, so density changes only how many rows show before it - /// scrolls, never the container. - static var rowAreaHeight: CGFloat { - containerHeight - breadcrumbBarHeight - 2 * padding - } - - /// How many current-list rows fit the fixed `rowAreaHeight` at `density` before the list must scroll — - /// `floor(rowAreaHeight / rowHeight)`, at least one. The view uses this to decide when to begin scrolling - /// the list to keep the highlighted row visible (refinement 3). - static func visibleRowCount(for density: FilesDensity) -> Int { - max(1, Int(rowAreaHeight / rowHeight(for: density))) - } -} diff --git a/Sources/ThreeFingerSwitcher/Overlay/LauncherModel.swift b/Sources/ThreeFingerSwitcher/Overlay/LauncherModel.swift index 76ce88e..9c2ecaa 100644 --- a/Sources/ThreeFingerSwitcher/Overlay/LauncherModel.swift +++ b/Sources/ThreeFingerSwitcher/Overlay/LauncherModel.swift @@ -14,55 +14,6 @@ import Combine final class LauncherModel: ObservableObject { enum Focus: Equatable { case bands, grid } - /// The held Open-With picker's state (see `filesPicker`): the relevant-apps `candidates` for the - /// highlighted file (already enumerated by `FileOpenService.openWithCandidates`, default app indicated) - /// and the `highlightedIndex` the user scrubs to. A value type so the whole sub-state publishes as one - /// `@Published` change; equality drives the view's diffing. - struct FilesPickerState: Equatable { - /// The picker rows: the external applications that can open the file, in the system's order (the - /// default external is `isDefault`). - var candidates: [OpenWithEntry] - /// The currently highlighted row (the entry a lift would choose). Always a valid index while non-empty. - var highlightedIndex: Int - - /// The highlighted entry, or nil for an (defensively) empty list. - var highlighted: OpenWithEntry? { - candidates.indices.contains(highlightedIndex) ? candidates[highlightedIndex] : nil - } - } - - /// The held **action menu** sub-state for the Files band (`files-action-menu`): the rows for the - /// highlighted `entry` that the menu excursion (the `+1`-finger lift) opened, plus the highlighted row. - /// A transient popup budded over the column navigator — scrubbed vertically, resolved on lift (commit the - /// row) / horizontal swipe (back out). Mirrors `FilesPickerState`; `entry` is retained so a back-out from - /// the app grid (entered via "Open in") can rebuild the menu. - struct FilesActionMenuState: Equatable { - /// The entry the menu acts on (file or folder). - var entry: FileEntry - /// The concrete, ordered rows (already resolved by `FilesActionMenu.visibleRows`). - var rows: [FilesMenuRow] - /// The highlighted row a lift would commit. Always valid while `rows` is non-empty. - var highlightedIndex: Int - - /// The highlighted row, or nil for an (defensively) empty menu. - var highlighted: FilesMenuRow? { - rows.indices.contains(highlightedIndex) ? rows[highlightedIndex] : nil - } - } - - /// A failed Files-band open, surfaced as observable bounded state (spec: "Failures are observable, never - /// silent"). Carries the clean, bounded `headline` (the `FileActionError` sentence — never raw error - /// text) and the opt-in copyable `details` (the raw OS/workspace text, surfaced only behind a "Show - /// details / Copy" disclosure; `nil` when the headline already says everything). Mirrored from - /// `FileOpenService.State.failed` by the coordinator's state sink; nil when there is no live failure. A - /// value type so the whole sub-state publishes as one `@Published` change and equality drives diffing. - struct FilesOpenFailure: Equatable { - /// The clean, bounded, user-facing message (never raw error text). - var headline: String - /// The opt-in raw text for a "Show details / Copy" disclosure, or nil when there is nothing extra. - var details: String? - } - // Grid source of truth. @Published private(set) var bands: [[LaunchItem]] = [] @Published private(set) var bandNames: [String] = [] @@ -90,64 +41,6 @@ final class LauncherModel: ObservableObject { /// single-column master-detail list and repurposes horizontal travel (see `stepHorizontal`). @Published private(set) var clipboardBandIndex: Int? - /// Index of the synthetic Files band, when present (mirrors `clipboardBandIndex`). It navigates as a - /// single-column **directory column**: horizontal drills (descend / ascend) instead of stepping a row, - /// vertical moves the highlight, and the band's items are reprojected from the column on every move - /// (see `stepHorizontal` / `stepVertical` and `FilesColumnController`). - @Published private(set) var filesBandIndex: Int? - - /// Owns the Files band's column navigation + the async-listing cache bridge while a Files band is - /// present (built in `setBands`). Nil when the Files band isn't injected. The view reads the highlighted - /// entry / preview off this; the model routes drill / highlight steps into it and reprojects the band. - @Published private(set) var filesColumn: FilesColumnController? - - /// The held **Open-With picker** sub-state for the Files band: the relevant-apps list a relative - /// +1-finger lift opened (`AppCoordinator.filesOpenWith` → `enterFilesPicker`), plus the highlighted - /// row. Nil when not picking — the picker is a transient overlay budded over the column navigator, - /// scrubbed vertically and resolved on the next lift (choose) / horizontal swipe (back to the list). - /// `FilesBandView` observes this to render the popup; the recognizer's depth/highlight/open intents are - /// routed to the picker (vs. the folder list) by the coordinator whenever it is non-nil. - @Published var filesPicker: FilesPickerState? - - /// The held **action menu** sub-state (`files-action-menu`): the rows a `+1`-finger lift opened over the - /// highlighted file/folder. Nil when not open. `FilesBandView` observes this to render the popup; the - /// coordinator routes highlight/lift/discard to it (vs. the folder list / the Open-With picker) while - /// non-nil. The menu and the Open-With picker are never both open — "Open in ▸" exits the menu and enters - /// the picker. - @Published var filesActionMenu: FilesActionMenuState? - - /// The current failed Files-band open, mirrored from `FileOpenService.State` by the coordinator's state - /// sink (set on `.failed`, cleared to nil on `.idle`/`.opening`/`.opened`). When non-nil, `FilesBandView` - /// renders a **bounded, non-blocking** failure row at the bottom of the navigator (headline + opt-in - /// details + Retry/Dismiss) — never an app-modal alert (spec: bounded + non-blocking, never silent). - @Published var filesOpenFailure: FilesOpenFailure? - - /// Whether the Files band is current AND the Open-With picker is open — the predicate the coordinator - /// branches the recognizer's Files intents on (picker-mode = scrub the app list, not the folder list). - var isPickingOpenWith: Bool { currentBandIsFiles && filesPicker != nil } - - /// Called when a Files-band depth change should persist the per-root remembered location (wired to - /// `AppSettings.rememberLocation(_:forRoot:)` by the controller). Keyed/valued by standardized path. - var onFilesRememberLocation: ((_ path: String, _ rootPath: String) -> Void)? - - /// Called when the Files-band failure row's **Retry** is tapped — re-fires the last open through the - /// coordinator (which re-prepares/commits it on `FileOpenService`, so a transient failure can be - /// retried without re-navigating). Wired by `AppCoordinator`; nil (no-op) when nothing can be retried. - var onFilesRetryOpen: (() -> Void)? - - /// Called when an **async** directory listing lands and reprojects the Files band (the column was a cache - /// miss and has now filled / shifted the highlight). The controller wires this to recharge the dwell-to-arm - /// on the newly-available row, so a freshly-descended folder never arms a stale/empty highlight. Fired only - /// from the async `onColumnChanged` path (synchronous moves recharge through the controller already). - var onFilesProjectionChanged: (() -> Void)? - - /// When non-nil, the launcher is showing the AI streaming preview canvas for the fired command - /// (the grid is replaced by the canvas; the overlay stays visible, non-activating). Cleared when - /// the canvas is dismissed (commit / discard). This is the only state the canvas-mode UI binds to; - /// the live model/streaming state lives in the injected `AICommandExecutor` the view observes. - @Published private(set) var canvasCommand: AICommand? - /// True while the AI preview canvas is on screen. - var canvasActive: Bool { canvasCommand != nil } /// Session-only set of clipboard entry ids whose pin was toggled this session — drives the pin /// marker without reordering the live list (the reorder is deferred to the next band build). @Published private(set) var sessionPinToggles: Set = [] @@ -168,43 +61,19 @@ final class LauncherModel: ObservableObject { var bandCount: Int { bands.count } /// Whether the current band is the Clipboard band (single column, repurposed horizontal). var currentBandIsClipboard: Bool { clipboardBandIndex == currentBand } - /// Whether the current band is the Files band (single column, horizontal drills the directory tree). - var currentBandIsFiles: Bool { filesBandIndex == currentBand } - - /// Whether the Files **directory drill** is engaged: the Files band is current AND focus has crossed - /// INTO the file column (`.grid`), not merely resting on the band-rail icon (`.bands`). The controller - /// gates the recognizer's `filesDrillActive` on THIS (not on `currentBandIsFiles`), so while the - /// highlight sits on the Files band icon the band behaves like any other — a horizontal step crosses - /// into the column, a vertical step switches bands, and a lift DISMISSES the launcher (refinement 1). - /// The drill/open behaviour (descend / ascend, a lift opens the highlighted entry) engages only once - /// the cursor is in the column. `currentBandIsFiles == false` ⇒ this is `false` too. - var filesDrillEngaged: Bool { currentBandIsFiles && focus == .grid } - /// Columns for the current band: the Clipboard and Files bands are single-column lists; others use the - /// grid. (The Files band's horizontal axis drills folders rather than stepping a row, but it is still a - /// one-wide column so row math stays single-column.) + /// Columns for the current band: the Clipboard band is a single-column list; others use the grid. private var currentColumns: Int { - (currentBandIsClipboard || currentBandIsFiles) ? 1 : LauncherGridLayout.columns + currentBandIsClipboard ? 1 : LauncherGridLayout.columns } func setBands(_ bands: [[LaunchItem]], names: [String], colors: [ItemColor], - icons: [ItemIcon] = [], startBand: Int, column: Int, clipboardBandIndex: Int? = nil, - filesBandIndex: Int? = nil, filesColumn: FilesColumnController? = nil) { + icons: [ItemIcon] = [], startBand: Int, column: Int, clipboardBandIndex: Int? = nil) { self.bands = bands self.bandNames = names self.bandColors = colors self.bandIcons = icons self.clipboardBandIndex = clipboardBandIndex - self.filesBandIndex = filesBandIndex - self.filesColumn = filesColumn - // Wire the controller's async-listing callback so a late listing landing reprojects the band's - // items + republishes (the depth-change → remembered-location persistence is driven separately, - // inline from the horizontal drill). - bindFilesColumn(filesColumn) - self.canvasCommand = nil - self.filesPicker = nil - self.filesActionMenu = nil - self.filesOpenFailure = nil self.sessionPinToggles = [] self.currentBand = clamp(startBand, 0, max(bands.count - 1, 0)) // Multi-band lands on the band list at the home band (nothing armed); a single band has no @@ -214,183 +83,25 @@ final class LauncherModel: ObservableObject { disarm() } - // MARK: - AI preview canvas - - /// Enter the AI streaming preview canvas for `command` (fired from an armed AI item). The grid is - /// replaced by the canvas; the panel stays visible and never becomes key. - func enterCanvas(_ command: AICommand) { - canvasCommand = command - disarm() - } - - /// Leave the AI preview canvas (commit / discard). Restores the normal grid presentation. - func exitCanvas() { - canvasCommand = nil - } - /// Visual pin state for a clipboard item: the entry's stored pin XOR a session toggle. func isPinned(_ item: LaunchItem) -> Bool { guard case let .clipboardEntry(entry) = item.kind else { return false } return entry.pinned != sessionPinToggles.contains(item.id) } - // MARK: - Files band (single-column directory drill) - - /// Wire a Files-band controller's async-listing callback to this model: when a late listing lands the - /// controller fires `onColumnChanged`, which reprojects the band's items and republishes. (Remembered- - /// location persistence is driven separately, inline from the horizontal drill.) Idempotent — called - /// from `setBands` whenever the controller is (re)assigned. - private func bindFilesColumn(_ controller: FilesColumnController?) { - controller?.onColumnChanged = { [weak self] in - self?.reprojectFilesBand() - self?.onFilesProjectionChanged?() // async landing → recharge the dwell on the new row - } - } - - /// Reproject the Files band's items from the controller's current column (the navigator's - /// `visibleEntries`) via `FilesBandBuilder`, and — when the Files band is the active band — refresh the - /// live `items` + selection to the highlighted row. Called after every Files move and on an async - /// listing landing. A no-op when there is no Files band / controller. - private func reprojectFilesBand() { - guard let index = filesBandIndex, let controller = filesColumn, - bands.indices.contains(index) else { return } - let rebuilt = FilesBandBuilder.build(currentColumn: controller.visibleEntries).items - bands[index] = rebuilt - guard currentBand == index else { return } // only touch the live cursor if it's showing - items = rebuilt - // Keep the launcher selection locked to the navigator's highlight (the source of truth for the - // Files band) rather than the grid's own index math. - selectedIndex = clamp(controller.highlightedIndex, 0, max(items.count - 1, 0)) - } - - /// Drill horizontally in the Files band: a RIGHT step descends into the highlighted folder, a LEFT step - /// ascends one level (or backs out to the roots list). `dir > 0` is descend regardless of the user's - /// reverse-direction setting — that inversion is applied upstream (in the recognizer / coordinator) - /// before the step reaches this model, exactly as it is for grid and clipboard horizontal travel. After - /// the move the band is reprojected and any depth change persists the remembered location. - private func stepFilesHorizontal(_ dir: Int) { - guard let controller = filesColumn else { return } - let before = controller.current - if dir > 0 { - controller.descend() - } else { - controller.ascend() - } - reprojectFilesBand() - if controller.current != before { persistFilesRememberedLocations() } - } - - /// Move the Files-band highlight vertically: `dir > 0` (up) moves toward the top, `dir < 0` (down) moves - /// toward the bottom — matching the grid/clipboard vertical convention (the reverse-vertical setting is - /// already applied upstream). An up-step at the top of the column simply clamps (no overflow, no search — - /// the navigator is pure-trackpad). Reprojects the band so the view follows the new highlight. - private func stepFilesVertical(_ dir: Int) { - guard let controller = filesColumn else { return } - if dir > 0 { - controller.highlightUp() - } else { - controller.highlightDown() - } - reprojectFilesBand() - } - - /// Persist the controller's per-root remembered locations through the injected sink (one call per root - /// that has a remembered deepest location). Keyed/valued by standardized path — the sink writes them to - /// `AppSettings`. - private func persistFilesRememberedLocations() { - guard let controller = filesColumn, let sink = onFilesRememberLocation else { return } - for (root, location) in controller.rememberedLocations { - sink(location.standardizedFileURL.path, root.standardizedFileURL.path) - } - } - - // MARK: - Files band Open-With picker (the held +1-finger app list) - - /// Enter the Open-With picker with the file's relevant apps (`FileOpenService.openWithCandidates`), - /// landing the highlight on the **default** application (the one a plain open would launch) so the most - /// likely choice is pre-selected; a fresh gesture then scrubs it. A no-op when `candidates` is empty - /// (the coordinator surfaces a "no app" notice instead and never enters here). - func enterFilesPicker(_ entries: [OpenWithEntry]) { - guard !entries.isEmpty else { return } - // Land the highlight on the default external app (the one a plain open would launch) when present, so - // the most likely choice is pre-selected; otherwise the first row (a built-in entry). - let start = entries.firstIndex(where: \.isDefault) ?? 0 - filesPicker = FilesPickerState(candidates: entries, highlightedIndex: start) - } - - /// Move the picker highlight vertically: `dir > 0` (up) toward the top of the list, `dir < 0` (down) - /// toward the bottom — the same convention as the folder list's vertical highlight (the reverse-vertical - /// setting is already applied upstream). Clamps at both ends (the picker is a short bounded list). A - /// no-op when the picker isn't open. - func filesPickerMove(_ dir: Int) { - guard dir != 0, var picker = filesPicker else { return } - let next = clamp(picker.highlightedIndex - dir, 0, max(picker.candidates.count - 1, 0)) - guard next != picker.highlightedIndex else { return } - picker.highlightedIndex = next - filesPicker = picker - } - - /// The entry a lift in picker mode would choose (the highlighted row), or nil when the picker isn't open - /// / is (defensively) empty. The coordinator routes the chosen entry to its external app. - func filesPickerSelected() -> OpenWithEntry? { - filesPicker?.highlighted - } - - /// Leave the Open-With picker (a choice was made, or a discard backed out of it). Returns the navigator - /// to the folder list — the column stays open (the coordinator does not dismiss on a picker discard). - func exitFilesPicker() { - filesPicker = nil - } - - // MARK: - Files action menu (the +1-finger menu, budded over the navigator) - - /// Open the action menu over `entry` with the resolved `rows` (already built by `FilesActionMenu`). - /// Lands the highlight on the first row. A no-op for an empty menu (the coordinator never enters here). - func enterFilesActionMenu(entry: FileEntry, rows: [FilesMenuRow]) { - guard !rows.isEmpty else { return } - filesActionMenu = FilesActionMenuState(entry: entry, rows: rows, highlightedIndex: 0) - } - - /// Move the action-menu highlight vertically (same convention as the picker/folder list — reverse-vertical - /// already applied upstream). Clamps at both ends. A no-op when the menu isn't open. - func filesActionMenuMove(_ dir: Int) { - guard dir != 0, var menu = filesActionMenu else { return } - let next = clamp(menu.highlightedIndex - dir, 0, max(menu.rows.count - 1, 0)) - guard next != menu.highlightedIndex else { return } - menu.highlightedIndex = next - filesActionMenu = menu - } - - /// The row a lift in menu mode would commit (the highlighted row), or nil when the menu isn't open. - func filesActionMenuSelected() -> FilesMenuRow? { filesActionMenu?.highlighted } - - /// Leave the action menu (a row was committed, or a discard backed out of it). - func exitFilesActionMenu() { - filesActionMenu = nil - } - // MARK: - Navigation (pure) /// Horizontal step (`dir > 0` = right). On the band list (left), RIGHT crosses into the grid at /// its home/first item and LEFT clamps (nothing sits to the left of the list); in the grid, /// moves the cursor within the current row, and from column 0 a LEFT step crosses back to the /// band list (when there is more than one band). - /// - /// Files-band crossing (refinements 1 + 2): a RIGHT from the band rail (`.bands`) into the Files band - /// CROSSES focus to `.grid` **without descending** — it lands on the column the navigator is *already - /// displaying* (with restore-at-open, the last folder visited), at the TOP of that column. The drill - /// (descend / ascend) only happens once focus is `.grid` (the `.grid` branch below). So crossing in is a - /// pure focus change with no jump: the displayed state and the landing match. func stepHorizontal(_ dir: Int) { guard dir != 0 else { return } switch focus { case .bands: if dir > 0, !items.isEmpty { // right → enter the grid at the first item focus = .grid - // The Files band's selection follows the navigator's live highlight (its source of truth), - // which restore-at-open already sits at the TOP of the displayed column — so crossing in - // lands exactly there (no descend, no jump). Every other band lands on its first cell. - selectedIndex = currentBandIsFiles ? (filesColumn?.highlightedIndex ?? 0) : 0 + selectedIndex = 0 } // left → already at the leftmost pane; nothing case .grid: @@ -400,17 +111,6 @@ final class LauncherModel: ObservableObject { stepClipboardHorizontal(dir) return } - // The Files band repurposes horizontal as a directory drill: RIGHT descends into the - // highlighted folder, LEFT ascends one level — and a LEFT while already at the roots list - // (nothing left to ascend) crosses back to the band list, matching the grid's column-0 escape. - if currentBandIsFiles { - if dir < 0, filesColumn?.canAscend == false, bands.count > 1 { - focus = .bands // back to the band rail; the drill disengages (`filesDrillEngaged`) - } else { - stepFilesHorizontal(dir) - } - return - } guard !items.isEmpty else { return } let cols = currentColumns let col = selectedIndex % cols @@ -484,12 +184,6 @@ final class LauncherModel: ObservableObject { currentBand = target applyCurrentBand(column: 0) case .grid: - // The Files band's vertical axis moves the navigator's highlight (and overflows into a - // focus-search request at the top), not the grid's row math. - if currentBandIsFiles { - stepFilesVertical(dir) - return - } let cols = currentColumns let col = selectedIndex % cols let row = selectedIndex / cols @@ -522,14 +216,8 @@ final class LauncherModel: ObservableObject { func disarm() { arming = false; armed = false } private func applyCurrentBand(column: Int) { - // Switching bands abandons any open Open-With picker — it belongs to the Files band's column, so - // leaving (or re-entering) the band drops it rather than leaving a stale popup floating. - if !currentBandIsFiles { filesPicker = nil; filesActionMenu = nil } items = bands.indices.contains(currentBand) ? bands[currentBand] : [] - // The Files band's selection follows the navigator's live highlight (the source of truth for its - // column), so switching back to it lands on the row the user left — not column 0. - let landing = (currentBandIsFiles ? filesColumn?.highlightedIndex : nil) ?? column - selectedIndex = clamp(landing, 0, max(items.count - 1, 0)) + selectedIndex = clamp(column, 0, max(items.count - 1, 0)) resetClipboardHoriz() } diff --git a/Sources/ThreeFingerSwitcher/Overlay/LauncherOverlayController.swift b/Sources/ThreeFingerSwitcher/Overlay/LauncherOverlayController.swift index 29a7464..5f4e506 100644 --- a/Sources/ThreeFingerSwitcher/Overlay/LauncherOverlayController.swift +++ b/Sources/ThreeFingerSwitcher/Overlay/LauncherOverlayController.swift @@ -13,34 +13,6 @@ final class LauncherOverlayController { var onFire: ((LaunchItem, ContextBand) -> Void)? /// Called when a RIGHT step pins/unpins the selected clipboard entry. Wired to `ClipboardStore`. var onTogglePin: ((LaunchItem) -> Void)? - /// Called when the AI preview canvas's commit gesture (a four-finger DOWN swipe) resolves. Wired by - /// the coordinator to `AICommandExecutor.commit()`. Only invoked while the canvas is active + ready. - var onCommitCanvas: (() -> Void)? - /// Called when the AI preview canvas is discarded (a horizontal swipe / cancel gesture). Wired to - /// `AICommandExecutor.cancel()` so any in-flight generation stops and nothing is written. - var onDiscardCanvas: (() -> Void)? - /// Called when the canvas opens (`true`) / closes (`false`). The coordinator uses it to put the - /// gesture recognizer into canvas-resolution mode, so a fresh four-finger swipe resolves the canvas - /// (horizontal = discard, down = apply) instead of opening the launcher again. - var onCanvasStateChanged: ((Bool) -> Void)? - /// Called when the Files band becomes the active band (`true`) / stops being it (`false`) — mirrors - /// `onCanvasStateChanged`. The coordinator uses it to put the gesture recognizer into the sustained - /// Files-drill mode (`filesDrillActive`), so horizontal travel drills the directory tree and a - /// resolving lift opens / Open-Withs the highlighted entry instead of stepping the grid. Flipped on - /// every band change (show / vertical band switch / horizontal escape) and cleared on `hide`. - var onFilesColumnStateChanged: ((Bool) -> Void)? - /// The executor whose observable streaming state the preview canvas binds to. Injected by the - /// coordinator before `show`; the panel's `LauncherView` is built with it (the panel is recreated - /// fresh on every `show`, so a late-set executor is picked up on the next open). - var executor: AICommandExecutor? - /// Called when an armed **screen-region** (vision) AI command is lifted: the launcher has already been - /// dismissed (to reveal the desktop), and the coordinator now runs the region picker, captures the - /// designated region, and on capture re-opens the canvas via `showCanvas(for:)` while firing the - /// executor with the captured image. A cancelled pick opens no canvas (the front app is restored). - var onScreenRegionCommand: ((AICommand) -> Void)? - /// Enable/download wiring for the canvas's `.unavailable` state (configuration-hub). Set by the - /// coordinator before `show`; picked up when the panel's `LauncherView` is (re)built on each open. - var aiAvailability: AICanvasAvailability? private var panel: SwitcherPanel? private var bands: [ContextBand] = [] private var dwell: Double = 0.5 @@ -56,82 +28,32 @@ final class LauncherOverlayController { private var edgeTicks = 0 /// Acceleration sensitivity (≥1); higher accelerates faster. Set from settings on `show`. var edgeAcceleration: Double = 1.0 - /// Deliberate horizontal steps required before a clipboard **pin** (RIGHT) or the canvas discard-flick - /// fires. Set from settings on `show`; pushed into the model so pinning isn't twitchy. (Left-exit from - /// the clipboard band is a single quick step, not gated by this.) + /// Deliberate horizontal steps required before a clipboard **pin** (RIGHT) fires. Set from settings + /// on `show`; pushed into the model so pinning isn't twitchy. (Left-exit from the clipboard band is + /// a single quick step, not gated by this.) var clipboardPinSteps: Int = 2 // MARK: - Show / navigate func show(bands: [ContextBand], startBand: Int, startColumn: Int, dwell: Double, - clipboardBandIndex: Int? = nil, - filesBandIndex: Int? = nil, filesColumn: FilesColumnController? = nil) { + clipboardBandIndex: Int? = nil) { self.bands = bands self.dwell = dwell model.dwell = dwell model.onPinToggle = onTogglePin model.clipboardPinStepThreshold = clipboardPinSteps - // A late async directory listing can shift which Files row is highlighted (a freshly-descended, - // not-yet-cached folder fills in) — recharge the dwell on the new row so it never arms a stale/empty - // highlight. Identity-keyed, so a listing that preserves the highlight is a no-op. - model.onFilesProjectionChanged = { [weak self] in self?.filesManageDwell() } model.setBands(bands.map(\.items), names: bands.map(\.name), colors: bands.map(\.color), icons: bands.map(\.resolvedIcon), startBand: startBand, column: startColumn, - clipboardBandIndex: clipboardBandIndex, - filesBandIndex: filesBandIndex, - filesColumn: filesColumn) + clipboardBandIndex: clipboardBandIndex) let panel = self.panel ?? makePanel() self.panel = panel layout(panel) panel.orderFrontRegardless() manageDwell() - // Publish the initial drill state. It engages only once focus is in the file column - // (`filesDrillEngaged`): a SINGLE-band Files launcher lands on `.grid` (no band rail), so the drill - // arms up front and the first horizontal swipe drills; a MULTI-band launcher lands on the band rail - // (`.bands`), so the drill stays OFF until a horizontal step crosses in — a lift on the Files icon - // dismisses like any other band (refinement 1). - syncFilesDrillState() - } - - /// Open the AI preview canvas DIRECTLY for `command`, with the launcher otherwise dismissed — the - /// screen-region picker flow's re-entry. The launcher closed to reveal the desktop, the user dragged a - /// region, and the canvas now opens to stream the vision result. Creates a fresh panel straight in - /// canvas mode and wires the resolution gestures, mirroring the canvas side of `end()`'s AI path; the - /// executor is fired by the coordinator (which holds the captured image), so this only presents the - /// canvas. The panel is recreated fresh (as on every open), so it binds the current Space — never a - /// ghost on a later Space switch. - func showCanvas(for command: AICommand) { - endEdgeAutoScroll() - canvasDiscardAccum = 0 - let panel = self.panel ?? makePanel() - self.panel = panel - model.enterCanvas(command) - onCanvasStateChanged?(true) // → recognizer enters canvas-resolution mode (swipe to resolve) - layout(panel, animated: false) // canvas metrics - panel.orderFrontRegardless() - // The panel stays PASS-THROUGH (not key) until the executor has read the input — becoming key first - // would steal the selection read (see `makeCanvasInteractive` / `AICommandExecutor.onReadyForInteraction`). - } - - /// Re-publish whether the Files directory **drill** is engaged so the coordinator can flip the - /// recognizer's sustained Files-drill mode (`filesDrillActive`). The drill engages only once focus has - /// crossed INTO the file column (`model.filesDrillEngaged` = Files current AND `focus == .grid`), NOT - /// merely when the Files band is current: while the highlight rests on the Files band ICON (`.bands`) the - /// band behaves like any other — a horizontal step crosses in, a vertical step switches bands, and a - /// lift DISMISSES the launcher (refinement 1). So this is re-synced after every step (focus can cross - /// without a band change), not only on band switches. Idempotent for the coordinator (it just sets a - /// flag); tracks the last published value so a navigation that doesn't cross the engaged boundary doesn't - /// churn the recognizer. - private var filesDrillPublished = false - private func syncFilesDrillState() { - let active = model.filesDrillEngaged - guard active != filesDrillPublished else { return } - filesDrillPublished = active - onFilesColumnStateChanged?(active) } /// Whether the cursor is on the left band-title list (where *vertical* travel switches bands). Lets @@ -141,31 +63,16 @@ final class LauncherOverlayController { /// Horizontal swipe step: cross between the band list and the grid, else move the grid cursor /// within its row (band switching now lives on the *vertical* axis of the band list). - /// - /// While the AI preview canvas is open the recognizer is in canvas-resolution mode and emits no grid - /// steps (it routes swipes to `launcherCanvasResolve`), so this branch is a **defensive fallback**: - /// should a horizontal step still arrive, a deliberate excursion discards (accumulated so a small - /// jitter doesn't discard), matching the canvas's horizontal-swipe = discard semantics. Vertical - /// scrubbing is inert in the canvas (there's nothing to navigate). func stepHorizontal(_ dir: Int) { - if model.canvasActive { - accumulateCanvasDiscard(dir) - return - } let bandBefore = model.currentBand model.stepHorizontal(dir) if model.currentBand != bandBefore { if let panel { layout(panel, animated: true) } } - // Re-sync the drill state on EVERY step: a horizontal step can cross focus `.bands` → `.grid` - // (engaging the Files drill) WITHOUT changing the band, so this can't be gated on a band change - // (refinement 1). Idempotent — a no-op when the engaged state didn't change. - syncFilesDrillState() manageDwell() } /// Vertical swipe step: switch the active band on the band list, else move between grid rows /// (which now clamps at row 0 — there's no header strip to rise onto). func stepVertical(_ dir: Int) { - if model.canvasActive { return } // the canvas isn't grid-navigable // Band switching now lives on the vertical axis (the band list), so re-size the panel here on a // band change — the width shifts when crossing into / out of the Clipboard band, and the frame // animates via the same path band switching used on the horizontal axis before. Height is stable @@ -173,211 +80,44 @@ final class LauncherOverlayController { let bandBefore = model.currentBand model.stepVertical(dir) if model.currentBand != bandBefore { if let panel { layout(panel, animated: true) } } - // A vertical step on the band rail switches bands (possibly off Files), which changes the engaged - // state; re-sync unconditionally (idempotent) rather than only on a band change (refinement 1). - syncFilesDrillState() manageDwell() } - // MARK: - Files band (the recognizer's sustained Files-drill routes here) - - /// One depth step while the Files band is current (the recognizer's `filesDepth`): RIGHT (`dir > 0`) - /// descends into the highlighted folder, LEFT ascends (or, at the roots list, escapes to the band - /// list). The direction is already reverse-adjusted upstream, exactly like grid/clipboard horizontal - /// travel; the model routes it to the directory drill because the Files band is current. A descend/ascend - /// lands on a new row, which **recharges the dwell-to-arm** (the coordinator calls `filesManageDwell` after - /// the move — add-files-band-dwell-arm): the Files lift (`filesOpen` / `filesOpenWith`) fires only when the - /// highlighted row has armed, mirroring the launcher. A descend / ascend never changes the active band - /// (still the Files band), so the panel size is stable across drilling — no re-fit needed. - /// - /// The one focus change here is the LEFT-at-roots ESCAPE: it crosses the cursor to the band rail - /// (`focus == .bands`), which **disengages** the drill (`filesDrillEngaged` flips false). Re-sync so the - /// recognizer leaves `filesDrillActive` the instant we land on the band icon — a lift there must then - /// DISMISS the launcher, not open the highlighted entry (refinement 1). - func filesDepth(_ dir: Int) { - guard model.currentBandIsFiles else { return } - model.stepHorizontal(dir) - syncFilesDrillState() - } - - /// One highlight step while the Files band is current (the recognizer's `filesHighlight`): UP - /// (`dir > 0`) moves toward the top, DOWN toward the bottom (reverse-adjusted upstream). An up-step at - /// the top of the column simply clamps (no search to overflow into). The new row recharges the - /// dwell-to-arm (see `filesDepth`; the coordinator calls `filesManageDwell` after the move). - /// The fixed container height (refinement 3) is stable across rows — the list scrolls inside to follow the - /// highlight — so no panel re-fit is needed. The drill is re-synced anyway (idempotent), matching the - /// other step paths. - func filesHighlight(_ dir: Int) { - guard model.currentBandIsFiles else { return } - model.stepVertical(dir) - syncFilesDrillState() - } - - /// The current Files highlight the recognizer's resolving lift acts on (open / Open-With), read off the - /// navigator. Nil when the Files band isn't current or the column is empty (the coordinator then has - /// nothing to open). - var filesHighlightedEntry: FileEntry? { - guard model.currentBandIsFiles else { return nil } - return model.filesColumn?.highlightedEntry - } - - /// Deliberate horizontal travel within the canvas accumulates toward one discard (mirrors the - /// Clipboard band's deliberate-excursion threshold so a tiny wobble never discards). Fallback path — - /// the primary discard is the recognizer's horizontal resolution swipe (`discardCanvas`). - private var canvasDiscardAccum = 0 - private func accumulateCanvasDiscard(_ dir: Int) { - guard dir != 0 else { return } - // Reset the accumulator if the direction reverses, so back-and-forth never silently sums. - if (canvasDiscardAccum > 0) != (dir > 0) { canvasDiscardAccum = 0 } - canvasDiscardAccum += dir - if abs(canvasDiscardAccum) >= max(1, clipboardPinSteps) { - canvasDiscardAccum = 0 - discardCanvas() - } - } - /// Lift: fire the armed item, else dismiss. Returns whether something fired. /// - /// Three cases: - /// 1. The AI preview canvas is already open → a lift is a **no-op** (returns true). The fingers are - /// already up from the firing lift, so the canvas is resolved by a FRESH four-finger swipe instead: - /// a DOWN swipe commits (`resolveCanvasCommit`), a horizontal swipe discards (`discardCanvas`). A - /// stray re-lift must never lose the result, so it does nothing here. - /// 2. An armed **AI command** item is lifted → **open the canvas** and begin the command WITHOUT - /// dismissing (the exception to the order-out-before-fire rule). The overlay stays visible and - /// non-activating; a fresh DOWN swipe then commits, a horizontal swipe discards. - /// 3. Any other armed item → the existing path: order the panel out BEFORE firing (so a - /// Space-switching action doesn't drag the panel onto the destination Space), then fire. + /// The panel is ordered out BEFORE firing (so a Space-switching action doesn't drag the panel onto + /// the destination Space), then the armed item fires. @discardableResult func end() -> Bool { dwellDriver.cancel() - // Case 1: a lift while the canvas is open is a NO-OP. After the firing lift (case 2) the fingers - // are already up, so the canvas is resolved by a FRESH four-finger swipe — horizontal = discard, - // DOWN = apply (`launcherCanvasResolve` → `discardCanvas` / `resolveCanvasCommit`) — not by - // lifting. Leaving the canvas up lets the user swipe to decide (a stray lift never loses it). - if model.canvasActive { return true } - - // Files DRILL: a lift is owned by the recognizer's Files-drill resolution (`filesOpen` / - // `filesOpenWith` / `filesDiscard`), which holds the defusable open with the panel STILL VISIBLE - // (mirroring the AI canvas's order-out exception, design D7). Should a lift reach here while the drill - // is engaged (focus IN the column), do NOT hide-before-fire — that would tear the navigator down - // mid-resolution and reset the drill. Leave the panel up; the drill path resolves it. - // - // Gate on `filesDrillEngaged` (Files current AND focus `.grid`), NOT on `currentBandIsFiles`: while the - // highlight rests on the Files band ICON (`.bands`) the drill is OFF, the recognizer routes the lift - // here through the normal launcher end path, and the band must behave like any other — a lift DISMISSES - // (refinement 1). The band icon never arms (`manageDwell` disarms on `.bands`), so falling through - // hits the unarmed `hide(); return false` below: a clean dismiss. - if model.filesDrillEngaged { return true } - guard model.armed, bands.indices.contains(model.currentBand), bands[model.currentBand].items.indices.contains(model.selectedIndex) else { hide(); return false } let band = bands[model.currentBand] let item = band.items[model.selectedIndex] - // Case 2: an AI command opens the streaming preview canvas instead of completing on lift. The - // panel is NOT ordered out (the canvas needs it visible); `onFire` routes to the executor. - if case let .aiCommand(command) = item.kind { - endEdgeAutoScroll() - canvasDiscardAccum = 0 - // Case 2a: a SCREEN-REGION (vision) command is a further exception WITHIN the AI-command - // exception (launcher-overlay spec): it must reveal the desktop so the user can drag a region, - // so it DISMISSES the launcher first (like a completing action) and hands off to the region- - // picker orchestration. That captures the designated region and re-opens the canvas via - // `showCanvas(for:)`; a cancelled pick opens no canvas. `onFire` is NOT called here — the - // coordinator fires the executor with the captured image. - if command.inputs.contains(.screenRegion) { - hide() // synchronous dismiss → reveal the desktop (ghost-safe) - onScreenRegionCommand?(command) - return true - } - model.enterCanvas(command) - onCanvasStateChanged?(true) // → recognizer enters canvas-resolution mode (swipe to resolve) - if let panel { layout(panel, animated: true) } // grow to the canvas metrics - onFire?(item, band) // fires the executor synchronously → sets its state (.unavailable / .loadingModel) - // The canvas becomes key-interactive (scroll / language pill / Thinking disclosure / the - // unavailable Enable+Download controls) only AFTER the executor has read its input — the panel - // stays PASS-THROUGH until `AICommandExecutor.onReadyForInteraction` calls back - // `makeCanvasInteractive()`. Becoming key at fire (a `.nonactivatingPanel` makeKey) would steal - // the front app's key focus BEFORE the async selection read runs, so the read comes back empty - // and the input wrongly falls through to the clipboard — the exact bug this defers past. - // The four-finger swipe-to-resolve rides the MULTITOUCH device (not window events), so commit - // (down) / discard (horizontal) keep working while the panel is still pass-through. - return true - } - - // Case 3: every other kind keeps the order-out-before-fire rule exactly as before. - // Dismiss the panel BEFORE firing. The panel is `.canJoinAllSpaces`, so a Space-switching - // action (Next/Previous Space) fired first would carry the still-visible panel onto the - // destination Space; ordering it out first lets the WindowServer drop it before the switch. + // Dismiss the panel BEFORE firing. The panel would otherwise be carried onto the destination + // Space by a Space-switching action (Next/Previous Space); ordering it out first lets the + // WindowServer drop it before the switch. hide() onFire?(item, band) return true } - /// Discard the open AI preview canvas (a horizontal swipe / cancel gesture): cancel any in-flight - /// generation, write nothing, and dismiss the overlay. A no-op when the canvas isn't open. - func discardCanvas() { - guard model.canvasActive else { return } - onDiscardCanvas?() - hide() - } - - /// Apply the AI preview canvas's result (a four-finger DOWN swipe — "bring it into the document"): - /// route the ready in-place result / confirmed task per the command's output target, then dismiss. - /// Only commits when the executor is in a committable state — a down swipe while still loading or - /// streaming is IGNORED (the user waits; only a horizontal swipe discards mid-flight). No-op when the - /// canvas isn't open. - func resolveCanvasCommit() { - guard model.canvasActive else { return } - // A one-shot fire commits only from a committable state (`.ready`/`.reviewingAction`); a down - // swipe while still loading/streaming is ignored (the user waits; a horizontal swipe discards). - if let executor { - guard executor.state.isCommittable else { return } - } - onCommitCanvas?() - hide() - } - func cancel() { dwellDriver.cancel() - // A hard cancel (gesture abandoned, disable, sleep) while the canvas is open must also stop any - // in-flight generation and write nothing — same as a discard swipe. - if model.canvasActive { onDiscardCanvas?() } hide() } func hide() { dwellDriver.cancel() endEdgeAutoScroll() - canvasDiscardAccum = 0 - let wasCanvas = model.canvasActive - model.exitCanvas() model.disarm() - if wasCanvas { onCanvasStateChanged?(false) } // → recognizer leaves canvas-resolution mode - // Leave the Files-drill mode too: the overlay is going away, so the recognizer must return to the - // normal launcher latch (a fresh four-finger swipe re-opens the launcher, not a stale drill). - if filesDrillPublished { filesDrillPublished = false; onFilesColumnStateChanged?(false) } - // TODO(files-band, tasks 11.2/11.3 — recede-on-leave exit): the Files navigator currently closes - // hard (the snap below) rather than receding its content out. Animating the BubbleMorph out before - // the panel goes away is intentionally NOT done here, because it would conflict with two invariants: - // (1) the ghost-panel teardown below MUST be synchronous — deferring `orderOut`/`close` behind an - // exit animation re-opens the verified ghost-on-Space-switch bug (a Next-Space ⌃→ processes - // before a delayed close flushes), the regression this method is explicitly guarding; and - // (2) it needs a handshake that does not exist yet — a published `closing` flag on `LauncherModel` - // for `FilesBandView` to drive its BubbleMorph exit off, plus a completion callback to tear the - // panel down only after the morph settles. `FilesBandView` is authored in the parallel View - // change, so there is nothing to coordinate the exit with today. - // When that seam lands, route a *resolving lift* (which already keeps the band visible per Stage 2) - // and a recede-on-leave through that flag and tear down in the completion, leaving the synchronous - // close below as the fallback for the ghost-critical Space-switch paths. - // Destroy the panel, don't just orderOut. The panel is `.canJoinAllSpaces`, and an orderOut'd - // all-Spaces panel leaves a rendered GHOST on the Space you switch to (verified: a Space-switch - // action left the launcher visible on the destination even though isVisible was already false). - // Closing the window removes it from the WindowServer entirely; `show()` recreates it fresh on - // the current Space. + // Destroy the panel, don't just orderOut. An orderOut'd panel can leave a rendered GHOST on the + // Space you switch to (verified: a Space-switch action left the launcher visible on the + // destination even though isVisible was already false). Closing the window removes it from the + // WindowServer entirely; `show()` recreates it fresh on the current Space. panel?.orderOut(nil) panel?.close() panel = nil @@ -386,10 +126,6 @@ final class LauncherOverlayController { var isVisible: Bool { panel?.isVisible ?? false } var currentBand: Int { model.currentBand } var bandCount: Int { model.bandCount } - /// Whether the AI streaming preview canvas is currently open (the overlay is mid-AI-command). The - /// coordinator reads this to gate a fresh launcher gesture so it commits/discards rather than - /// re-showing the grid from scratch. - var canvasActive: Bool { model.canvasActive } // MARK: - Edge-triggered auto-repeat @@ -403,16 +139,12 @@ final class LauncherOverlayController { /// Auto-repeat stepping while a contact is held at a trackpad edge. `dx`/`dy` are −1 / 0 / +1 per /// axis (vertical switches bands on the band list / steps grid rows; horizontal crosses between the - /// band list and the grid, then steps items). Horizontal is suppressed only in the Clipboard band (there - /// horizontal is the deliberate pin / cross-to-band-list action). The Files band KEEPS horizontal - /// auto-repeat: holding at the edge auto-drills the directory tree (descend/ascend), exactly like the - /// launcher's item axis — the user opted into this. Vertical auto-repeat is kept everywhere. When both - /// axes are zero the timer stops. A step that doesn't move the selection (clamped at an end) does NOT - /// reset the dwell, so holding at a dead edge still lets the current item arm and fire. + /// band list and the grid, then steps items). Horizontal is suppressed in the Clipboard band (there + /// horizontal is the deliberate pin / cross-to-band-list action). Vertical auto-repeat is kept + /// everywhere. When both axes are zero the timer stops. A step that doesn't move the selection + /// (clamped at an end) does NOT reset the dwell, so holding at a dead edge still lets the current + /// item arm and fire. func setEdgeAutoScroll(dx: Int, dy: Int) { - // No grid auto-scroll while the canvas is open (it isn't grid-navigable); horizontal is the - // discard swipe, handled by the recognizer's canvas resolution, not the edge timer. - if model.canvasActive { endEdgeAutoScroll(); return } let hx = model.currentBandIsClipboard ? 0 : dx guard hx != edgeDX || dy != edgeDY else { return } edgeDX = hx @@ -445,9 +177,6 @@ final class LauncherOverlayController { let moved = model.currentBand != beforeBand || model.selectedIndex != beforeIndex || model.focus != beforeFocus if moved { if model.currentBand != beforeBand { if let panel { layout(panel, animated: true) } } - // Re-sync the drill on a band OR focus change (auto-repeat can cross focus `.bands` ⇄ `.grid` - // just like a manual step) — idempotent, so harmless when nothing relevant changed (refinement 1). - if model.currentBand != beforeBand || model.focus != beforeFocus { syncFilesDrillState() } manageDwell() // a real move resets the dwell (so auto-repeat never arms mid-scroll) } edgeTicks += 1 @@ -461,18 +190,7 @@ final class LauncherOverlayController { /// After any navigation step: arm the selected app (restart dwell) when the cursor is on a grid /// item; disarm when it's on the band list (band titles don't fire). - /// - /// The Files drill (and its sub-columns) share the SAME dwell timer + arm state but key the restart on - /// the highlighted *thing* changing rather than the grid index (`filesManageDwell`), so route there - /// while a Files surface is engaged. The cross-IN (`stepHorizontal` flips `focus` to `.grid` on the - /// Files band) and the edge-tick both reach here, so the landing row and the auto-drill reset funnel - /// through one mechanism with the drill's own moves. private func manageDwell() { - if model.filesDrillEngaged || model.filesPicker != nil || model.filesActionMenu != nil { - filesManageDwell() - return - } - filesArming.reset() // left every Files surface → a later re-entry charges from scratch dwellDriver.cancel() if model.focus == .grid, model.selectedItem != nil { startDwell() @@ -498,74 +216,8 @@ final class LauncherOverlayController { DwellArmDriver.hapticTick() } - // MARK: - Dwell-to-arm (Files drill + sub-columns) - - /// The identity-keyed restart decision (pure, `FilesDwellArming`): the controller feeds it the highlighted - /// Files thing's identity and acts on `.restart` / `.keep` / `.disarm`. - private var filesArming = FilesDwellArming() - - /// Files analogue of `manageDwell()`: (re)charge the shared dwell-to-arm whenever the highlighted Files - /// thing changes — a folder row, or an open sub-column's row (action menu / Open-With grid). Resting past - /// `dwell` then arms (haptic + the same charge pill the launcher uses). Identity-keyed (via - /// `FilesDwellArming`), so it is safe to call after every Files move and on async re-list landings: it does - /// NOTHING while the same thing stays highlighted, which is exactly why the `+1`-finger morph (it moves no - /// highlight, the recognizer emits no step) preserves the arm. An empty column disarms. The lift gate - /// (`AppCoordinator.filesOpen` / `filesOpenWith`) reads `model.armed`, mirroring the launcher's `end()`. - func filesManageDwell() { - let inFilesSurface = model.filesDrillEngaged || model.filesPicker != nil || model.filesActionMenu != nil - guard inFilesSurface else { dwellDriver.cancel(); model.disarm(); filesArming.reset(); return } - switch filesArming.update(identity: filesArmIdentityNow()) { - case .keep: - break // same thing → let the running dwell (or settled arm) continue - case .disarm: - dwellDriver.cancel(); model.disarm() - case .restart: - dwellDriver.cancel() - model.beginArming() - dwellDriver.charge(after: dwell) { [weak self] in self?.arm() } - } - } - - /// Force a fresh Files dwell even if the highlighted thing is unchanged — for the re-arm seams where the - /// drill is restarted in place (a delivery that failed and kept the navigator open). A bare - /// `filesManageDwell` would `.keep` there (same identity); resetting guarantees the user must re-dwell - /// before another committing lift fires (task 2.3). - func filesRearmDwell() { filesArming.reset(); filesManageDwell() } - - private func filesArmIdentityNow() -> String? { - if let menu = model.filesActionMenu { return "menu:\(menu.highlightedIndex)" } - if let picker = model.filesPicker { return "picker:\(picker.highlightedIndex)" } - return model.filesColumn?.highlightedEntry?.id - } - // MARK: - Panel - /// Make the panel clickable + key-capable while the canvas shows a control that needs the mouse — - /// the AI "unavailable" Enable / Download / model-picker controls, OR the in-canvas language - /// dropdown shown for a translate-style command (a runtime-parameter command). Otherwise the panel - /// stays pass-through (gesture-only). Reset implicitly on `hide()` (the panel is destroyed and - /// recreated per open). - /// - /// This is SAFE for the captured front app's focus: the panel is a `.nonactivatingPanel`, so - /// becoming key here never activates *this* app; the four-finger swipe-to-resolve is recognized off - /// the multitouch device (not window mouse events), so it keeps working while the panel is - /// interactive; and write-back re-activates the captured app on commit (`SelectionService` calls - /// `app.activate` + PID-targeted AX / key events), so the destination's focus is restored. - private func setCanvasInteractive(_ on: Bool) { - guard let panel else { return } - panel.ignoresMouseEvents = !on - panel.keyInteractive = on - if on { panel.makeKeyAndOrderFront(nil) } - } - - /// Make the canvas key-interactive — called by the coordinator from `AICommandExecutor.onReadyForInteraction`, - /// i.e. AFTER the executor has read the fire's input, so taking key focus no longer steals the selection - /// read. Guarded on the canvas actually being open (the hook may arrive a beat after a fast discard). - func makeCanvasInteractive() { - guard model.canvasActive else { return } - setCanvasInteractive(true) - } - private func makePanel() -> SwitcherPanel { let panel = SwitcherPanel( contentRect: NSRect(x: 0, y: 0, width: 800, height: SwitcherLayout.panelHeight), @@ -586,8 +238,7 @@ final class LauncherOverlayController { panel.collectionBehavior = [.fullScreenAuxiliary, .ignoresCycle] panel.isFloatingPanel = true panel.hidesOnDeactivate = false - panel.contentView = NSHostingView(rootView: LauncherView(model: model, executor: executor, - availability: aiAvailability)) + panel.contentView = NSHostingView(rootView: LauncherView(model: model)) return panel } @@ -598,58 +249,30 @@ final class LauncherOverlayController { // Master-detail shell: a left band-title list (only when there's more than one band) plus the // right content pane (the icon grid, or the Clipboard band's larger metrics). Width is the band // column + the content width; height is the taller of the two panes clamped to the window's - // min/max bounds, so switching bands never jitters the frame (each pane scrolls inside). The AI - // preview canvas is a full-surface replacement and keeps its own (larger) metrics. Both axes are - // clamped to the screen. - let width: CGFloat - let height: CGFloat - if model.canvasActive { - // The canvas owns the whole surface (no band list) — unchanged sizing path. - width = min(ClipboardBandLayout.containerWidth, frame.width - SwitcherLayout.sideMargin * 2) - height = min(ClipboardBandLayout.containerHeight, frame.height - 100) + // min/max bounds, so switching bands never jitters the frame (each pane scrolls inside). Both + // axes are clamped to the screen. + let contentWidth: CGFloat = model.currentBandIsClipboard + ? ClipboardBandLayout.containerWidth + : LauncherGridLayout.containerWidth + // The left band column (only when there's a choice of bands) is a fixed-width icon strip. + let bandColumn = model.bandCount > 1 ? LauncherGridLayout.bandColumnWidth : 0 + let width = min(bandColumn + contentWidth, frame.width - SwitcherLayout.sideMargin * 2) + + // Height is the LARGER of the active band's content demand — the icon grid's item rows or + // the Clipboard band's taller detail pane — and the band-icon list's demand, clamped to the + // window's min/max. The list term matters with many bands: its icons are fixed-size, so + // sizing from the content alone re-stretches the container a beat after each band switch + // (fit the rows, then grow for the list — the mid-move jitter). One max() computation sizes + // the frame once per switch, stable across short bands. + let bandListWanted = LauncherGridLayout.bandListHeight(bandCount: model.bandCount) + let wanted: CGFloat + if model.currentBandIsClipboard { + wanted = min(max(max(ClipboardBandLayout.containerHeight, bandListWanted), LauncherGridLayout.minHeight), + LauncherGridLayout.maxHeight) } else { - // Content pane width: the Clipboard band's master-detail width, the Files band's bounded - // column-navigator width (the FIXED Clipboard-sized container — refinement 3, constant at any - // depth and any column setting), else the icon-grid width. - let contentWidth: CGFloat - if model.currentBandIsClipboard { - contentWidth = ClipboardBandLayout.containerWidth - } else if model.currentBandIsFiles { - // A FIXED container equal to the Clipboard band's exact width — it never resizes on crossing - // in or changing depth; the current-folder list scrolls inside instead (refinement 3). - contentWidth = FilesBandLayout.containerWidth - } else { - contentWidth = LauncherGridLayout.containerWidth - } - // The left band column (only when there's a choice of bands) is a fixed-width icon strip. - let bandColumn = model.bandCount > 1 ? LauncherGridLayout.bandColumnWidth : 0 - width = min(bandColumn + contentWidth, frame.width - SwitcherLayout.sideMargin * 2) - - // Height is the LARGER of the active band's content demand — the icon grid's item rows, - // the Clipboard band's taller detail pane, or the Files navigator's row-count height — and - // the band-icon list's demand, clamped to the window's min/max. The list term matters with - // many bands: its icons are fixed-size, so sizing from the content alone re-stretches the - // container a beat after each band switch (fit the rows, then grow for the list — the - // mid-move jitter). One max() computation sizes the frame once per switch, stable across - // short bands. - let bandListWanted = LauncherGridLayout.bandListHeight(bandCount: model.bandCount) - let wanted: CGFloat - if model.currentBandIsClipboard { - wanted = min(max(max(ClipboardBandLayout.containerHeight, bandListWanted), LauncherGridLayout.minHeight), - LauncherGridLayout.maxHeight) - } else if model.currentBandIsFiles { - // A FIXED container height equal to the Clipboard band's exact height — no per-density and - // no per-depth variation (refinement 3). A folder taller than the fixed row area scrolls - // inside (the view follows the highlight); the frame never grows for it. Still take the max - // with the band-icon list so a tall band rail isn't clipped, clamped to the window bounds. - let filesWanted = FilesBandLayout.containerHeight - wanted = min(max(max(filesWanted, bandListWanted), LauncherGridLayout.minHeight), - LauncherGridLayout.maxHeight) - } else { - wanted = LauncherGridLayout.windowHeight(itemCount: model.items.count, bandCount: model.bandCount) - } - height = min(wanted, frame.height - 100) + wanted = LauncherGridLayout.windowHeight(itemCount: model.items.count, bandCount: model.bandCount) } + let height = min(wanted, frame.height - 100) // Round to whole points so the window never gets a fractional frame the WindowServer would snap // to a pixel boundary (the `* 0.62` anchor otherwise yields a fractional y). diff --git a/Sources/ThreeFingerSwitcher/Overlay/LauncherView.swift b/Sources/ThreeFingerSwitcher/Overlay/LauncherView.swift index 80fc2ea..7af600f 100644 --- a/Sources/ThreeFingerSwitcher/Overlay/LauncherView.swift +++ b/Sources/ThreeFingerSwitcher/Overlay/LauncherView.swift @@ -16,11 +16,6 @@ extension Color { /// dwell, then arms (haptic). struct LauncherView: View { @ObservedObject var model: LauncherModel - /// The AI command executor whose streaming state the preview canvas observes (nil when AI commands - /// aren't wired — the canvas is then never reached because no `.aiCommand` item can be fired). - var executor: AICommandExecutor? = nil - /// Enable/download wiring for the canvas's `.unavailable` state (configuration-hub). - var availability: AICanvasAvailability? = nil private var columns: [GridItem] { Array(repeating: GridItem(.fixed(LauncherGridLayout.cellWidth), spacing: LauncherGridLayout.spacing), @@ -28,29 +23,17 @@ struct LauncherView: View { } var body: some View { - Group { - // The AI streaming preview canvas replaces the whole surface while it is open (an AI command - // was fired and is generating / awaiting commit) — no band list alongside it. Everything - // else is the master-detail shell: the band list on the left, the content on the right. - if model.canvasActive { - canvas + HStack(spacing: 0) { + // The left band-title list only exists when there's more than one band to choose + // between; a single band shows just its content (lands on `.grid`, item 0). + if model.bandCount > 1 { + bandList + } + // The right pane: the Clipboard band's master-detail, else the icon grid. + if model.currentBandIsClipboard { + ClipboardBandView(model: model) } else { - HStack(spacing: 0) { - // The left band-title list only exists when there's more than one band to choose - // between; a single band shows just its content (lands on `.grid`, item 0). - if model.bandCount > 1 { - bandList - } - // The right pane, in band-priority order: the Clipboard band's master-detail, then - // the Files band's column navigator, else the icon grid (the default fallback). - if model.currentBandIsClipboard { - ClipboardBandView(model: model) - } else if model.currentBandIsFiles { - FilesBandView(model: model) - } else { - grid - } - } + grid } } .padding(LauncherGridLayout.containerPadding) @@ -60,19 +43,6 @@ struct LauncherView: View { ) } - /// The AI preview canvas, bound to the executor's streaming state. When no executor is wired it - /// falls back to an empty surface (defensive — the canvas is unreachable without one). - @ViewBuilder - private var canvas: some View { - if let executor, let command = model.canvasCommand { - AICommandCanvasView(executor: executor, command: command, - tint: command.tint.map(Color.init) ?? Color(model.currentBandColor), - availability: availability) - } else { - Color.clear - } - } - // MARK: Band icon list (the left column) /// The vertical list of band **icons** on the left, at a fixed small spacing and centered in the @@ -218,7 +188,6 @@ struct LauncherView: View { switch item.kind { case .preset: return "square.stack.3d.up.fill" case .script: return "terminal.fill" - case .aiCommand: return "sparkles" case .claudeProject, .claudeProjectPrompt: return "terminal.fill" case .terminalCommand, .terminalCommandPrompt: return "terminal.fill" default: return nil diff --git a/Sources/ThreeFingerSwitcher/Overlay/MediaCanvasPlayerOverlay.swift b/Sources/ThreeFingerSwitcher/Overlay/MediaCanvasPlayerOverlay.swift deleted file mode 100644 index 53b62b0..0000000 --- a/Sources/ThreeFingerSwitcher/Overlay/MediaCanvasPlayerOverlay.swift +++ /dev/null @@ -1,156 +0,0 @@ -import AppKit -import AVKit -import SwiftUI - -// FLAGGED: user xcodebuild + stable-signed build. -// -// Output #2's NATIVE surface (design D7, §6.2): the canvas media preview/player. It renders the pure -// `MediaJobState` (Core, `swift test`-verified) and is resolved by the pure `MediaCanvasResolver` (Core). -// This file is the AppKit/SwiftUI shell that displays it. It compiles under `swift build` (the Overlay -// layer lives in `ThreeFingerSwitcherCore`, which may import AppKit/SwiftUI — it just stays MLX-free), but -// its REAL behavior — a `.nonactivatingPanel` that never becomes key/main, a synchronous `orderOut` -// teardown, the `BubbleMorph` bud-in, and AVKit video playback — is only verifiable in the user's -// stable-signed build (an agent never builds/signs the `.app`). -// -// Pattern: the `DockPreviewOverlay` species (the project's non-activating, synchronous-teardown overlay). -// It NEVER becomes key/main (no keyboard, never steals focus — the foreground app stays the extract -// target), and teardown is synchronous `orderOut` (the files-band / dock ghost-on-Space-switch landmine -// applies here too). - -@MainActor -final class MediaCanvasPlayerController { - /// The live job state the view renders (generating preview → finished image / video player). - let model = MediaCanvasPlayerModel() - private var panel: SwitcherPanel? - - /// The user resolved the finished preview by the compass: extract (DOWN-at-top) or discard (RIGHT). - /// The owner turns `.extract` into the chosen `MediaExtractIntent` side effect; `.discard` just - /// dismisses (the asset is already durable in the gallery, so the file is never lost). - var onResolve: ((MediaCanvasResolution) -> Void)? - - private func makePanel() -> SwitcherPanel { - let panel = SwitcherPanel( - contentRect: NSRect(x: 0, y: 0, width: 520, height: 520), - styleMask: [.borderless, .nonactivatingPanel], - backing: .buffered, - defer: false - ) - panel.isOpaque = false - panel.backgroundColor = .clear - panel.hasShadow = true - // Read-only preview: it never takes the pointer (unlike the Dock popup) and never becomes key/main. - panel.ignoresMouseEvents = true - panel.isFloatingPanel = true - panel.hidesOnDeactivate = false - panel.level = .popUpMenu - panel.collectionBehavior = [.canJoinAllSpaces, .fullScreenAuxiliary, .ignoresCycle] - panel.contentView = NSHostingView(rootView: MediaCanvasPlayerView(model: model)) - return panel - } - - var isVisible: Bool { panel?.isVisible ?? false } - - /// Show the preview at `rect` (the canvas region). Buds in with `BubbleMorph` via the view's transition. - func show(at rect: CGRect) { - let panel = self.panel ?? makePanel() - self.panel = panel - panel.setFrame(rect, display: true) - panel.orderFrontRegardless() - } - - /// Synchronous teardown (no deferred close — Space-switch ghost landmine). The asset stays in the - /// gallery; this only dismisses the preview. - func hide() { - panel?.orderOut(nil) - } -} - -/// The observable bridge from the pure `MediaJobState` to the SwiftUI view (FLAGGED). The Core sink's -/// `MediaJobObserving` callbacks drive `state`; the view re-renders. Conforms to `MediaJobObserving` so the -/// sink can target it directly in the live app. -@MainActor -final class MediaCanvasPlayerModel: ObservableObject { - @Published var state: MediaJobState = .idle - @Published var busyHeadline: String? - - func apply(_ progress: MediaProgress) { state.advance(progress) } - func finish(_ asset: MediaAsset) { state = .finished(asset) } - func fail(headline: String) { state = .failed(headline: headline) } - func cancel() { state = .cancelled } -} - -/// The preview/player view (FLAGGED). Generating → the latest intermediate preview frame + a step caption; -/// finished image → the image; finished video → an `AVPlayer` view; failed → a bounded, non-blocking error -/// card (clean headline + opt-in copyable details, NEVER an `NSAlert`). Buds in with `BubbleMorph`. -struct MediaCanvasPlayerView: View { - @ObservedObject var model: MediaCanvasPlayerModel - - var body: some View { - content - .frame(maxWidth: .infinity, maxHeight: .infinity) - .background(.ultraThinMaterial, in: RoundedRectangle(cornerRadius: 20)) - .transition(.bubbleMorph()) - } - - @ViewBuilder private var content: some View { - switch model.state { - case .idle: - Color.clear - case let .generating(index, total, preview): - VStack(spacing: 10) { - previewImage(preview) - Text("Painting… step \(index + 1)/\(max(total, index + 1))") - .font(.callout).foregroundStyle(.secondary) - if let busy = model.busyHeadline { - Text(busy).font(.caption).foregroundStyle(.secondary) - .multilineTextAlignment(.center) - } - } - .padding() - case let .finished(asset): - finishedView(asset) - case let .failed(headline): - errorCard(headline) - case .cancelled: - Color.clear // a discard just dismisses; no failed indicator (design D10) - } - } - - @ViewBuilder private func previewImage(_ data: Data?) -> some View { - if let data, let img = NSImage(data: data) { - Image(nsImage: img).resizable().scaledToFit().clipShape(RoundedRectangle(cornerRadius: 12)) - } else { - RoundedRectangle(cornerRadius: 12).fill(.quaternary).frame(minHeight: 160) - } - } - - @ViewBuilder private func finishedView(_ asset: MediaAsset) -> some View { - switch asset.kind { - case .image: - if let img = NSImage(contentsOf: asset.url) { - Image(nsImage: img).resizable().scaledToFit().clipShape(RoundedRectangle(cornerRadius: 12)) - .padding() - } else { - errorCard("The generated image couldn't be loaded.") - } - case .video: - // A finished clip plays from its FILE (no continuous screen-record / SCStream — design D7). - VideoPlayer(player: AVPlayer(url: asset.url)) - .clipShape(RoundedRectangle(cornerRadius: 12)) - .padding() - } - } - - /// Bounded, non-blocking error card — clean headline, capped, never an `NSAlert`. - @ViewBuilder private func errorCard(_ headline: String) -> some View { - VStack(spacing: 8) { - Image(systemName: "exclamationmark.triangle").foregroundStyle(.secondary) - Text(headline) - .font(.callout) - .multilineTextAlignment(.center) - .lineLimit(3) - .truncationMode(.middle) - } - .padding() - } -} diff --git a/Sources/ThreeFingerSwitcher/Overlay/NotchHomeZoneController.swift b/Sources/ThreeFingerSwitcher/Overlay/NotchHomeZoneController.swift deleted file mode 100644 index 3cac10a..0000000 --- a/Sources/ThreeFingerSwitcher/Overlay/NotchHomeZoneController.swift +++ /dev/null @@ -1,389 +0,0 @@ -import AppKit -import SwiftUI - -/// Wires the notch home-zone rail together exactly like `DockPreviewController` (design §8): a passive -/// `GlobalCursorMonitor` (no Input Monitoring) feeds cursor points; while the rail is hidden geometry is -/// read only when the cursor is near the top-center zone (the `nearDockEdge` analogue — cheap idle); a -/// pure `NotchRevealModel` decides reveal/keep/dismiss (`now:` injected); a coarse re-feed timer advances -/// grace/relayout only while shown; teardown is **synchronous**. Gated by the agent-feature opt-in (when -/// off, the monitor isn't installed). A `hasNeedsYou` flag tracks whether any session is `.needsYou` -/// (the rail badges those rows on reveal); the earlier always-on animated glow panel was removed — it -/// pinned the main thread while idle (see docs/postmortem-idle-cpu-spin.md). -@MainActor -final class NotchHomeZoneController { - private let overlay = NotchHomeZoneOverlayController() - private let reveal = NotchRevealModel() - private let cursor: CursorMonitor - private let now: () -> TimeInterval - - /// The data source: the parked rows + their stored conversations (the scheduler/store seam). - var sessionsProvider: () -> [ParkedSession] = { [] } - /// Expand a session's card IN PLACE into its conversation view (`notch-native-conversations`). - var onExpand: ((AgentSessionID) -> Void)? - /// The "+ New chat" card was activated — create a session (durable at birth) and expand it. - var onNewSession: (() -> Void)? - /// The expanded conversation asked to collapse (chevron / Esc / notch-nub click). - var onCollapse: (() -> Void)? - /// Discard a parked session (deletion — cancel pending + remove). - var onDiscard: ((AgentSessionID) -> Void)? - /// Resolve the engine driving a session's expanded conversation (owned by `ParkController`). - var engineProvider: (AgentSessionID) -> NotchSessionEngine? = { _ in nil } - /// The notch tuning dial's read seam (`notch-timeline-and-tuning`): the persisted stop the settings - /// zone opens on. - var tuningProvider: () -> NotchTuning = { .balanced } - /// The active model's architectural context maximum (caps the "Max" stop's token caption). - var modelMaxProvider: () -> Int = { 131_072 } - /// The slider landed on a stop — persist it (applies to each FOLLOWING new chat). - var onTuningChanged: ((NotchTuning) -> Void)? - - private var enabled = false - private var refreshTimer: Timer? - private let refreshInterval: TimeInterval = 0.12 - /// True while a conversation is expanded in place — pins the panel open (cursor departure and the - /// grace-dismiss apply to RAIL mode only; an open conversation never tears down under the reader). - private var isConversationExpanded: Bool { - if case .expanded = overlay.model.mode { return true } - return false - } - - /// Whether any parked session currently needs the user. Tracked for the rail badge + tests; the - /// earlier always-on animated glow panel was removed (see docs/postmortem-idle-cpu-spin.md). - private var hasNeedsYou = false - - init(cursor: CursorMonitor? = nil, - now: @escaping () -> TimeInterval = { ProcessInfo.processInfo.systemUptime }) { - self.cursor = cursor ?? GlobalCursorMonitor() - self.now = now - overlay.onExpand = { [weak self] id in self?.onExpand?(id) } - overlay.onNewSession = { [weak self] in self?.onNewSession?() } - overlay.onCollapse = { [weak self] in self?.onCollapse?() } - overlay.onDiscard = { [weak self] id in self?.onDiscard?(id) } - overlay.onOpenSettings = { [weak self] in self?.openSettings() } - overlay.onCloseSettings = { [weak self] in self?.closeSettings() } - overlay.onTuningChanged = { [weak self] tuning in self?.onTuningChanged?(tuning) } - } - - /// Set the reveal DWELL — how long the cursor must stay crossed behind the notch before the rail reveals - /// (0 ⇒ immediate). Live-settable from the Hub; takes effect on the next reveal (an in-flight dwell is - /// left to complete against its current deadline). - func setRevealDwell(_ interval: TimeInterval) { - reveal.dwellInterval = max(0, interval) - } - - func setEnabled(_ on: Bool) { - guard on != enabled else { return } - enabled = on - if on { - cursor.onMove = { [weak self] point in self?.handleCursor(point) } - cursor.start() - } else { - cursor.stop() - cursor.onMove = nil - dismiss(animated: false) // immediate teardown when the feature is switched off - } - } - - /// Refresh the rail rows + the needs-you flag from the current parked set (call after a scheduler - /// advance / escalate). `hasNeedsYou` is true iff any session is `.needsYou`. - func refresh() { - let sessions = sessionsProvider() - overlay.model.sessions = sessions - hasNeedsYou = sessions.contains { $0.state == .needsYou } - } - - // MARK: - Test seams (the rail view-model the overlay renders + the needs-you predicate) - - /// The rail rows the overlay currently renders (after the last `refresh`/reveal). Test-only read. - var overlayModelSessionsForTest: [ParkedSession] { overlay.model.sessions } - /// Whether any parked session currently needs the user. Test-only read. - var hasNeedsYouForTest: Bool { hasNeedsYou } - - // MARK: - Expand / collapse in place (`notch-native-conversations` D4) - - /// Switch the panel into the in-place EXPANDED conversation for `id`: bind the session's engine to - /// the view-model, flip the mode, and resize the same panel to the expanded content solve. The rail's - /// reveal machinery is then PINNED (see `handleCursor`) so cursor departure never tears it down. - func expandSession(_ id: AgentSessionID) { - let point = NSEvent.mouseLocation - guard let m = metrics(for: point) ?? NSScreen.main.flatMap(metricsFor) else { return } - overlay.model.engine = engineProvider(id) - overlay.model.mode = .expanded(id) - overlay.model.attachment = attachmentFor(m) - let size = NotchHomeZoneLayout.solveExpanded(visibleFrame: m.visibleFrame) - let rect: CGRect - if let notch = m.notch { - rect = NotchHomeZoneAnchor.attachedPanelRect(contentSize: size, notch: notch, - screenFrame: m.screenFrame) - } else { - rect = NotchHomeZoneAnchor.railRect(zone: zoneRectFor(m), size: size, - visibleFrame: m.visibleFrame) - } - // Stretch the border out to the expanded size (a fluid frame animation) when already shown; a hidden - // panel (new chat straight to expanded) grows from the notch point via the first-show spring instead. - overlay.reveal(at: rect, animateResize: true) - manageRefreshTimer() - } - - /// Return the panel to RAIL mode (the expanded conversation collapsed): drop key FIRST (the D5 - /// ordering — never tear down/resize while holding key), unbind the engine, resize back to the rail - /// solve, and let the normal rail grace behavior resume. - func collapseToRail() { - overlay.setKeyCapable(false) - overlay.model.engine = nil - overlay.model.mode = .rail - guard overlay.isVisible else { return } - let point = NSEvent.mouseLocation - guard let m = metrics(for: point) ?? NSScreen.main.flatMap(metricsFor) else { return } - overlay.model.sessions = sessionsProvider() - overlay.model.attachment = attachmentFor(m) - // Stretch the border back down to the rail size (fluid frame animation). - overlay.reveal(at: railRectFor(m, zone: zoneRectFor(m)), animateResize: true) - manageRefreshTimer() - } - - // MARK: - The in-notch settings zone (`notch-timeline-and-tuning` D5) - - /// Morph the panel into the SETTINGS zone (the gear above the "+ New chat" card): seed the dial from - /// the persisted stop, flip the mode, and stretch the same panel to the settings solve — the - /// `expandSession` recipe without an engine. The panel never takes key status in this mode (the - /// slider is mouse-only), and the rail's grace-dismiss keeps applying (see `handleCursor`). - func openSettings() { - let point = NSEvent.mouseLocation - guard let m = metrics(for: point) ?? NSScreen.main.flatMap(metricsFor) else { return } - overlay.model.tuning = tuningProvider() - overlay.model.modelMaxContextTokens = modelMaxProvider() - overlay.model.mode = .settings - overlay.model.attachment = attachmentFor(m) - overlay.reveal(at: settingsRectFor(m), animateResize: true) - manageRefreshTimer() - } - - /// The settings zone's back affordance: return the panel to RAIL mode (the border stretches back - /// down to the rail solve). A no-op unless settings mode is showing. - func closeSettings() { - guard case .settings = overlay.model.mode else { return } - overlay.model.mode = .rail - guard overlay.isVisible else { return } - let point = NSEvent.mouseLocation - guard let m = metrics(for: point) ?? NSScreen.main.flatMap(metricsFor) else { return } - overlay.model.sessions = sessionsProvider() - overlay.model.attachment = attachmentFor(m) - overlay.reveal(at: railRectFor(m, zone: zoneRectFor(m)), animateResize: true) - manageRefreshTimer() - } - - /// The settings zone's panel rect: the compact settings solve through the same attach-or-tab anchor - /// the rail/expanded solves use. - private func settingsRectFor(_ m: ScreenMetrics) -> CGRect { - let size = NotchHomeZoneLayout.solveSettings(visibleFrame: m.visibleFrame) - if let notch = m.notch { - return NotchHomeZoneAnchor.attachedPanelRect(contentSize: size, notch: notch, - screenFrame: m.screenFrame) - } - return NotchHomeZoneAnchor.railRect(zone: zoneRectFor(m), size: size, - visibleFrame: m.visibleFrame) - } - - /// Straight-CLOSE the expanded conversation (the swipe-up "minimize" path): shrink the panel from its - /// current expanded size DIRECTLY into the point behind the notch — no rail-size intermediate to dwell - /// on. The conversation stays BOUND through the shrink (the caller defers its state-collapse to `then`), - /// so the real conversation shrinks — never flashing the empty/new-chat state or the rail. Once hidden: - /// reset the panel to rail mode (clears the conversation pin, clean rail for the next reveal), force the - /// reveal model hidden (a fresh cross-behind is required to re-open), then run `then` (the state-collapse). - func collapseAndClose(then finish: (() -> Void)? = nil) { - overlay.setKeyCapable(false) - reveal.reset() // a fresh cross-behind is required to re-reveal - overlay.hide(animated: true) { [weak self] in - self?.overlay.model.engine = nil - self?.overlay.model.mode = .rail // clean rail for the next reveal; clears the pin - finish?() - } - manageRefreshTimer() - } - - // MARK: - Cursor handling (edge-gated, mirrors DockPreviewController) - - private func handleCursor(_ point: CGPoint) { - guard enabled, let m = metrics(for: point) else { return } - // The expanded-conversation PIN (`notch-native-conversations` D4): while a conversation is open - // in place, the reveal model is not fed at all — cursor departure and the grace-dismiss apply to - // rail mode only (the `menuSuppressedPID` consumer-side suppression precedent). The panel closes - // only via explicit collapse, feature-off, or the synchronous teardown paths. - if isConversationExpanded { return } - let zone = zoneRectFor(m) - // The reveal TRIGGER: the cursor must cross UP *behind the notch* (notched display) or slam to the - // physical top edge (notchless/external) — not merely graze the resting zone below it. The nub/zone - // stays only as a keep-open bridge inside the live region (below). See docs/notch-geometry-reference.md. - let trigger = triggerRectFor(m) - // Edge-gate: compute when something is shown, a dwell is in progress (so it can be advanced OR - // cancelled once the cursor leaves the trigger), or the cursor is near the trigger / live region. - guard overlay.isVisible || reveal.isDwelling - || nearTopZone(point, zone: zone, trigger: trigger, metrics: m) else { return } - - let railFrame = overlay.isVisible ? overlay.frame : nil - // The ONE contiguous live region (zone + connecting band + container, extended up into the notch - // pixels) — so moving the cursor UP into the notch keeps the rail .shown (docks), never dismisses. - let liveZone = liveZoneFor(m, zone: zone, rail: railFrame) - let decision = reveal.feed(cursor: point, zoneRect: trigger, railFrame: railFrame, - liveZone: liveZone, now: now()) - - switch decision { - case .idle: - if overlay.isVisible { dismiss() } - case .dismiss: - dismiss() - case .reveal: - if case .settings = overlay.model.mode, overlay.isVisible { - // Keep-open tick while the SETTINGS zone is showing: reposition at the settings solve — - // never the rail rect, which would snap the morphed panel back to rail size. Grace - // still applies through the same feed (settings dismisses like the rail). - overlay.reveal(at: settingsRectFor(m)) - } else { - overlay.model.sessions = sessionsProvider() - // Set the attachment BEFORE showing so the view carves the notch / insets content correctly on - // the very first frame (no flash of an un-carved rounded rect). - overlay.model.attachment = attachmentFor(m) - let rail = railRectFor(m, zone: zone) - overlay.reveal(at: rail) // spreads out of the notch on first show; repositions after - } - } - manageRefreshTimer() - } - - /// Dismiss the rail (or the settings zone — it grace-dismisses like the rail). `animated` (the - /// default, used for the cursor-left grace-dismiss) recedes it back into the notch on the ease-in-out - /// spread; `animated == false` (restore / feature-off) tears it down synchronously so focus/teardown - /// are immediate (the ghost-on-Space-switch landmine path). A dismissed settings zone resets to rail - /// mode once hidden, so the NEXT reveal always opens the rail. - private func dismiss(animated: Bool = true) { - let wasSettings: Bool = { if case .settings = overlay.model.mode { return true }; return false }() - overlay.hide(animated: animated) { [weak self] in - if wasSettings { self?.overlay.model.mode = .rail } - } - manageRefreshTimer() - } - - private func manageRefreshTimer() { - // Pause the re-feed while an animated recede is in flight (isReceding) — otherwise each tick would - // re-issue the dismiss and the recede would never complete. Also run it while a reveal DWELL is - // counting down, so a perfectly still cursor (no move events) still reveals once the dwell elapses. - let wantTimer = (overlay.isVisible && !overlay.isReceding) || reveal.isDwelling - if wantTimer, refreshTimer == nil { - refreshTimer = Timer.scheduledTimer(withTimeInterval: refreshInterval, repeats: true) { [weak self] _ in - MainActor.assumeIsolated { self?.tick() } - } - } else if !wantTimer, let t = refreshTimer { - t.invalidate() - refreshTimer = nil - } - } - - private func tick() { - // Re-feed at the current cursor so grace advances without depending on move events. - let point = NSEvent.mouseLocation - handleCursor(point) - } - - // MARK: - Geometry (top-center zone on the active screen; notch-attached OR tab-degraded) - - /// The active screen's metrics + the resolved physical notch box (nil ⇒ notchless/external → tab mode). - /// The one place `NSScreen`'s notch geometry (`safeAreaInsets` + the aux menu-bar areas) is read; the - /// pure `NotchHomeZoneAnchor` does the rest. - private struct ScreenMetrics { - let screen: NSScreen - let notch: CGRect? - var screenFrame: CGRect { screen.frame } - var visibleFrame: CGRect { screen.visibleFrame } - var safeAreaTop: CGFloat { screen.safeAreaInsets.top } - } - - private func activeScreen(for point: CGPoint) -> NSScreen? { - NSScreen.screens.first { NSMouseInRect(point, $0.frame, false) } ?? NSScreen.main - } - - private func metrics(for point: CGPoint) -> ScreenMetrics? { - guard let screen = activeScreen(for: point) else { return nil } - return metricsFor(screen) - } - - private func metricsFor(_ screen: NSScreen) -> ScreenMetrics { - let notch = NotchHomeZoneAnchor.notchRect( - screenFrame: screen.frame, - safeAreaTop: screen.safeAreaInsets.top, - auxLeft: screen.auxiliaryTopLeftArea, - auxRight: screen.auxiliaryTopRightArea) - return ScreenMetrics(screen: screen, notch: notch) - } - - private var zoneSize: CGSize { - CGSize(width: NotchHomeZoneLayout.zoneWidth, height: NotchHomeZoneLayout.zoneHeight) - } - - /// The reveal target: in attached mode a thin nub hugging the notch's bottom edge; else the honest - /// top-center tab a margin below the menu bar. - private func zoneRectFor(_ m: ScreenMetrics) -> CGRect { - if let notch = m.notch { - return NotchHomeZoneAnchor.attachedNubRect(size: zoneSize, notch: notch, screenFrame: m.screenFrame) - } - return NotchHomeZoneAnchor.zoneRect(size: zoneSize, visibleFrame: m.visibleFrame, safeAreaTop: m.safeAreaTop) - } - - /// The reveal TRIGGER region (distinct from the resting zone/nub, which only bridges keep-open): the - /// physical notch cutout on a notched display — reveal fires ONLY when the cursor crosses UP behind the - /// notch — or a thin top-edge band at top-center on a notchless/external display (slam to the top edge). - private func triggerRectFor(_ m: ScreenMetrics) -> CGRect { - if let notch = m.notch { - return NotchHomeZoneAnchor.notchTriggerRect(notch: notch) - } - return NotchHomeZoneAnchor.topEdgeTriggerRect(width: zoneSize.width, - visibleFrame: m.visibleFrame, - screenFrame: m.screenFrame) - } - - /// The revealed panel: in attached mode it merges into the notch (top at the physical top, content band - /// below); else it hangs flush below the tab. Both hug their sessions via the shared content-fit solve. - private func railRectFor(_ m: ScreenMetrics, zone: CGRect) -> CGRect { - // The rail always renders the persistent "+ New chat" card PLUS one per session, so the width solve - // must count that extra card — otherwise the panel is one card too narrow and the row scrolls. The - // dock is non-scrollable: it hugs all cards and EXPANDS as sessions are added. - let cardCount = overlay.model.sessions.count + 1 - let solved = NotchHomeZoneLayout.solve(count: cardCount, visibleFrame: m.visibleFrame) - if let notch = m.notch { - // Size the panel taller so that CENTERING the row (in the view) yields symmetric padding of - // `notch.height + a little` on top and bottom — the cards clear the notch and sit balanced. - let contentH = NotchHomeZoneLayout.attachedRailContentHeight(notchHeight: notch.height) - let contentSize = CGSize(width: solved.contentSize.width, height: contentH) - return NotchHomeZoneAnchor.attachedPanelRect( - contentSize: contentSize, notch: notch, screenFrame: m.screenFrame) - } - return NotchHomeZoneAnchor.railRect(zone: zone, size: solved.contentSize, visibleFrame: m.visibleFrame) - } - - private func liveZoneFor(_ m: ScreenMetrics, zone: CGRect, rail: CGRect?) -> CGRect { - if let notch = m.notch { - return NotchHomeZoneAnchor.attachedLiveZone(nub: zone, panel: rail, notch: notch) - } - return NotchHomeZoneAnchor.liveZoneRect(zone: zone, rail: rail, visibleFrame: m.visibleFrame) - } - - private func attachmentFor(_ m: ScreenMetrics) -> NotchAttachment { - if let notch = m.notch { - return .notch(cutout: CGSize(width: notch.width, height: notch.height)) - } - return .tab - } - - /// Edge-gate: the cursor is near the contiguous live region OR near the reveal trigger (cheap idle, like - /// `nearDockEdge`). Covers the connecting band + the notch pixels above the zone so the cursor isn't - /// dropped on the way UP into the notch (the dismiss-on-move-into-notch bug); the trigger is included - /// explicitly because the notchless top-edge band sits ABOVE the menu-bar-capped live region. - private func nearTopZone(_ point: CGPoint, zone: CGRect, trigger: CGRect, metrics m: ScreenMetrics) -> Bool { - guard zone != .zero else { return false } - let railFrame = overlay.isVisible ? overlay.frame : nil - let live = liveZoneFor(m, zone: zone, rail: railFrame) - // A small slack ring (the `nearDockEdge` slop) around both the live region and the reveal trigger. - return live.insetBy(dx: -40, dy: -8).contains(point) - || trigger.insetBy(dx: -40, dy: -8).contains(point) - } - -} diff --git a/Sources/ThreeFingerSwitcher/Overlay/NotchHomeZoneOverlay.swift b/Sources/ThreeFingerSwitcher/Overlay/NotchHomeZoneOverlay.swift deleted file mode 100644 index 6876091..0000000 --- a/Sources/ThreeFingerSwitcher/Overlay/NotchHomeZoneOverlay.swift +++ /dev/null @@ -1,1113 +0,0 @@ -import AppKit -import SwiftUI -import Combine - -/// The solved content-fit size of the notch rail (design D5), mirroring `SwitcherGridLayout`: the rail -/// HUGS its actual sessions in width AND height like `SwitcherLayout.solveGrid` + `OverlayController.layout` -/// — width = chrome + summed card widths + inter-card spacing (clamped to a fraction of the visible -/// frame), height fitted (clamped to a fraction, overflowing → scroll). Both the view (which renders) -/// and the controller (which sizes the panel) read THIS one result so the rendered rail and the panel -/// frame cannot drift. -struct NotchRailLayout: Equatable { - /// The hugged content size (the panel frame), already clamped to the available screen fraction. - let contentSize: CGSize - /// True when the natural width exceeds the clamp (the row scrolls horizontally). - let overflowsHorizontally: Bool - /// True when the natural height exceeds the clamp (the row scrolls vertically — many tall cards). - let overflowsVertically: Bool - - static let zero = NotchRailLayout(contentSize: .zero, - overflowsHorizontally: false, - overflowsVertically: false) -} - -/// Layout metrics + the content-fit solve for the notch home-zone rail. The rail is a NON-scrollable row -/// of parked cards that EMERGES FROM the notch and HUGS its sessions (no hardcoded width) — it EXPANDS as -/// cards are added rather than scrolling. The `overflows*` flags remain as safety signals (a pathological -/// count beyond the parked-session cap) but no longer drive a scroll view. -enum NotchHomeZoneLayout { - static let cardWidth: CGFloat = 168 - static let cardHeight: CGFloat = 92 - static let cardSpacing: CGFloat = 12 - /// Horizontal chrome padding (each side of the card row). - static let padding: CGFloat = 14 - /// The card row is **centered** in the panel with **symmetric** vertical padding. On a notched display - /// that padding is the **notch height + a little** (`railNotchClearance`) on BOTH the top and the bottom - /// — so the row clears the notch by a small gap and sits balanced (never shoved to the bottom). Because - /// it depends on the runtime notch height, the attached panel height is computed by the controller - /// (which knows the cutout); this constant is only the "a little". On a notchless/external display there - /// is no notch to clear, so a fixed comfortable vertical padding (`railTabVerticalPadding`) is used. - static let railNotchClearance: CGFloat = 8 - static let railTabVerticalPadding: CGFloat = 16 - - /// The in-place EXPANDED conversation panel's content-fit solve (`notch-native-conversations` D4): - /// a fixed comfortable reading size clamped to fractions of the visible frame (small screens stay - /// sane; content scrolls inside). Same anchor functions as the rail — only the content size differs. - static func solveExpanded(visibleFrame: CGRect) -> CGSize { - let maxW = visibleFrame.width > 1 ? visibleFrame.width * 0.46 : expandedWidth - let maxH = visibleFrame.height > 1 ? visibleFrame.height * 0.62 : expandedHeight - return CGSize(width: min(expandedWidth, maxW), height: min(expandedHeight, maxH)) - } - static let expandedWidth: CGFloat = 560 - static let expandedHeight: CGFloat = 520 - - /// The in-notch SETTINGS zone's content-fit solve (`notch-timeline-and-tuning` D5): a compact fixed - /// card clamped to fractions of the visible frame — the expanded-conversation idiom, smaller (one - /// slider plus captions, no thread). - static func solveSettings(visibleFrame: CGRect) -> CGSize { - let maxW = visibleFrame.width > 1 ? visibleFrame.width * 0.40 : settingsWidth - let maxH = visibleFrame.height > 1 ? visibleFrame.height * 0.42 : settingsHeight - return CGSize(width: min(settingsWidth, maxW), height: min(settingsHeight, maxH)) - } - static let settingsWidth: CGFloat = 440 - static let settingsHeight: CGFloat = 252 - - /// The resting zone (the reveal target + the ambient-glow host) — a slim top-center tab. - static let zoneWidth: CGFloat = 120 - static let zoneHeight: CGFloat = 10 - - /// Fraction of the active screen's visible frame the rail may occupy before it scrolls (the - /// `SwitcherLayout.canvasWidthFraction`/`canvasHeightFraction` idiom). - static let maxWidthFraction: CGFloat = 0.86 - static let maxHeightFraction: CGFloat = 0.60 - - /// The tab-mode (notchless) rail content height: the card row centered with `railTabVerticalPadding` - /// on top and bottom. The attached (notched) height is computed per-notch via `attachedRailContentHeight`. - static var railHeight: CGFloat { cardHeight + 2 * railTabVerticalPadding } - - /// The below-notch content height to hand `attachedPanelRect` so the card row ends up **centered** in the - /// merged panel with `notchHeight + railNotchClearance` padding on the top and bottom. `attachedPanelRect` - /// re-adds the notch band on top of this, so it is `card + notch + 2*clearance` — making the total panel - /// `card + 2*(notch + clearance)`, which centered yields the symmetric notch-clearing padding. - static func attachedRailContentHeight(notchHeight: CGFloat) -> CGFloat { - cardHeight + notchHeight + 2 * railNotchClearance - } - - /// Rail width for `count` cards, clamped to the available screen width (scrolls when it overflows). - /// Retained as the width primitive `solve` builds on; a single card hugs to a one-card floor. - static func railWidth(count: Int, maxWidth: CGFloat) -> CGFloat { - return min(max(naturalWidth(count: count), oneCardWidth), maxWidth) - } - - /// The un-clamped natural width for `count` cards (chrome + cards + inter-card spacing). - static func naturalWidth(count: Int) -> CGFloat { - let n = max(count, 0) - guard n > 0 else { return oneCardWidth } - return padding * 2 + cardWidth * CGFloat(n) + cardSpacing * CGFloat(max(n - 1, 0)) - } - - /// The one-card floor — a single (or empty) rail hugs to exactly one card plus chrome. - static var oneCardWidth: CGFloat { padding * 2 + cardWidth } - - /// Solve the content-fit size for `count` parked cards within `visibleFrame` (design D5): width hugs - /// the cards (chrome + sum + spacing) so the dock EXPANDS per card, clamped to `maxWidthFraction` of the - /// visible width with a one-card floor (a safety ceiling for the pathological beyond-cap case — the dock - /// is non-scrollable, so past the ceiling it clips rather than scrolls); height is a single band - /// (`railHeight`), clamped to `maxHeightFraction` of the visible height. Mirrors `SwitcherLayout.solveGrid`'s - /// contentSize + overflow-flag contract (the `overflows*` flags are retained as signals only). - static func solve(count: Int, visibleFrame: CGRect) -> NotchRailLayout { - let maxW = visibleFrame.width > 1 ? visibleFrame.width * maxWidthFraction : naturalWidth(count: count) - let maxH = visibleFrame.height > 1 ? visibleFrame.height * maxHeightFraction : railHeight - - let natural = naturalWidth(count: count) - let width = min(max(natural, oneCardWidth), maxW) - let height = min(railHeight, maxH) - - return NotchRailLayout( - contentSize: CGSize(width: width, height: height), - overflowsHorizontally: natural > maxW + 0.5, - overflowsVertically: railHeight > maxH + 0.5) - } -} - -/// How the panel attaches to the display top: `.notch` merges into a physical notch (the NotchNook look — -/// the opaque-black panel top reaches the physical top and spans behind the black notch, no carving), `.tab` -/// is the honest top-center rounded pill on a notchless/external display. Carries the notch cutout size -/// (panel-local points); its height is the notch-band headroom the view keeps the centered content clear of. -enum NotchAttachment: Equatable { - case notch(cutout: CGSize) - case tab -} - -/// The observable rail state (the AppKit/SwiftUI seam). The pure scheduler/store own the data; this is -/// the view-model the overlay renders. `@MainActor` — UI state. -@MainActor -final class NotchHomeZoneViewModel: ObservableObject { - /// What the panel is showing (`notch-native-conversations` D4): the card RAIL, ONE session EXPANDED - /// in place into its conversation view, or the in-notch SETTINGS zone (`notch-timeline-and-tuning`). - /// The same panel and chrome serve all three — never a second panel. - enum Mode: Equatable { - case rail - case expanded(AgentSessionID) - case settings - } - - @Published var sessions: [ParkedSession] = [] - /// The current attachment mode + notch cutout (set by the controller before each reveal/reposition). - /// Defaults to `.tab` so the view is well-defined before the first geometry solve. - @Published var attachment: NotchAttachment = .tab - /// Drives the smooth ease-in-out **spread**: `false` collapses the panel toward the notch (a tiny seed - /// at the top-center), `true` spreads it out to full size. The controller flips it inside `withAnimation` - /// on reveal (→ true) and animated hide (→ false), so the panel grows out of / recedes into the notch. - @Published var isExpanded: Bool = false - /// Rail vs in-place expanded conversation (`notch-native-conversations` D4). - @Published var mode: Mode = .rail - /// The engine driving the currently-expanded session's conversation (set by the controller on - /// expand, cleared on collapse). The expanded view observes it directly. - @Published var engine: NotchSessionEngine? - /// The notch tuning dial the settings zone renders (seeded from settings by the controller on - /// open; written back through `onTuningChanged` on every slider move). - @Published var tuning: NotchTuning = .balanced - /// The active model's architectural context maximum — caps the "Max" stop's token caption. - @Published var modelMaxContextTokens: Int = 131_072 - /// True iff at least one parked session is in `.needsYou` — drives the ambient glow. - var hasNeedsYou: Bool { sessions.contains { $0.state == .needsYou } } -} - -/// Owns the **mouse-interactive, non-activating** notch home-zone panel — the `DockPreviewOverlayController` -/// recipe (design §5/§8): `[.borderless, .nonactivatingPanel]`, `ignoresMouseEvents = false`, -/// `acceptsMouseMovedEvents = true`, `level = .popUpMenu`, `.canJoinAllSpaces`, NEVER key/main (so it -/// never steals focus from the foreground app — the restore target). Teardown is **synchronous** -/// (`orderOut`) — the ghost-on-Space-switch landmine applies here exactly as in the Files band + Dock -/// preview. `show(at:)` orders front (reveal); `move(to:)` repositions only (never re-fronts, so the -/// ambient glow / any layer above is never stomped); `hide()` orders out synchronously. -@MainActor -final class NotchHomeZoneOverlayController { - let model = NotchHomeZoneViewModel() - private var panel: SwitcherPanel? - - /// A card was clicked — expand that session IN PLACE into its conversation view. - var onExpand: ((AgentSessionID) -> Void)? - /// The "+ New chat" card was activated — create a fresh session (durable at birth) and expand it. - var onNewSession: (() -> Void)? - /// The expanded conversation asked to collapse (chevron / Esc / notch-nub click). - var onCollapse: (() -> Void)? - /// A card was discarded (deletion — from the card's context menu or the expanded header). - var onDiscard: ((AgentSessionID) -> Void)? - /// The gear above the "+ New chat" card was clicked — morph the panel into the settings zone. - var onOpenSettings: (() -> Void)? - /// The settings zone's back affordance — return the panel to rail mode. - var onCloseSettings: (() -> Void)? - /// The settings zone's slider landed on a stop — persist it (applies to following NEW chats). - var onTuningChanged: ((NotchTuning) -> Void)? - - /// Flip the panel's key-capability while the expanded composer is focused (`notch-native-conversations` - /// D5): the panel stays `.nonactivatingPanel` (becoming key never activates the app; the front app - /// keeps focus) and NEVER becomes main. Dropping key BEFORE any order-out is the caller's job on - /// collapse/teardown paths (the focus-leak ordering). - func setKeyCapable(_ on: Bool) { - guard let panel else { return } - panel.keyInteractive = on - if on { - panel.makeKey() - } else if panel.isKeyWindow { - panel.resignKey() - } - } - - private func makePanel() -> SwitcherPanel { - // The initial content rect is the one-card floor; the controller's content-fit solve resizes it to - // hug the actual sessions on `show(at:)`/`move(to:)` (no hardcoded width — the placement-bug fix). - let panel = SwitcherPanel( - contentRect: NSRect(x: 0, y: 0, - width: NotchHomeZoneLayout.oneCardWidth, - height: NotchHomeZoneLayout.railHeight), - styleMask: [.borderless, .nonactivatingPanel], - backing: .buffered, - defer: false - ) - panel.isOpaque = false - panel.backgroundColor = .clear - panel.hasShadow = true - // The rail takes the pointer (click/drag a card to restore) — like the Dock preview, the lone - // mouse-interactive overlay species. Still non-activating + never key/main → no focus steal. - panel.ignoresMouseEvents = false - panel.acceptsMouseMovedEvents = true - panel.isFloatingPanel = true - panel.hidesOnDeactivate = false - panel.level = .popUpMenu - // Attached mode positions the panel's TOP at the physical top of the display (over the menu bar) so - // its black merges with the notch — skip AppKit's constrain-below-the-menu-bar so the frame sticks. - panel.reachesPhysicalTop = true - panel.collectionBehavior = [.canJoinAllSpaces, .fullScreenAuxiliary, .ignoresCycle] - panel.contentView = NSHostingView(rootView: NotchHomeZoneRailView( - model: model, - onExpand: { [weak self] id in self?.onExpand?(id) }, - onNewSession: { [weak self] in self?.onNewSession?() }, - onCollapse: { [weak self] in self?.onCollapse?() }, - onDiscard: { [weak self] id in self?.onDiscard?(id) }, - onComposerFocusChanged: { [weak self] focused in self?.setKeyCapable(focused) }, - onOpenSettings: { [weak self] in self?.onOpenSettings?() }, - onCloseSettings: { [weak self] in self?.onCloseSettings?() }, - onTuningChanged: { [weak self] tuning in self?.onTuningChanged?(tuning) } - )) - return panel - } - - var isVisible: Bool { panel?.isVisible ?? false } - var frame: CGRect { panel?.frame ?? .zero } - - /// The fluid "droplet" spread duration — the orderOut delay after an animated shrink, and the duration - /// of the rail↔expanded frame stretch. - static let spreadDuration: TimeInterval = 0.34 - /// The spring driving the grow-from-a-point (reveal) / shrink-to-a-point (hide) content scale — a fluid - /// droplet spread (a touch of settle, no bounce-heavy overshoot). - private static let growSpring: Animation = .spring(response: 0.42, dampingFraction: 0.78) - /// A scheduled animated-hide teardown, cancelled if a fresh reveal arrives mid-recede. - private var pendingHide: DispatchWorkItem? - /// True while an animated recede is in flight (its `orderOut` is scheduled but not yet run) — the - /// controller reads this to pause its re-feed timer so it doesn't re-trigger the recede every tick. - var isReceding: Bool { pendingHide != nil } - /// True while a rail↔expanded frame STRETCH is animating — per-tick reposition setFrames are skipped so - /// they don't snap the frame and kill the stretch mid-flight. - private var isResizing = false - - /// Reveal / resize / reposition the panel at `rect`. Three paths, all fade-free: - /// - **First show** → place at the target instantly, order front, and GROW the whole panel (border and - /// all) out of a point behind the notch via a spring on the content scale (anchored at the top). - /// - **Mode change** (`animateResize`, rail↔expanded) → STRETCH the window frame to the new size fluidly - /// (the SwiftUI shape fills the window, so the frame edge IS the border). - /// - **Per-tick reanchor** → reposition instantly (never re-fronts a visible panel — matching the - /// Dock-preview `move(to:)` discipline — and skipped entirely while a stretch is mid-flight). - /// A reveal arriving during an animated recede cancels the pending teardown. - func reveal(at rect: CGRect, animateResize: Bool = false) { - pendingHide?.cancel(); pendingHide = nil - let panel = self.panel ?? makePanel() - self.panel = panel - // The window (the "outer container") casts NO shadow in attached mode — a window drop-shadow at the - // top would read as a border where the panel merges into the notch/menu bar. Tab mode keeps it. - let attached: Bool = { if case .notch = model.attachment { return true }; return false }() - if panel.hasShadow != !attached { - panel.hasShadow = !attached - panel.invalidateShadow() - } - if !panel.isVisible { - // FIRST SHOW: seed the panel as a point behind the notch, order front, then spring it open. - model.isExpanded = false - panel.setFrame(rect, display: true) - panel.orderFrontRegardless() - withAnimation(Self.growSpring) { model.isExpanded = true } - return - } - if animateResize, panel.frame != rect { - // MODE CHANGE (rail↔expanded): stretch the border to the new size. - model.isExpanded = true - animateFrame(to: rect) - return - } - // PER-TICK reanchor: reposition instantly (skipped while a stretch is mid-flight so it isn't snapped), - // and RE-GROW if this reveal arrived mid-shrink (the pending grace-dismiss teardown was just cancelled - // above) so the panel springs back open instead of staying collapsed as a point. - if !isResizing { panel.setFrame(rect, display: true) } - if !model.isExpanded { - withAnimation(Self.growSpring) { model.isExpanded = true } - } - } - - /// Fluidly stretch the panel's window frame to `rect` (the rail↔expanded border stretch). One-shot and - /// self-terminating — the idle-CPU-spin landmine is about *perpetual* animations; this settles and stops. - private func animateFrame(to rect: CGRect) { - guard let panel else { return } - isResizing = true - NSAnimationContext.runAnimationGroup({ ctx in - ctx.duration = Self.spreadDuration - ctx.timingFunction = CAMediaTimingFunction(controlPoints: 0.2, 0.9, 0.25, 1) // fluid ease-out - panel.animator().setFrame(rect, display: true) - }, completionHandler: { [weak self] in - MainActor.assumeIsolated { self?.isResizing = false } - }) - } - - /// Tear down the panel. `animated` SHRINKS whatever is showing (rail OR the expanded conversation) - /// straight back into the point behind the notch (spring on the scale, no fade) then orders out on - /// completion — the gentle grace-dismiss AND the swipe-up straight-close. `animated == false` is the - /// **synchronous** `orderOut` (the ghost-on-Space-switch landmine) — restore/disable, teardown immediate. - /// `completion` runs AFTER the order-out (used by the straight-close to reset the panel to rail mode). - func hide(animated: Bool, completion: (() -> Void)? = nil) { - guard let panel else { completion?(); return } - // Drop key BEFORE any order-out (D5 ordering): a panel holding key through its teardown can - // strand first-responder state on a Space switch. Harmless when it was never key. - setKeyCapable(false) - guard animated else { - pendingHide?.cancel(); pendingHide = nil - model.isExpanded = false - panel.orderOut(nil) // synchronous — ghost-on-Space-switch landmine - completion?() - return - } - if pendingHide != nil { return } // already receding — don't restart the animation - withAnimation(Self.growSpring) { model.isExpanded = false } - let work = DispatchWorkItem { [weak self] in - self?.panel?.orderOut(nil) - self?.pendingHide = nil - completion?() - } - pendingHide = work - DispatchQueue.main.asyncAfter(deadline: .now() + Self.spreadDuration, execute: work) - } -} - -/// The rail: a NON-scrollable row of parked cards that hugs and EXPANDS with its sessions. In attached mode it's a **plain black rounded -/// rectangle** whose top edge reaches the physical top so its black simply spans up **behind** the (also -/// black) notch — no cutout is carved, because both are black and read as one shape. The whole container -/// spreads out of / recedes into the notch on the controller's ease-in-out `isExpanded` transition. The -/// cards are **centered** (both axes) in the panel — horizontally so a few sessions don't hug the left, and -/// vertically so they sit clear of the notch. A bounded `failed` badge carries the clean headline ONLY (raw -/// text lives behind an opt-in disclosure on the restored canvas, never here). -struct NotchHomeZoneRailView: View { - @ObservedObject var model: NotchHomeZoneViewModel - let onExpand: (AgentSessionID) -> Void - let onNewSession: () -> Void - let onCollapse: () -> Void - let onDiscard: (AgentSessionID) -> Void - /// The composer focus seam (`notch-native-conversations` D5): the controller flips the panel's - /// key-capability so keystrokes reach the field only while it is focused. - let onComposerFocusChanged: (Bool) -> Void - /// The settings-zone seams (`notch-timeline-and-tuning`): gear → settings mode, back → rail, - /// slider → persisted stop. - let onOpenSettings: () -> Void - let onCloseSettings: () -> Void - let onTuningChanged: (NotchTuning) -> Void - - private var isAttached: Bool { - if case .notch = model.attachment { return true } - return false - } - - var body: some View { - Group { - switch model.mode { - case .rail: - railBody - case let .expanded(id): - if let engine = model.engine { - NotchConversationView( - engine: engine, - session: model.sessions.first { $0.id == id }, - isAttached: isAttached, - onCollapse: onCollapse, - onDelete: { onDiscard(id) }, - onComposerFocusChanged: onComposerFocusChanged) - } else { - // Defensive: an expanded id with no engine (discarded underneath) renders the rail. - railBody - } - case .settings: - NotchTuningSettingsView( - model: model, - isAttached: isAttached, - onBack: onCloseSettings, - onTuningChanged: onTuningChanged) - } - } - .background(chromeFill) - .overlay(chromeStroke) - .clipShape(chromeShape) - // The fluid "droplet" SPREAD: the whole panel — border and all — GROWS from a point behind the notch - // to full size (reveal) and shrinks back (hide), anchored at its TOP edge so it unfurls downward AND - // out to both sides. Driven by the controller's spring on `isExpanded`. There is NO opacity fade — - // the border itself stretches out of the point (rail↔expanded resizes stretch the window frame). - .scaleEffect(model.isExpanded ? 1 : Self.seedScale, anchor: .top) - } - - /// The card rail (the `.rail` mode body): the persistent "+ New chat" card, then one card per - /// session. Present even with zero sessions — the dock is never an empty dead end. - /// - /// NON-scrollable: the panel is sized (by the shared `solve`, counting this same "+ New chat" card) to - /// HUG every rendered card, so the dock simply EXPANDS as sessions are added rather than scrolling. The - /// panel is **height-sized by the controller** so that simply CENTERING the row here yields the intended - /// symmetric vertical padding — the notch height + a little on a notched display (so the cards clear the - /// notch and sit balanced), or a fixed gap in tab mode. - private var railBody: some View { - HStack(spacing: NotchHomeZoneLayout.cardSpacing) { - NotchNewChatCard(action: onNewSession, onOpenSettings: onOpenSettings) - ForEach(model.sessions) { session in - NotchParkedCard(session: session, - onExpand: { onExpand(session.id) }, - onDiscard: { onDiscard(session.id) }) - } - } - .padding(.horizontal, NotchHomeZoneLayout.padding) - .frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .center) - } - - /// The collapsed seed scale — a near-point at the notch the panel springs out of; non-zero so the - /// `.top` scale anchor stays well-defined during the grow (a literal `0` collapses the anchor with it). - private static let seedScale: CGFloat = 0.03 - - /// The chrome silhouette. Attached mode is a plain rounded rectangle with a **flat top** (square top - /// corners so it grows cleanly from the physical top / behind the notch) and rounded bottom corners; the - /// tab-degradation path is a fully-rounded pill. No notch is carved — the black spans behind the notch. - private var chromeShape: AnyShape { - if isAttached { - return AnyShape(UnevenRoundedRectangle( - topLeadingRadius: 0, bottomLeadingRadius: 20, - bottomTrailingRadius: 20, topTrailingRadius: 0)) - } - return AnyShape(RoundedRectangle(cornerRadius: 18)) - } - - /// In attached mode the fill is opaque black so it reads as one shape with the hardware notch's black - /// pixels (a translucent material would reveal a seam); in tab mode it stays the app's `.regularMaterial`. - @ViewBuilder private var chromeFill: some View { - if isAttached { - chromeShape.fill(Color.black) - } else { - chromeShape.fill(.regularMaterial) - } - } - - /// The hairline border. In attached mode it traces only the **sides and bottom** — never the top edge — - /// so there is no line where the panel merges into the notch / menu bar at the physical top; the tab - /// path keeps a full rounded-rect border. - @ViewBuilder private var chromeStroke: some View { - if isAttached { - NotchPanelBorderShape().stroke(.white.opacity(0.08), lineWidth: 1) - } else { - chromeShape.stroke(.white.opacity(0.08), lineWidth: 1) - } - } -} - -/// An OPEN border tracing the attached panel's left side, rounded bottom, and right side — but NOT the top -/// edge — so the merged panel shows no hairline at the physical top where it meets the notch/menu bar. -private struct NotchPanelBorderShape: Shape { - var bottomRadius: CGFloat = 20 - func path(in rect: CGRect) -> Path { - let W = rect.width, H = rect.height - let r = min(bottomRadius, min(W, H) / 2) - var p = Path() - p.move(to: CGPoint(x: 0, y: 0)) // top-left (top edge NOT drawn) - p.addLine(to: CGPoint(x: 0, y: H - r)) - p.addQuadCurve(to: CGPoint(x: r, y: H), control: CGPoint(x: 0, y: H)) - p.addLine(to: CGPoint(x: W - r, y: H)) - p.addQuadCurve(to: CGPoint(x: W, y: H - r), control: CGPoint(x: W, y: H)) - p.addLine(to: CGPoint(x: W, y: 0)) // up to top-right, then stop - return p - } -} - -/// The persistent "+ New chat" card (`notch-native-conversations`): the notch is the ONLY birthplace of -/// conversational sessions, so the rail always offers one — including on an empty dock. The settings -/// GEAR sits above the plus icon (`notch-timeline-and-tuning`): clicking it morphs the panel into the -/// in-notch settings zone (the thinking+context dial for following new chats). -struct NotchNewChatCard: View { - let action: () -> Void - let onOpenSettings: () -> Void - - var body: some View { - VStack(spacing: 4) { - Button(action: onOpenSettings) { - Image(systemName: "gearshape.fill") - .font(.system(size: 11, weight: .semibold)) - .foregroundStyle(.secondary) - .frame(maxWidth: .infinity, minHeight: 18) - .contentShape(Rectangle()) - } - .buttonStyle(.plain) - .help("Tune thinking + context for new chats") - Button(action: action) { - VStack(spacing: 6) { - Image(systemName: "plus.bubble") - .font(.system(size: 20, weight: .medium)) - .foregroundStyle(Color.accentColor) - Text("New chat") - .font(.system(size: 12, weight: .semibold)) - .foregroundStyle(.secondary) - } - .frame(maxWidth: .infinity, maxHeight: .infinity) - .contentShape(Rectangle()) - } - .buttonStyle(.plain) - .help("Start a new conversation") - } - .frame(width: NotchHomeZoneLayout.cardWidth, - height: NotchHomeZoneLayout.cardHeight) - .padding(10) - .background(.black.opacity(0.18), in: RoundedRectangle(cornerRadius: 12)) - .overlay(RoundedRectangle(cornerRadius: 12) - .strokeBorder(style: StrokeStyle(lineWidth: 1, dash: [5, 4])) - .foregroundStyle(.white.opacity(0.18))) - } -} - -/// One parked-session card: a title + a state badge. Clicking it expands it in place. -struct NotchParkedCard: View { - let session: ParkedSession - let onExpand: () -> Void - let onDiscard: () -> Void - - var body: some View { - Button(action: onExpand) { - VStack(alignment: .leading, spacing: 6) { - Text(session.title.isEmpty ? "Session" : session.title) - .font(.system(size: 12, weight: .semibold)) - .lineLimit(1).truncationMode(.middle) - ParkStateBadge(state: session.state, badgeCount: session.badgeCount, - isScheduled: session.nextRunAt != nil) - Spacer(minLength: 0) - } - .frame(width: NotchHomeZoneLayout.cardWidth, - height: NotchHomeZoneLayout.cardHeight, alignment: .topLeading) - .padding(10) - .background(.black.opacity(0.18), in: RoundedRectangle(cornerRadius: 12)) - .overlay(RoundedRectangle(cornerRadius: 12) - .strokeBorder(session.state == .needsYou ? Color.accentColor : .white.opacity(0.10), - lineWidth: session.state == .needsYou ? 2 : 1)) - } - .buttonStyle(.plain) - .contextMenu { Button("Discard", role: .destructive, action: onDiscard) } - } -} - -/// The per-card state badge — thinking / done-count / needs-you accent / failed-with-clean-headline. -struct ParkStateBadge: View { - let state: ParkState - let badgeCount: Int - /// Whether a `.parked` row is scheduled or actively advancing (`nextRunAt` set, or being served - /// right now) vs DORMANT awaiting the user (a confirm-tier pause). Irrelevant for other states. - var isScheduled: Bool = true - - var body: some View { - switch state { - case .parked where isScheduled: - label("Thinking…", system: "ellipsis", tint: .secondary) - case .parked: - // Dormant: a paused step waits until the user opens the session (no escalation, no glow). - label("Waiting for you", system: "hourglass", tint: .secondary) - case .idle where badgeCount > 0, .active where badgeCount > 0: - label("\(badgeCount) new", system: "checkmark.circle.fill", tint: .green) - case .active: - // The foreground session: expanded right now, or collapsed while its turn still streams - // (the engine keeps it `.active` so the background scheduler never double-advances it). - label("Working…", system: "circle.dotted", tint: .secondary) - case .idle: - label("Parked", system: "moon.zzz", tint: .secondary) - case .needsYou: - label("Needs you", system: "exclamationmark.circle.fill", tint: .accentColor) - } - } - - private func label(_ text: String, system: String, tint: Color) -> some View { - Label(text, systemImage: system) - .font(.system(size: 10, weight: .medium)) - .foregroundStyle(tint) - .lineLimit(1) - } -} - -/// The in-place EXPANDED conversation (`notch-native-conversations` D4): one session's thread + composer -/// inside the same merged-notch chrome. A cursor-and-keyboard surface — Approve/Skip and Copy are -/// buttons; Enter sends; Esc collapses. The panel becomes key ONLY while the composer is focused (the -/// `onComposerFocusChanged` seam), so the previously-front app keeps focus otherwise. -struct NotchConversationView: View { - @ObservedObject var engine: NotchSessionEngine - /// The session's rail row (title/state), when it still exists. - let session: ParkedSession? - /// Attached (merged-notch, over the physical top) vs tab mode — drives the top headroom. - let isAttached: Bool - let onCollapse: () -> Void - let onDelete: () -> Void - let onComposerFocusChanged: (Bool) -> Void - - @State private var composerText = "" - /// The user-toggled OPEN thinking blocks, keyed `messageID-segmentIndex` (`live-N` for the in-flight - /// turn). Settled blocks default collapsed; the live streaming tail is forced open while it streams. - @State private var expandedThinking: Set = [] - @FocusState private var composerFocused: Bool - - var body: some View { - VStack(alignment: .leading, spacing: 0) { - header - // Attached mode: the panel's top band sits behind the physical notch — keep the header - // clear of it (the same headroom idea the centered rail uses). - .padding(.top, isAttached ? 34 : 10) - .padding(.horizontal, 14) - Divider().opacity(0.25).padding(.vertical, 8) - thread - .padding(.horizontal, 14) - composer - .padding(.horizontal, 14) - .padding(.vertical, 10) - } - .onChange(of: composerFocused) { _, focused in onComposerFocusChanged(focused) } - // Collapse on Esc from anywhere in the view (the composer owns key events while focused). - .onExitCommand { onCollapse() } - .onDisappear { onComposerFocusChanged(false) } // never leave the panel key-capable behind - } - - private var header: some View { - HStack(spacing: 10) { - Image(systemName: "text.bubble.fill") - .font(.system(size: 14, weight: .semibold)) - .foregroundStyle(Color.accentColor) - Text(engine.conversation?.title ?? session?.title ?? "Conversation") - .font(.system(size: 14, weight: .semibold)) - .foregroundStyle(.white) - .lineLimit(1).truncationMode(.middle) - if !engine.isSessionReasoningEnabled { - // Born-with visibility (`notch-timeline-and-tuning`): a session running WITHOUT - // reasoning is marked, so "no streamed thinking" reads as this chat's tuning — the - // only Thinking content it will show is the router's rationale, which arrives whole. - Label("Thinking off", systemImage: "bolt.fill") - .font(.system(size: 10, weight: .medium)) - .foregroundStyle(.orange.opacity(0.85)) - .help("This chat was created with thinking off (the Quick stop). Move the notch settings slider and start a new chat to stream the model's reasoning.") - } - Spacer() - Button(action: onDelete) { - Image(systemName: "trash") - .font(.system(size: 12, weight: .semibold)) - .foregroundStyle(.secondary) - } - .buttonStyle(.plain) - .help("Delete this session") - Button(action: onCollapse) { - Image(systemName: "chevron.up") - .font(.system(size: 13, weight: .semibold)) - .foregroundStyle(.secondary) - } - .buttonStyle(.plain) - .help("Collapse back to the dock") - } - } - - // MARK: Thread - - private var thread: some View { - ScrollViewReader { proxy in - ScrollView { - VStack(alignment: .leading, spacing: 10) { - if engine.conversation?.messages.isEmpty ?? true { - emptyHint - } - ForEach(engine.conversation?.messages ?? []) { message in - turnBubble(message) - } - liveTurnBubble - toolStepsList - stateExtras - Color.clear.frame(height: 1).id(Self.tailID) - } - .frame(maxWidth: .infinity, alignment: .leading) - .padding(.vertical, 2) - } - // Open at the END of the conversation (latest turn), not the top — an existing session with - // history lands at its most recent messages; also keeps the thread pinned to the bottom as it - // streams. The explicit scroll-to-tail below still animates the jump on each new turn. - .defaultScrollAnchor(.bottom) - .onChange(of: engine.conversation?.messages.count ?? 0) { - withAnimation(.easeOut(duration: 0.15)) { proxy.scrollTo(Self.tailID, anchor: .bottom) } - } - // Pin to the tail as the timeline streams — a superset of the old partial/thinking triggers - // (both channels append here in arrival order). - .onChange(of: engine.liveSegments) { - proxy.scrollTo(Self.tailID, anchor: .bottom) - } - } - } - - private static let tailID = "notch-conversation-tail" - - /// Whether an assistant turn is actively streaming (`.conversing`) — the live timeline entry's - /// forced-open thinking tail and "Thinking…" label key off this. - private var isTurnStreaming: Bool { - if case .conversing = engine.state { return true } - return false - } - - private var emptyHint: some View { - VStack(spacing: 6) { - Image(systemName: "text.bubble") - .font(.system(size: 24, weight: .light)) - .foregroundStyle(.secondary) - Text("Ask anything — this chat lives here at the notch until it expires or you delete it.") - .font(.system(size: 12)) - .foregroundStyle(.secondary) - .multilineTextAlignment(.center) - } - .frame(maxWidth: .infinity) - .padding(.vertical, 24) - } - - @ViewBuilder - private func turnBubble(_ message: AgentMessage) -> some View { - VStack(alignment: .leading, spacing: 3) { - HStack { - Text(message.role == .user ? "You" : message.role == .assistant ? "AI" : "Context") - .font(.system(size: 10, weight: .semibold)) - .foregroundStyle(.secondary).textCase(.uppercase) - Spacer() - if message.role == .assistant, !message.text.isEmpty { - Button { - NSPasteboard.general.clearContents() - NSPasteboard.general.setString(message.text, forType: .string) - } label: { - Image(systemName: "doc.on.doc") - .font(.system(size: 10, weight: .medium)) - .foregroundStyle(.secondary) - } - .buttonStyle(.plain) - .help("Copy this answer") - } - } - if message.role == .assistant { - // The TIMELINE (`notch-timeline-and-tuning`): the turn's persisted thinking/answer - // segments in original arrival order (or the legacy flat-thinking fallback), thinking - // blocks collapsed to compact expandable rows. - segmentTimeline(message.displaySegments, - keyPrefix: message.id.uuidString, - liveTail: false) - } else { - BidiText(text: message.text.isEmpty ? "…" : message.text, fontSize: 13) - .frame(maxWidth: .infinity, alignment: .leading) - } - } - .padding(10) - .frame(maxWidth: .infinity, alignment: .leading) - .background(RoundedRectangle(cornerRadius: 10) - .fill(message.role == .user ? Color.accentColor.opacity(0.12) : Color.white.opacity(0.05))) - } - - /// The IN-FLIGHT assistant turn as a live timeline entry: `engine.liveSegments` streaming in arrival - /// order through the SAME segment renderer the persisted messages use, so the settle handoff (live - /// entry → appended message) is seamless. Also shown while a turn is paused at an approval or has - /// failed mid-stream (the segments so far stay visible), matching the old always-visible thinking. - @ViewBuilder - private var liveTurnBubble: some View { - if isTurnStreaming || !engine.liveSegments.isEmpty { - VStack(alignment: .leading, spacing: 3) { - Text("AI") - .font(.system(size: 10, weight: .semibold)) - .foregroundStyle(.secondary).textCase(.uppercase) - if engine.liveSegments.isEmpty { - // The pre-first-token phase — in the routed path this is the tool-routing pass, a - // full non-streaming generation that can run for seconds. An honest static label - // (no repeating animation — the idle-CPU-spin rule) instead of a bare ellipsis. - Text("Choosing how to answer…") - .font(.system(size: 12)) - .foregroundStyle(.secondary) - } else { - segmentTimeline(engine.liveSegments, keyPrefix: "live", liveTail: isTurnStreaming) - } - } - .padding(10) - .frame(maxWidth: .infinity, alignment: .leading) - .background(RoundedRectangle(cornerRadius: 10).fill(Color.white.opacity(0.05))) - } - } - - /// Render one turn's ordered segments: answer segments as normal turn text, thinking segments as - /// muted per-block collapsibles. `liveTail` marks the streaming turn — its LAST thinking segment - /// (the one still growing) renders forced-open so you watch the model think, collapsing to the - /// compact row when the turn settles. - @ViewBuilder - private func segmentTimeline(_ segments: [TurnSegment], keyPrefix: String, liveTail: Bool) -> some View { - ForEach(Array(segments.enumerated()), id: \.offset) { index, segment in - switch segment.kind { - case .thinking: - thinkingBlock(segment.text, - key: "\(keyPrefix)-\(index)", - isStreaming: liveTail && index == segments.count - 1) - case .answer: - BidiText(text: segment.text.isEmpty ? "…" : segment.text, fontSize: 13) - .frame(maxWidth: .infinity, alignment: .leading) - } - } - } - - /// One collapsible thinking block in a turn's timeline (the per-block successor of the old single - /// bottom "Thinking" section). No repeating animation — streaming text itself is the motion (the - /// idle-CPU-spin rule). - private func thinkingBlock(_ text: String, key: String, isStreaming: Bool) -> some View { - let isExpanded = isStreaming || expandedThinking.contains(key) - return VStack(alignment: .leading, spacing: 6) { - Button { - if expandedThinking.contains(key) { - expandedThinking.remove(key) - } else { - expandedThinking.insert(key) - } - } label: { - HStack(spacing: 6) { - Image(systemName: "sparkles") - .font(.system(size: 11, weight: .semibold)).foregroundStyle(Color.accentColor) - Text(isStreaming ? "Thinking…" : "Thinking") - .font(.system(size: 11, weight: .medium)).foregroundStyle(.secondary) - Spacer() - Image(systemName: isExpanded ? "chevron.down" : "chevron.right") - .font(.system(size: 9, weight: .semibold)).foregroundStyle(.secondary) - } - .contentShape(Rectangle()) - } - .buttonStyle(.plain) - .disabled(isStreaming) // the live tail stays open while it streams - if isExpanded { - ScrollView { - BidiText(text: text, fontSize: 11, color: .secondaryLabelColor) - .frame(maxWidth: .infinity, alignment: .topLeading) - } - .frame(maxHeight: 110) - } - } - .padding(8) - .frame(maxWidth: .infinity, alignment: .leading) - .background(RoundedRectangle(cornerRadius: 8).fill(Color.white.opacity(0.04))) - } - - /// The in-flight assistant turn + per-state extras: the streaming bubble, the approval card with - /// Approve/Skip buttons, the bounded failed card, and the availability hint. - @ViewBuilder - private var stateExtras: some View { - switch engine.state { - case .loadingModel: - HStack(spacing: 8) { - ProgressView().controlSize(.small) - Text("Loading the model…").font(.system(size: 12)).foregroundStyle(.secondary) - } - .padding(.vertical, 4) - case .conversing: - // The live timeline entry (`liveTurnBubble`, above the tool steps) renders the streaming - // turn — thinking and answer interleaved in arrival order — so nothing renders here. - EmptyView() - case let .awaitingApproval(review): - approvalCard(review) - case let .failed(message): - failedCard(message) - case .unavailable: - VStack(alignment: .leading, spacing: 4) { - Label("AI is unavailable", systemImage: "exclamationmark.circle.fill") - .font(.system(size: 12, weight: .semibold)).foregroundStyle(.orange) - Text("Enable AI commands and download the model in the Hub, then send again.") - .font(.system(size: 11)).foregroundStyle(.secondary) - } - .padding(10) - .frame(maxWidth: .infinity, alignment: .leading) - .background(RoundedRectangle(cornerRadius: 10).fill(Color.white.opacity(0.05))) - case .idle, .awaitingTurn: - EmptyView() - } - } - - /// A paused tool step's review card — resolved by BUTTONS (the notch is the cursor world; the - /// launcher's two-finger compass is not imported here). - private func approvalCard(_ review: TaskReview) -> some View { - VStack(alignment: .leading, spacing: 10) { - if case let .action(title, fields, _) = review { - Label(title, systemImage: "exclamationmark.shield.fill") - .font(.system(size: 13, weight: .semibold)).foregroundStyle(.orange) - ForEach(Array(fields.enumerated()), id: \.offset) { _, field in - VStack(alignment: .leading, spacing: 1) { - Text(field.label).font(.system(size: 10, weight: .semibold)) - .foregroundStyle(.secondary).textCase(.uppercase) - Text(field.value).font(.system(size: 12)).foregroundStyle(.white) - .naturalTextDirection(for: field.value) - } - } - } else { - Label("This step needs your approval", systemImage: "exclamationmark.shield.fill") - .font(.system(size: 13, weight: .semibold)).foregroundStyle(.orange) - } - HStack(spacing: 8) { - Button { engine.approve() } label: { - Label("Approve", systemImage: "checkmark.circle.fill") - .font(.system(size: 12, weight: .semibold)) - } - .buttonStyle(.borderedProminent) - .controlSize(.small) - Button { engine.skip() } label: { - Label("Skip", systemImage: "arrow.uturn.forward") - .font(.system(size: 12, weight: .medium)) - } - .buttonStyle(.bordered) - .controlSize(.small) - } - } - .padding(12) - .frame(maxWidth: .infinity, alignment: .leading) - .background(RoundedRectangle(cornerRadius: 10).fill(Color.orange.opacity(0.08))) - .overlay(RoundedRectangle(cornerRadius: 10).strokeBorder(Color.orange.opacity(0.35))) - } - - /// The bounded, non-blocking failed card: a clean headline only; raw detail is not rendered here - /// (one taxonomy, one translator — the headline came through `AIError.message(for:)`). - private func failedCard(_ message: String) -> some View { - VStack(alignment: .leading, spacing: 6) { - Label("Something went wrong", systemImage: "exclamationmark.triangle.fill") - .font(.system(size: 12, weight: .semibold)).foregroundStyle(.red) - Text(message).font(.system(size: 11)).foregroundStyle(.secondary) - .lineLimit(3).truncationMode(.middle) - } - .padding(10) - .frame(maxWidth: .infinity, alignment: .leading) - .background(RoundedRectangle(cornerRadius: 10).fill(Color.red.opacity(0.06))) - } - - /// The route loop's ran tool steps — a compact status list (summary + glyph per step). - @ViewBuilder - private var toolStepsList: some View { - if !engine.toolSteps.isEmpty { - VStack(alignment: .leading, spacing: 5) { - ForEach(Array(engine.toolSteps.enumerated()), id: \.offset) { _, step in - HStack(alignment: .firstTextBaseline, spacing: 7) { - Image(systemName: Self.stepGlyph(step.status)) - .font(.system(size: 10, weight: .semibold)) - .foregroundStyle(Self.stepColor(step.status)) - Text(step.summary.isEmpty ? step.tool : step.summary) - .font(.system(size: 11)).foregroundStyle(.secondary) - .frame(maxWidth: .infinity, alignment: .leading) - .lineLimit(2).truncationMode(.middle) - } - } - } - .padding(8) - .frame(maxWidth: .infinity, alignment: .leading) - .background(RoundedRectangle(cornerRadius: 8).fill(Color.white.opacity(0.04))) - } - } - - private static func stepGlyph(_ status: ToolStepStatus) -> String { - switch status { - case .done: return "checkmark.circle.fill" - case .declined: return "minus.circle.fill" - case .failed: return "exclamationmark.triangle.fill" - case .awaitingApproval: return "exclamationmark.shield.fill" - } - } - - private static func stepColor(_ status: ToolStepStatus) -> Color { - switch status { - case .done: return .green - case .declined: return .secondary - case .failed: return .red - case .awaitingApproval: return .orange - } - } - - // MARK: Composer - - private var composer: some View { - HStack(spacing: 8) { - TextField("Message…", text: $composerText) - .textFieldStyle(.plain) - .focused($composerFocused) - .font(.system(size: 13)) - .foregroundStyle(.white) - .onSubmit(sendComposer) - .padding(.horizontal, 12).padding(.vertical, 8) - .background(RoundedRectangle(cornerRadius: 10).fill(Color.white.opacity(0.07))) - .overlay(RoundedRectangle(cornerRadius: 10) - .strokeBorder(Color.accentColor.opacity(composerFocused ? 0.5 : 0.15))) - Button(action: sendComposer) { - Image(systemName: "arrow.up.circle.fill") - .font(.system(size: 20)).foregroundStyle(Color.accentColor) - } - .buttonStyle(.plain) - .help("Send") - } - } - - private func sendComposer() { - let text = composerText.trimmingCharacters(in: .whitespacesAndNewlines) - guard !text.isEmpty else { return } - composerText = "" - engine.send(text) - } -} - -/// The in-notch SETTINGS zone (`notch-timeline-and-tuning` D5/D6): the same merged-notch panel, -/// mode-switched from the rail, hosting the ONE thinking+context slider over `NotchTuning`'s ordered -/// stops. Mouse-only on the non-activating panel (no text field lives here, so the panel never takes -/// key status in this mode); the back chevron returns to the rail. The chosen stop persists via -/// `onTuningChanged` and shapes each FOLLOWING new chat (born-with — current chats keep theirs). -struct NotchTuningSettingsView: View { - @ObservedObject var model: NotchHomeZoneViewModel - /// Attached (merged-notch, over the physical top) vs tab mode — drives the top headroom. - let isAttached: Bool - let onBack: () -> Void - let onTuningChanged: (NotchTuning) -> Void - - /// The discrete slider ↔ stop mapping: positions 0…3 in `NotchTuning` declaration order; a write - /// rounds to the nearest stop, updates the view-model, and persists through the seam. - private var sliderBinding: Binding { - Binding(get: { Double(model.tuning.sliderIndex) }, - set: { raw in - let stop = NotchTuning.fromSliderIndex(Int(raw.rounded())) - guard stop != model.tuning else { return } - model.tuning = stop - onTuningChanged(stop) - }) - } - - var body: some View { - VStack(alignment: .leading, spacing: 0) { - header - // Attached mode: the panel's top band sits behind the physical notch — keep the header - // clear of it (the expanded conversation's headroom idea). - .padding(.top, isAttached ? 34 : 10) - .padding(.horizontal, 14) - Divider().opacity(0.25).padding(.vertical, 8) - VStack(alignment: .leading, spacing: 12) { - Text("How hard new chats think — and how much they remember.") - .font(.system(size: 11)) - .foregroundStyle(.secondary) - Slider(value: sliderBinding, in: 0...Double(NotchTuning.allCases.count - 1), step: 1) - stopLabels - effectDetail - Text("Applies to each new chat you start, until you change it. Current chats keep the tuning they were born with.") - .font(.system(size: 10)) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - } - .padding(.horizontal, 14) - Spacer(minLength: 10) - } - } - - private var header: some View { - HStack(spacing: 10) { - Button(action: onBack) { - Image(systemName: "chevron.left") - .font(.system(size: 13, weight: .semibold)) - .foregroundStyle(.secondary) - } - .buttonStyle(.plain) - .help("Back to the dock") - Image(systemName: "gearshape.fill") - .font(.system(size: 14, weight: .semibold)) - .foregroundStyle(Color.accentColor) - Text("New chat tuning") - .font(.system(size: 14, weight: .semibold)) - .foregroundStyle(.white) - Spacer() - } - } - - /// The four stop titles under the slider, the selected one accented. - private var stopLabels: some View { - HStack(spacing: 0) { - ForEach(Array(NotchTuning.allCases.enumerated()), id: \.offset) { index, stop in - Text(stop.title) - .font(.system(size: 10, weight: stop == model.tuning ? .semibold : .regular)) - .foregroundStyle(stop == model.tuning ? Color.accentColor : .secondary) - if index < NotchTuning.allCases.count - 1 { Spacer() } - } - } - } - - /// What the selected stop MEANS: the thinking state + the effective context-token count (the "Max" - /// stop capped by the active model's architectural maximum). - private var effectDetail: some View { - let tuning = model.tuning - let tokens = tuning.contextTokens(modelMax: model.modelMaxContextTokens) - return HStack(spacing: 14) { - Label(tuning.reasoning ? "Thinking on" : "Thinking off", - systemImage: tuning.reasoning ? "sparkles" : "bolt.fill") - .font(.system(size: 11, weight: .medium)) - .foregroundStyle(tuning.reasoning ? Color.accentColor : .orange) - Label("\(tokens.formatted()) token context", systemImage: "square.stack.3d.up") - .font(.system(size: 11, weight: .medium)) - .foregroundStyle(.secondary) - } - } -} diff --git a/Sources/ThreeFingerSwitcher/Overlay/OverlayController.swift b/Sources/ThreeFingerSwitcher/Overlay/OverlayController.swift index e389f2a..73a914c 100644 --- a/Sources/ThreeFingerSwitcher/Overlay/OverlayController.swift +++ b/Sources/ThreeFingerSwitcher/Overlay/OverlayController.swift @@ -219,22 +219,8 @@ final class OverlayController { } } -/// NSPanel subclass that refuses key/main so it can never steal focus — EXCEPT when `keyInteractive` -/// is set, which the launcher flips on for the AI "unavailable" canvas so its Enable/Download/model -/// controls are clickable. Being a `.nonactivatingPanel`, becoming key there does not activate the app (the -/// captured front app stays frontmost); the flag is reset/destroyed when the canvas dismisses. (The Files -/// navigator never needs this — it is purely gesture-driven and stays non-key throughout.) +/// NSPanel subclass that refuses key/main so it can never steal focus. final class SwitcherPanel: NSPanel { - /// When true, the panel may become key so hosted controls (buttons/picker) receive clicks. - var keyInteractive = false - override var canBecomeKey: Bool { keyInteractive } + override var canBecomeKey: Bool { false } override var canBecomeMain: Bool { false } - - /// When true, `setFrame` is trusted verbatim — AppKit's menu-bar constraint is skipped so the panel can - /// reach the PHYSICAL top of the display and merge with the notch (the `notch-attached-park-dock` look). - /// Default off, so every other overlay keeps the standard constrain-below-the-menu-bar behavior. - var reachesPhysicalTop = false - override func constrainFrameRect(_ frameRect: NSRect, to screen: NSScreen?) -> NSRect { - reachesPhysicalTop ? frameRect : super.constrainFrameRect(frameRect, to: screen) - } } diff --git a/Sources/ThreeFingerSwitcher/Overlay/RegionPickerOverlay.swift b/Sources/ThreeFingerSwitcher/Overlay/RegionPickerOverlay.swift deleted file mode 100644 index fc10fed..0000000 --- a/Sources/ThreeFingerSwitcher/Overlay/RegionPickerOverlay.swift +++ /dev/null @@ -1,138 +0,0 @@ -import AppKit - -/// Owns the **mouse-interactive** screen-region picker overlay (spec `screen-region-picker`): a -/// full-screen, dimmed, crosshair surface over the revealed desktop on which the user drags a rectangle -/// to capture. Like `DockPreviewOverlayController` it is the rare overlay that does NOT ignore the mouse; -/// it is still a `.nonactivatingPanel` and never becomes key/main (no keyboard — cancellation is a -/// click-without-drag, never Esc), so the captured front app stays frontmost. Teardown is **synchronous** -/// (`orderOut` + `close`) — the files-band / Dock ghost-on-Space-switch landmine applies here too. -/// -/// The pure geometry + the click-vs-drag verdict live in `RegionPickerModel`; this controller only wires -/// AppKit mouse events to it and renders. v1 scopes a pick to a single display: the screen under the -/// cursor at show time (falling back to the main screen). -@MainActor -final class RegionPickerOverlay { - private var panel: NSPanel? - - /// Resolved once per session: a designated rectangle (Cocoa global, bottom-left) to capture, or a - /// cancel (a click without a drag). The overlay is torn down before this fires. - private var onResolve: ((RegionPickerModel.Resolution) -> Void)? - - var isVisible: Bool { panel?.isVisible ?? false } - - /// Present the picker over the screen under the cursor and call `onResolve` exactly once when the drag - /// resolves. The overlay orders itself out (synchronously) before invoking the callback, so a capture - /// that fronts a window / a Space switch never carries the dimming surface along. - func show(onResolve: @escaping (RegionPickerModel.Resolution) -> Void) { - self.onResolve = onResolve - let screen = Self.screenUnderCursor() - let panel = makePanel(on: screen) - self.panel = panel - panel.setFrame(screen.frame, display: true) - panel.orderFrontRegardless() - } - - /// Synchronous teardown (no deferred close — Space-switch ghost landmine). Idempotent. - func hide() { - panel?.orderOut(nil) - panel?.close() - panel = nil - } - - private func makePanel(on screen: NSScreen) -> NSPanel { - let panel = NSPanel( - contentRect: screen.frame, - styleMask: [.borderless, .nonactivatingPanel], - backing: .buffered, - defer: false - ) - panel.isOpaque = false - panel.backgroundColor = .clear - panel.hasShadow = false - panel.ignoresMouseEvents = false // the picker takes the pointer (drag to select) - panel.acceptsMouseMovedEvents = true - panel.isFloatingPanel = true - panel.hidesOnDeactivate = false - panel.level = .popUpMenu // above normal windows, over the revealed desktop - panel.collectionBehavior = [.canJoinAllSpaces, .fullScreenAuxiliary, .ignoresCycle] - let canvas = RegionPickerCanvas(frame: NSRect(origin: .zero, size: screen.frame.size)) - canvas.onResolve = { [weak self] resolution in self?.resolve(resolution) } - panel.contentView = canvas - return panel - } - - /// Tear down the overlay FIRST (synchronous), then deliver the resolution — so a capture that fronts a - /// window never grabs the dimming surface, and the front app is restored before the callback runs. - private func resolve(_ resolution: RegionPickerModel.Resolution) { - let callback = onResolve - onResolve = nil - hide() - callback?(resolution) - } - - /// The screen under the cursor at show time (single-display v1), falling back to the main screen. - private static func screenUnderCursor() -> NSScreen { - let mouse = NSEvent.mouseLocation - return NSScreen.screens.first { $0.frame.contains(mouse) } ?? NSScreen.main ?? NSScreen.screens[0] - } -} - -/// The drawing + mouse-tracking surface inside the picker panel. Non-flipped (default bottom-left), so its -/// coordinates align with Cocoa screen handedness; it feeds window-space points to a `RegionPickerModel` -/// and converts the resolved rectangle to **global screen** coordinates once, at mouse-up. -private final class RegionPickerCanvas: NSView { - var onResolve: ((RegionPickerModel.Resolution) -> Void)? - private var model = RegionPickerModel() - - override var acceptsFirstResponder: Bool { true } - /// Take the very first click without requiring the panel to activate first (it never becomes key). - override func acceptsFirstMouse(for event: NSEvent?) -> Bool { true } - - override func resetCursorRects() { - addCursorRect(bounds, cursor: .crosshair) // "select mode" cursor over the whole surface - } - - override func mouseDown(with event: NSEvent) { - model.begin(at: event.locationInWindow) - needsDisplay = true - } - - override func mouseDragged(with event: NSEvent) { - model.drag(to: event.locationInWindow) - needsDisplay = true - } - - override func mouseUp(with event: NSEvent) { - let resolution = model.end(at: event.locationInWindow) - needsDisplay = true - switch resolution { - case .cancel: - onResolve?(.cancel) - case let .region(windowRect): - onResolve?(.region(rectToScreen(windowRect))) - } - } - - /// Convert a rectangle in window coordinates to global screen coordinates (both bottom-left). The - /// content view fills the window at the origin, so window coords == view coords; only the origin needs - /// the window→screen hop (size is translation-invariant). - private func rectToScreen(_ windowRect: CGRect) -> CGRect { - guard let window else { return windowRect } - let origin = window.convertPoint(toScreen: windowRect.origin) - return CGRect(origin: origin, size: windowRect.size) - } - - override func draw(_ dirtyRect: NSRect) { - // Dim the whole surface so the revealed desktop reads as "pick a region" mode. - NSColor.black.withAlphaComponent(0.28).setFill() - bounds.fill() - guard let sel = model.liveRect, sel.width > 0, sel.height > 0 else { return } - // Punch the selection clear (reveal the live desktop through the window) and outline it. - NSColor.clear.set() - sel.fill(using: .copy) - NSColor.controlAccentColor.setStroke() - let outline = NSBezierPath(rect: sel) - outline.lineWidth = 2 - outline.stroke() - } -} diff --git a/Sources/ThreeFingerSwitcher/RegionPicker/RegionPickerModel.swift b/Sources/ThreeFingerSwitcher/RegionPicker/RegionPickerModel.swift deleted file mode 100644 index 1ba2fd3..0000000 --- a/Sources/ThreeFingerSwitcher/RegionPicker/RegionPickerModel.swift +++ /dev/null @@ -1,69 +0,0 @@ -import CoreGraphics - -/// The pure brain of the interactive screen-region picker (spec `screen-region-picker`): a synchronous, -/// AppKit-free state machine that turns a press → drag → release into either a designated rectangle to -/// capture or a **cancel**. The AppKit `RegionPickerOverlay` feeds it mouse events and renders its -/// `liveRect`; keeping the geometry and the click-vs-drag verdict here makes both unit-testable headless -/// (no overlay, no ScreenCaptureKit), mirroring how `DockHoverModel` is the pure brain under the Dock -/// preview overlay. -/// -/// Coordinates are **Cocoa global (bottom-left origin)** throughout — the overlay converts at the -/// boundary so the model never juggles handedness, matching the project convention (`DockHoverModel`). -struct RegionPickerModel: Equatable { - - /// Below this straight-line drag distance (in points) a release is a **click, not a drag** → cancel. - /// This is the single "click-without-drag cancels" threshold (spec): a press-and-release that barely - /// moves defuses the picker, matching the ⌘⇧4 muscle memory of pressing without dragging. Small - /// enough that any deliberate drag clears it, large enough that an unsteady click does not. - static let minDragDistance: CGFloat = 6 - - /// The press anchor; nil when no drag is in progress. - private(set) var origin: CGPoint? - /// The latest pointer position during a drag; nil when no drag is in progress. - private(set) var current: CGPoint? - - /// The resolution of a completed pick. - enum Resolution: Equatable { - /// Capture this rectangle (Cocoa global coords). - case region(CGRect) - /// A click without a drag — defuse the picker, capture nothing. - case cancel - } - - /// The live selection rectangle to draw while dragging, or nil when no drag is in progress. - var liveRect: CGRect? { - guard let origin, let current else { return nil } - return Self.rect(from: origin, to: current) - } - - /// Begin a drag at `point` (mouse-down): anchor the origin. - mutating func begin(at point: CGPoint) { - origin = point - current = point - } - - /// Track the pointer during a drag (mouse-dragged). A no-op if no drag is in progress. - mutating func drag(to point: CGPoint) { - guard origin != nil else { return } - current = point - } - - /// Resolve the pick at `point` (mouse-up) and reset for the next drag. A release whose straight-line - /// travel from the origin is below `minDragDistance` is a **cancel** (a click); otherwise the - /// designated rectangle. A release with no in-progress drag is also a cancel. - mutating func end(at point: CGPoint) -> Resolution { - defer { origin = nil; current = nil } - guard let origin else { return .cancel } - let dx = point.x - origin.x, dy = point.y - origin.y - if (dx * dx + dy * dy).squareRoot() < Self.minDragDistance { - return .cancel // pressed without dragging → defuse, capture nothing - } - return .region(Self.rect(from: origin, to: point)) - } - - /// The normalized rectangle spanned by two points (handles any drag direction). - static func rect(from a: CGPoint, to b: CGPoint) -> CGRect { - CGRect(x: min(a.x, b.x), y: min(a.y, b.y), - width: abs(a.x - b.x), height: abs(a.y - b.y)) - } -} diff --git a/Sources/ThreeFingerSwitcher/Settings/AppDataReset.swift b/Sources/ThreeFingerSwitcher/Settings/AppDataReset.swift index 3a38b72..8da733d 100644 --- a/Sources/ThreeFingerSwitcher/Settings/AppDataReset.swift +++ b/Sources/ThreeFingerSwitcher/Settings/AppDataReset.swift @@ -11,8 +11,6 @@ struct DangerZoneSelection: OptionSet, Equatable { static let appData = DangerZoneSelection(rawValue: 1 << 0) /// ~/Library/Caches/ and ~/Library/HTTPStorages/. static let caches = DangerZoneSelection(rawValue: 1 << 1) - /// The multi-GB weights directory (the AI opt-in is turned off first by the coordinator). - static let aiModels = DangerZoneSelection(rawValue: 1 << 2) /// `tccutil reset` for every service the app can hold. static let permissions = DangerZoneSelection(rawValue: 1 << 3) } @@ -24,18 +22,14 @@ struct DangerZoneSelection: OptionSet, Equatable { final class AppDataReset { /// Every TCC service this app can hold a grant for, in `tccutil reset` spelling. static let tccServices = [ - "Accessibility", // window enumeration / raising / selection I/O - "ScreenCapture", // thumbnails + vision input + "Accessibility", // window enumeration / raising + "ScreenCapture", // thumbnails "ListenEvent", // Input Monitoring (usually never granted) - "AppleEvents", // the per-site keyboard "allow browser control" reader - "Calendar", // AI tasks (lazy) - "Reminders", - "AddressBook" // Contacts + "AppleEvents" // the per-site keyboard "allow browser control" reader ] /// What to delete for a selection: directories removed whole, plus directories whose CONTENTS - /// are removed except named survivors (the App-data/AI-models split: App data keeps `models/` - /// unless AI models is also selected, in which case the whole root goes). + /// are removed except named survivors. struct FilesystemTargets: Equatable { var removeWhole: [URL] = [] var removeContentsExcept: [(directory: URL, keep: Set)] = [] @@ -48,24 +42,16 @@ final class AppDataReset { } /// Pure: the filesystem footprint for a selection. `library` is `~/Library`; the Application - /// Support root is the app's `ThreeFingerSwitcher` directory (clipboard, projects, models). + /// Support root is the app's `ThreeFingerSwitcher` directory (clipboard, projects). nonisolated static func filesystemTargets(for selection: DangerZoneSelection, library: URL, bundleID: String) -> FilesystemTargets { var targets = FilesystemTargets() let appSupportRoot = library.appendingPathComponent("Application Support/ThreeFingerSwitcher", isDirectory: true) - let modelsDir = appSupportRoot.appendingPathComponent("models", isDirectory: true) - switch (selection.contains(.appData), selection.contains(.aiModels)) { - case (true, true): + if selection.contains(.appData) { targets.removeWhole.append(appSupportRoot) - case (true, false): - targets.removeContentsExcept.append((appSupportRoot, ["models"])) - case (false, true): - targets.removeWhole.append(modelsDir) - case (false, false): - break } if selection.contains(.appData) { targets.removeWhole.append( diff --git a/Sources/ThreeFingerSwitcher/Settings/AppSettings.swift b/Sources/ThreeFingerSwitcher/Settings/AppSettings.swift index 790d8da..30654ef 100644 --- a/Sources/ThreeFingerSwitcher/Settings/AppSettings.swift +++ b/Sources/ThreeFingerSwitcher/Settings/AppSettings.swift @@ -1,65 +1,6 @@ import Foundation import Combine -// MARK: - Files-band tunable enums - -/// Which entry field the Files-band column sorts on. The list's secondary tiebreak (a stable name -/// compare) lives in the lister, so this only names the primary key. Persisted by `rawValue`. -enum FilesSortField: String, Codable, CaseIterable, Identifiable { - case name // case-insensitive display name - case date // last content-modification date - case kind // coarse `FileKind`, then name - var id: String { rawValue } -} - -/// Ascending vs. descending for the Files-band sort. Kept separate from the field so any field can -/// be flipped without multiplying the field enum. Persisted by `rawValue`. -enum FilesSortDirection: String, Codable, CaseIterable, Identifiable { - case ascending - case descending - var id: String { rawValue } -} - -/// Row height / padding of the Files-band current column — how tightly rows pack. Persisted by -/// `rawValue`; the concrete point metrics for each case live in the view layer. -enum FilesDensity: String, Codable, CaseIterable, Identifiable { - case compact - case comfortable - case spacious - var id: String { rawValue } -} - -/// Whether a Files-band row leads with the file's plain type icon or a live QuickLook preview -/// thumbnail. (The dedicated preview pane is separate; this governs the per-row leading glyph.) -/// Persisted by `rawValue`. -enum FilesIconStyle: String, Codable, CaseIterable, Identifiable { - case icon // the file/folder type icon (cheap, no QuickLook) - case preview // a QuickLook thumbnail when one is available, icon fallback - var id: String { rawValue } -} - -/// The default-open action committed for a highlighted **file** (a folder always opens as a Finder -/// window per the spec, regardless of this). `defaultApp` opens in the system default app; -/// `openWith` lands on the Open-With chooser instead of launching immediately. Persisted by `rawValue`. -enum FilesDefaultOpen: String, Codable, CaseIterable, Identifiable { - case defaultApp // open in the file's default application - case openWith // present the Open-With chooser instead of launching - var id: String { rawValue } -} - -/// Which secondary metadata a Files-band row shows beside its name. An `OptionSet` (mirroring -/// `DangerZoneSelection`) so several can show at once; persisted as the `Int` `rawValue`. -struct FilesRowMetadata: OptionSet, Equatable { - let rawValue: Int - - /// Show the last-modified date. - static let date = FilesRowMetadata(rawValue: 1 << 0) - /// Show the coarse kind label (e.g. "Folder", "Image"). - static let kind = FilesRowMetadata(rawValue: 1 << 1) - /// Show the file size (folders show item count in the view layer). - static let size = FilesRowMetadata(rawValue: 1 << 2) -} - /// Tunable parameters for the gesture, persisted in UserDefaults and applied live. /// All distance values are in *normalized* trackpad units (0..1 across the surface), /// since OpenMultitouchSupport reports normalized positions. @@ -84,12 +25,11 @@ final class AppSettings: ObservableObject { /// Wrap around at the ends of the list instead of clamping. @Published var wrapAtEnds: Bool { didSet { defaults.set(wrapAtEnds, forKey: Keys.wrapAtEnds) } } - /// The user's resolution-gesture bindings for the three remappable surfaces (the AI canvas resolve, - /// the Files-drill resolution, and the switcher per-axis scrub directions). Persisted as a single - /// JSON blob (`Data` is plist-native, mirroring `FavoritesStore`) so the per-surface vocabularies - /// stay co-versioned. Defaults to exactly today's behavior. The switcher axes are the single source - /// of truth for the former `reverseDirection` / `reverseVerticalDirection` booleans, which are now - /// computed accessors onto this binding (no duplicate persisted keys). + /// The user's gesture bindings (the switcher per-axis scrub directions). Persisted as a single + /// JSON blob (`Data` is plist-native, mirroring `FavoritesStore`). Defaults to exactly today's + /// behavior. The switcher axes are the single source of truth for the former `reverseDirection` / + /// `reverseVerticalDirection` booleans, which are now computed accessors onto this binding (no + /// duplicate persisted keys). @Published var gestureBindings: GestureBindings { didSet { persist(gestureBindings, Keys.gestureBindings) } } /// Invert slide direction (slide right → previous instead of next). A computed view onto the @@ -167,8 +107,7 @@ final class AppSettings: ObservableObject { @Published var launcherActivationThreshold: Double { didSet { persist(launcherActivationThreshold, Keys.launcherActivationThreshold) } } /// Normalized centroid travel to move the selection by one item — horizontally between items and - /// vertically between grid rows (odometer, with carry). Also drives the Files navigator's depth / - /// highlight stepping. + /// vertically between grid rows (odometer, with carry). @Published var launcherStepDistance: Double { didSet { persist(launcherStepDistance, Keys.launcherStepDistance) } } /// Normalized vertical travel on the band list to switch one band (odometer, with carry). Coarser @@ -225,295 +164,6 @@ final class AppSettings: ObservableObject { /// Bundle ids whose copies are never recorded (e.g. password managers the user wants excluded). @Published var clipboardExcludedApps: [String] { didSet { defaults.set(clipboardExcludedApps, forKey: Keys.clipboardExcludedApps) } } - // MARK: - AI commands (opt-in; default OFF) - - /// Opt-in to the AI command band and the on-device model. Unlike the Space-row / launcher opt-ins - /// this relocates NO native gesture and needs NO re-login; unlike the clipboard opt-in, turning it - /// ON does allow the (later) multi-gigabyte model download + residency, and the first calendar task - /// will request the Calendar permission lazily. Default OFF — set only via explicit consent. - /// Older settings that predate this feature have no key and decode with the opt-in OFF, leaving the - /// band absent, nothing downloaded, and no commands surfaced. - @Published var aiCommandsEnabled: Bool { didSet { defaults.set(aiCommandsEnabled, forKey: Keys.aiCommandsEnabled) } } - - /// The pinned on-device model id the model-management surface selects, or nil for "registry - /// default". Stored so a deliberate model choice survives across launches; nil encodes as absent, - /// so older settings (and a never-chosen default) read back identically. - @Published var aiSelectedModelID: String? { didSet { defaults.set(aiSelectedModelID, forKey: Keys.aiSelectedModelID) } } - - /// The fleet roster's pinned ACTIVE CHAT model id (the radio AMONG chat-role members), or nil for the - /// registry/fleet chat default. Distinct from the capability toggles below: a chat model is the single - /// resident conversational brain, so it is a radio choice. nil encodes as absent (legacy-safe). - @Published var aiSelectedChatModelID: String? { - didSet { defaults.set(aiSelectedChatModelID, forKey: Keys.aiSelectedChatModelID) } - } - - /// The set of ENABLED capability-model ids (image / ternary / video) — INDEPENDENT toggles, NOT a - /// radio, because these co-reside with (or are evicted around) the chat model rather than replacing - /// it. Enabling one inserts its id here AND triggers its download if not already on disk. Persisted as - /// a string array; absent ⇒ empty (legacy-safe — pre-wave settings read back with nothing enabled). - @Published var aiEnabledCapabilityModelIDs: Set { - didSet { defaults.set(Array(aiEnabledCapabilityModelIDs), forKey: Keys.aiEnabledCapabilityModelIDs) } - } - - // MARK: - Release Full Potential (master gate + sub-flags; default OFF — addendum §D1) - - /// The master **Release Full Potential** opt-in (`ai-full-potential-toggle`, addendum §D1). Default - /// OFF — V2.5 ships calm; this one deliberate, user-owned act lights up the heavy AI fleet. Like the - /// clipboard/device-link opt-ins it relocates NO native gesture, needs NO re-login, and requests NO - /// new permission; it takes effect immediately. Turning it OFF closes every sub-capability gate at - /// once (the calm panic-off) WITHOUT zeroing the sub-flags below — re-arming restores the prior - /// selection (the gate relocks by computation, never by mutating the stored flags). Older settings - /// have no key and decode with the master OFF. Preserved by `resetToDefaults` like the other AI opt-ins. - @Published var fullPotentialEnabled: Bool { didSet { defaults.set(fullPotentialEnabled, forKey: Keys.fullPotentialEnabled) } } - - /// Sub-flag: the CPU ternary lane (`ai-compute-tiers`). Gated under the master. Cost: heat/battery — - /// a second (CPU) lane runs concurrently; short structured bursts only, CPU per-token is slower. - /// Default OFF; persisted; preserved by reset. - @Published var cpuLaneEnabled: Bool { didSet { defaults.set(cpuLaneEnabled, forKey: Keys.cpuLaneEnabled) } } - - /// Sub-flag: the K-stream GPU batched runtime + growable context (`ai-batched-runtime-and-context`). - /// Gated under the master. Cost: RAM + latency — multiplexes K sessions over one weight read; larger - /// context = more resident KV; latency rises under load. Default OFF; persisted; preserved by reset. - @Published var batchedRuntimeEnabled: Bool { didSet { defaults.set(batchedRuntimeEnabled, forKey: Keys.batchedRuntimeEnabled) } } - - /// Sub-flag: image/video generation tools (`ai-media-runtime` + backends). Gated under the master. - /// Cost: RAM (eviction) + latency + disk — a heavy gen EVICTS chat ("the assistant goes quiet while - /// it paints"); minutes per clip; tens of GB of weights. Default OFF; persisted; preserved by reset. - @Published var mediaGenEnabled: Bool { didSet { defaults.set(mediaGenEnabled, forKey: Keys.mediaGenEnabled) } } - - /// Sub-flag: parked auto-vs-escalate + whitelist + audit (`ai-background-autonomy`). Gated under the - /// master. Cost: unattended action — the agent may act while you are away (whitelisted/contained - /// writes only; dangerous ones still escalate; all audited). Default OFF; persisted; preserved by reset. - @Published var backgroundAutonomyEnabled: Bool { didSet { defaults.set(backgroundAutonomyEnabled, forKey: Keys.backgroundAutonomyEnabled) } } - - /// Sub-flag: cloud fleet members — Claude / GLM-5.2 (`ai-model-fleet` cloud members). Gated under the - /// master. Cost: $ + network + data off-device — sends prompts to a paid cloud model; budget-capped + - /// audited; off until armed. Default OFF; persisted; preserved by reset. - @Published var fleetCloudEscalationEnabled: Bool { didSet { defaults.set(fleetCloudEscalationEnabled, forKey: Keys.fleetCloudEscalationEnabled) } } - - /// The pure `FullPotentialFlags` the gate consumes, assembled from the six persisted keys plus the - /// existing AI-commands opt-in (consumed verbatim — the AI-commands opt-in is owned by - /// `tunable-settings`, not redefined here). Read at consult time so a flag edit live-applies. - var fullPotentialFlags: FullPotentialFlags { - FullPotentialFlags(aiCommandsEnabled: aiCommandsEnabled, - fullPotentialEnabled: fullPotentialEnabled, - cpuLane: cpuLaneEnabled, - batchedRuntime: batchedRuntimeEnabled, - mediaGen: mediaGenEnabled, - backgroundAutonomy: backgroundAutonomyEnabled, - fleetCloud: fleetCloudEscalationEnabled) - } - - /// The Full Potential gate every heavy slice consults via one `isUnlocked(_:)` check before - /// activating. A computed convenience over `fullPotentialFlags` — pure, total, never throws. - var fullPotentialGate: FullPotentialGate { - FullPotentialGate(flags: fullPotentialFlags) - } - - /// The selected VIDEO backend behind the `MediaRuntime` seam (`ai-video-animation-generation`, - /// addendum §1). `.cloud` (the honest default — a hosted API, nothing downloaded) or `.localLTXV` - /// (the frontier — 35 GB+, gated by the master toggle). Persisted by `rawValue`; an unreadable / - /// absent value reads back as the calm `.cloud` default. The master-gate validity of `.localLTXV` - /// is enforced at selection by `VideoProvider.isSelectable(...)`, not here. - @Published var videoProvider: VideoProvider { didSet { defaults.set(videoProvider.rawValue, forKey: Keys.videoProvider) } } - - /// The per-rolling-24h CLOUD-video budget cap (`ai-video-animation-generation`, addendum §1). Cloud - /// video spends real money + uploads bytes, so it is `.dangerous` AND rate-capped: the `VideoBudget` - /// admits at most this many cloud generations per rolling-24h window (NOT a calendar reset). A - /// conservative default so an autonomous loop physically cannot rack up spend before the user - /// deliberately raises it; 0 effectively disables cloud video. - @Published var mediaVideoBudgetPerDay: Int { didSet { defaults.set(mediaVideoBudgetPerDay, forKey: Keys.mediaVideoBudgetPerDay) } } - - /// Per-command remembered runtime-parameter language (spec: "Per-command runtime-parameter - /// persistence"), keyed by the command's identifier string → the last chosen language. Out-of-band - /// from the command itself, so seeds/catalog/band edits are unaffected; orphan keys (deleted - /// commands) are harmless and pruned opportunistically. Stored as a `[String: String]` dictionary. - @Published var aiCommandLanguages: [String: String] { didSet { defaults.set(aiCommandLanguages, forKey: Keys.aiCommandLanguages) } } - - /// Let the on-device model reason before answering (thinking is filtered from the result). Default - /// ON; gated behind the AI opt-in like the other AI prefs. - @Published var aiReasoningEnabled: Bool { didSet { defaults.set(aiReasoningEnabled, forKey: Keys.aiReasoningEnabled) } } - - /// The agent's context-size preset (`ai-batched-runtime-and-context`, design D5). Drives - /// `agentContextTokens`; longer context trades background concurrency + speed for recall (the Hub - /// surfaces the RAM/stream cost). Default Balanced. - @Published var agentContextPreset: AgentContextPreset { didSet { defaults.set(agentContextPreset.rawValue, forKey: Keys.agentContextPreset) } } - /// The resolved context-token budget (clamped to the model max). Feeds conversation-runtime - /// compaction through the injected `ContextBudgetProviding`, so growing it raises the compaction - /// trigger and the two never disagree about "the budget." - @Published var agentContextTokens: Int { didSet { defaults.set(agentContextTokens, forKey: Keys.agentContextTokens) } } - /// Compact long contexts with 8-bit KV cache — a longer context fits the same RAM at a small quality - /// cost (design D6). Default OFF. - @Published var agentCompactKV: Bool { didSet { defaults.set(agentCompactKV, forKey: Keys.agentCompactKV) } } - - /// The NOTCH chat's own thinking+context dial (`notch-timeline-and-tuning`): the in-notch settings - /// zone's slider stop, applied to each conversation BORN at the notch after the change (snapshot at - /// birth, carried for life). Deliberately separate from the global `aiReasoningEnabled` / - /// `agentContextPreset`, which keep steering the launcher band and other agent surfaces. - @Published var notchTuning: NotchTuning { didSet { defaults.set(notchTuning.rawValue, forKey: Keys.notchTuning) } } - - /// Idle-TTL for the RESIDENT MODEL WEIGHTS (`model-idle-ttl-and-memory-pressure`): after the AI - /// system has been fully quiescent this many minutes, the ~17 GB resident model is evicted from - /// memory (weights stay on disk; the next request transparently lazy-reloads). `0` = never — the - /// pre-change keep-forever behavior. Memory-pressure eviction is NOT a setting (always armed). - @Published var aiIdleEvictMinutes: Int { didSet { defaults.set(aiIdleEvictMinutes, forKey: Keys.aiIdleEvictMinutes) } } - - /// Voice conversation opt-in (`add-voice-computer-use-agent`): push-to-talk with the agent. - /// Default OFF; requires macOS 26 (SpeechAnalyzer) and — on first actual press — the microphone - /// permission (the app's first new TCC grant; the mic opens ONLY while push-to-talk is held). - @Published var voiceConversationEnabled: Bool { didSet { defaults.set(voiceConversationEnabled, forKey: Keys.voiceConversationEnabled) } } - /// The push-to-talk hold key (default 61 = Right Option — a bare modifier, no typing collision). - @Published var voicePTTKeyCode: Int { didSet { defaults.set(voicePTTKeyCode, forKey: Keys.voicePTTKeyCode) } } - /// Computer-use opt-in (`add-voice-computer-use-agent`): the agent may read windows and — behind - /// the approval gate — click and type in them. Default OFF; reuses the existing Accessibility - /// grant (no new permission). A human trackpad touch always aborts an in-flight act. - @Published var computerUseEnabled: Bool { didSet { defaults.set(computerUseEnabled, forKey: Keys.computerUseEnabled) } } - /// Per-step wall-clock timeout for non-gated agent tool steps, seconds (design D8). - @Published var agentStepTimeoutSeconds: Int { didSet { defaults.set(agentStepTimeoutSeconds, forKey: Keys.agentStepTimeoutSeconds) } } - /// Per-turn active wall-clock deadline for the agent loop, seconds (human approval time exempt). - @Published var agentTurnDeadlineSeconds: Int { didSet { defaults.set(agentTurnDeadlineSeconds, forKey: Keys.agentTurnDeadlineSeconds) } } - - /// Soft target for the parked-session set (`ai-parked-sessions`, design §7). When exceeded, the - /// least-recently-updated IDLE session is evicted (never an active/needs-you/thinking one). - @Published var agentMaxParkedSessions: Int { didSet { defaults.set(agentMaxParkedSessions, forKey: Keys.agentMaxParkedSessions) } } - /// How long a parked session may stay idle before it summarizes-and-sleeps (drops its KV cache, keeps - /// a one-line resume). Seconds. RETIRED by design D1 (no live consumer) — kept only so an existing - /// stored value migrates cleanly; the live aging behavior is now `agentParkAutoDismissCountdown`. - @Published var agentParkIdleTimeout: TimeInterval { didSet { defaults.set(agentParkIdleTimeout, forKey: Keys.agentParkIdleTimeout) } } - /// The OPT-IN auto-dismiss countdown (`refactor-park-and-background-agents`): an idle, fully-seen - /// session untouched for this long is dismissed forever through the authoritative discard path. - /// Seconds; **0 = never (the default)** — chats persist until the user deletes them (or the - /// max-parked eviction bounds the rail). Unseen results always protect a session from expiry. - @Published var agentParkAutoDismissCountdown: TimeInterval { didSet { defaults.set(agentParkAutoDismissCountdown, forKey: Keys.agentParkAutoDismissCountdown) } } - /// The two-finger UP excursion (normalized) past the canvas bottom that parks the conversation. Sits - /// ABOVE the incidental scroll threshold (`canvasResolveThreshold`) so a normal scroll-to-bottom - /// never parks. - @Published var agentOverscrollParkThreshold: Double { didSet { defaults.set(agentOverscrollParkThreshold, forKey: Keys.agentOverscrollParkThreshold) } } - /// A small **dwell** the cursor must remain crossed behind the notch before the parked-session dock - /// reveals (`ai-parked-sessions`) — so a quick pass THROUGH the notch (reaching for the menu bar, or - /// travelling to another corner) doesn't pop the dock. Seconds; `0` reveals immediately. Default 0.3. - @Published var agentNotchRevealDwell: TimeInterval { didSet { defaults.set(agentNotchRevealDwell, forKey: Keys.agentNotchRevealDwell) } } - /// Peak smoothed centroid velocity (normalized units/sec) a canvas excursion must reach before its - /// lift counts as a FLICK (commit/park) rather than a slow reading-scroll (D4). A sub-threshold peak — - /// or fingers held down without a prompt lift — is SCROLL and never resolves the canvas. Run-verify - /// tuning (real trackpad EMA + frame cadence). - @Published var flickVelocityThreshold: Double { didSet { defaults.set(flickVelocityThreshold, forKey: Keys.flickVelocityThreshold) } } - /// Maximum delay (seconds) between the last high-velocity in-contact frame and the lift for that lift to - /// count as a flick. A longer pause before lifting means the fingers decelerated to a scroll/hold, so it - /// is NOT a flick (D4). Run-verify tuning. - @Published var flickLiftWindow: Double { didSet { defaults.set(flickLiftWindow, forKey: Keys.flickLiftWindow) } } - - // MARK: - Background autonomy whitelist (ai-background-autonomy; default EMPTY) - - /// The user's trusted folder path prefixes — a parked agent may auto-run a `confirm` write whose - /// target standardizes under one of these (component-boundary match). Default empty: a fresh install - /// trusts nothing arbitrary (the app's own memory/project stores are CONTAINED and auto WITHOUT a - /// whitelist row). A privacy/trust choice, so it is NOT cleared by `resetToDefaults` (mirrors the - /// AI/clipboard/files opt-in handling). - @Published var agentWhitelistPaths: [String] { didSet { defaults.set(agentWhitelistPaths, forKey: Keys.agentWhitelistPaths) } } - - /// The user's trusted command patterns (anchored `*`/`?` globs against a tool/Shortcut name or a - /// shell `argv[0]`). Default empty; NOT cleared by `resetToDefaults` (same trust-choice rationale). - @Published var agentWhitelistCommands: [String] { didSet { defaults.set(agentWhitelistCommands, forKey: Keys.agentWhitelistCommands) } } - - /// The pure `Whitelist` value the routing loop's `BackgroundPolicyResolver` consumes — assembled from - /// the two persisted lists. Read at resolution time so edits live-apply on the next step. - var agentWhitelist: Whitelist { - Whitelist(trustedPathPrefixes: agentWhitelistPaths, trustedCommandPatterns: agentWhitelistCommands) - } - - // MARK: - Files band (opt-in; default OFF) - - /// Opt-in to the launcher's Files band — a local-only Finder-mimic column navigator. Like the - /// clipboard opt-in (and unlike the gesture opt-ins) this relocates no native gesture, needs no - /// re-login, and requests no new permission; it reads the local filesystem on demand. There is NO - /// `is…Effective` gate — the flip takes effect immediately: ON injects the band on the next launcher - /// open, OFF removes it. Default OFF. Older settings have no key and decode with the opt-in OFF. - @Published var filesBandEnabled: Bool { didSet { defaults.set(filesBandEnabled, forKey: Keys.filesBandEnabled) } } - - /// The user-configured **local** root folders the Files band opens onto (its entry column), as - /// standardized absolute paths in display order. Stored as `[String]` paths (mirroring - /// `clipboardExcludedApps`) because `AppSettings` persists only plist-native primitives and there is - /// no security-scoped-bookmark precedent in this app; the Hub roots editor rejects network/iCloud - /// locations at the boundary. Empty by default (the Hub seeds a sensible set on first configuration). - @Published var filesRoots: [String] { didSet { defaults.set(filesRoots, forKey: Keys.filesRoots) } } - - /// Per-root remembered deepest location: a `root path → last deepest path` map so each root restores - /// where the user left off (spec: "A root remembers where you left off"). Stored as a `[String: String]` - /// exactly like `aiCommandLanguages`; orphan keys (a removed root) are harmless and pruned - /// opportunistically. Use `rememberedLocation(forRoot:)` / `rememberLocation(_:forRoot:)`. - @Published var filesRememberedLocations: [String: String] { didSet { defaults.set(filesRememberedLocations, forKey: Keys.filesRememberedLocations) } } - - /// Whether the Files band reopens **displaying the last folder you were in** rather than the roots - /// list. When ON (default) the band opens straight onto the remembered deepest location of a configured - /// root — restored AT OPEN, so the main column already shows that folder while the highlight is still on - /// the band icon, and crossing into the column lands you exactly there with no jump. When OFF the band - /// opens fresh on the roots list. The per-root remembered map (`filesRememberedLocations`) keeps tracking - /// either way; this toggle only governs whether init consults it. Older settings decode with it ON. - @Published var filesRememberLocation: Bool { didSet { defaults.set(filesRememberLocation, forKey: Keys.filesRememberLocation) } } - - /// Width of the Files band's current-list column, in points. Drives the bounded overlay width together - /// with the thin ancestor icon-rail and the preview pane. - @Published var filesColumnWidth: Double { didSet { persist(filesColumnWidth, Keys.filesColumnWidth) } } - - /// How tightly the current column's rows pack (row height / padding). Persisted by `rawValue`. - @Published var filesDensity: FilesDensity { didSet { defaults.set(filesDensity.rawValue, forKey: Keys.filesDensity) } } - - /// The Files band's accent tint, stored as a `#RRGGBB` hex string (plist-native, single-property — - /// matching how `AppSettings` persists every other setting; the band builder/view resolve it to the - /// codebase's `ItemColor`/SwiftUI `Color` at their boundary). A synthetic band, so this is the one - /// place its tint is configured (it has no entry in the authored bands store). - @Published var filesBandTint: String { didSet { defaults.set(filesBandTint, forKey: Keys.filesBandTint) } } - - /// Whether a row leads with the plain type icon or a live QuickLook preview thumbnail. Persisted by - /// `rawValue`. - @Published var filesIconStyle: FilesIconStyle { didSet { defaults.set(filesIconStyle.rawValue, forKey: Keys.filesIconStyle) } } - - /// Primary sort key for a listed folder's entries. Persisted by `rawValue`; applied live by re-listing. - @Published var filesSortField: FilesSortField { didSet { defaults.set(filesSortField.rawValue, forKey: Keys.filesSortField) } } - - /// Ascending vs. descending for `filesSortField`. Persisted by `rawValue`. - @Published var filesSortDirection: FilesSortDirection { didSet { defaults.set(filesSortDirection.rawValue, forKey: Keys.filesSortDirection) } } - - /// The default-open action committed for a highlighted **file** (a folder always opens as a Finder - /// window regardless). Persisted by `rawValue`. - @Published var filesDefaultOpen: FilesDefaultOpen { didSet { defaults.set(filesDefaultOpen.rawValue, forKey: Keys.filesDefaultOpen) } } - - /// Which secondary metadata each row shows beside its name (date / kind / size — any combination). - /// An `OptionSet` persisted as its `Int` `rawValue` (mirrors `DangerZoneSelection`). - @Published var filesRowMetadata: FilesRowMetadata { didSet { defaults.set(filesRowMetadata.rawValue, forKey: Keys.filesRowMetadata) } } - - /// What the Files-band **lift** (the drill's primary resolve excursion) does on commit: `deliver` - /// (default) pastes the highlighted entry into the captured front app (`files-contextual-delivery`); - /// `open` opens it (file → default app per `filesDefaultOpen`, folder → Finder window). Orthogonal to - /// `filesDefaultOpen`, which only refines what an *open* of a file does. Persisted by `rawValue`. - @Published var filesLiftAction: FilesLiftAction { didSet { defaults.set(filesLiftAction.rawValue, forKey: Keys.filesLiftAction) } } - - /// The user-configurable Files **action-menu** contents, per entry type (`files-action-menu`). Persisted - /// as a JSON blob (like `gestureBindings`); defaults to the specified per-type menus. - @Published var filesActionMenu: FilesActionMenu { didSet { persistCodable(filesActionMenu, Keys.filesActionMenu) } } - - /// Bundle ids of detected terminals/editors the user has **disabled** from the action menu. The curated - /// set is "all detected tools, minus these," so default empty = every detected tool enabled. - @Published var filesToolsDisabled: [String] { didSet { defaults.set(filesToolsDisabled, forKey: Keys.filesToolsDisabled) } } - - /// The remembered deepest path last navigated to inside `rootPath`, or nil if none has been recorded - /// yet (cold start, or the root was just added). - func rememberedLocation(forRoot rootPath: String) -> String? { filesRememberedLocations[rootPath] } - - /// Remember `path` as the deepest location inside `rootPath` (written when the user leaves the band or - /// changes depth), so re-entering that root restores it. - func rememberLocation(_ path: String, forRoot rootPath: String) { - filesRememberedLocations[rootPath] = path - } - - /// Best-effort orphan cleanup: drop remembered-location entries whose root is no longer configured. - /// A no-op when nothing is orphaned (so it doesn't churn UserDefaults needlessly). - func pruneRememberedLocations(keepingRoots liveRoots: Set) { - let kept = filesRememberedLocations.filter { liveRoots.contains($0.key) } - if kept.count != filesRememberedLocations.count { filesRememberedLocations = kept } - } - // MARK: - Per-app keyboard language (opt-in; default OFF) /// Opt-in to remembering and re-selecting the keyboard input source per application (bundle id), @@ -549,7 +199,7 @@ final class AppSettings: ObservableObject { /// Opt-in to the Dock-hover window previews — the switcher "from another angle": hover an app's Dock /// tile to fan out its current-Space windows (including minimized), peek any one live, and click to - /// raise it. Like the clipboard / Files opt-ins (and unlike the gesture opt-ins) this relocates NO + /// raise it. Like the clipboard opt-in (and unlike the gesture opt-ins) this relocates NO /// native gesture, needs NO re-login, and requests NO new permission — it reuses the already-granted /// Accessibility (read the Dock's AX tree + raise) and Screen Recording (thumbnails) grants. There is /// NO `is…Effective` gate: flipping it ON installs the cursor monitor + Dock reader, OFF tears them @@ -604,21 +254,6 @@ final class AppSettings: ObservableObject { } } - /// The language last chosen for `commandID`, or nil if none has been chosen yet (cold start). - func rememberedLanguage(for commandID: UUID) -> String? { aiCommandLanguages[commandID.uuidString] } - - /// Remember `language` as the next-run default for `commandID` (written when the user repicks). - func rememberLanguage(_ language: String, for commandID: UUID) { - aiCommandLanguages[commandID.uuidString] = language - } - - /// Best-effort orphan cleanup: drop persisted language entries whose command id is not in - /// `liveIDs`. A no-op when nothing is orphaned (so it doesn't churn UserDefaults needlessly). - func pruneCommandLanguages(keeping liveIDs: Set) { - let live = Set(liveIDs.map(\.uuidString)) - let kept = aiCommandLanguages.filter { live.contains($0.key) } - if kept.count != aiCommandLanguages.count { aiCommandLanguages = kept } - } /// Shared singleton uses the standard user defaults. private convenience init() { @@ -674,58 +309,6 @@ final class AppSettings: ObservableObject { clipboardEdgeAcceleration = defaults.object(forKey: Keys.clipboardEdgeAcceleration) as? Double ?? Defaults.clipboardEdgeAcceleration clipboardPinDistance = defaults.object(forKey: Keys.clipboardPinDistance) as? Double ?? Defaults.clipboardPinDistance clipboardExcludedApps = defaults.object(forKey: Keys.clipboardExcludedApps) as? [String] ?? Defaults.clipboardExcludedApps - aiCommandsEnabled = defaults.object(forKey: Keys.aiCommandsEnabled) as? Bool ?? Defaults.aiCommandsEnabled - aiSelectedModelID = defaults.object(forKey: Keys.aiSelectedModelID) as? String ?? Defaults.aiSelectedModelID - aiSelectedChatModelID = defaults.object(forKey: Keys.aiSelectedChatModelID) as? String ?? Defaults.aiSelectedChatModelID - aiEnabledCapabilityModelIDs = Set(defaults.object(forKey: Keys.aiEnabledCapabilityModelIDs) as? [String] - ?? Defaults.aiEnabledCapabilityModelIDs) - // Full Potential master + sub-flags (addendum §D1): absent key ⇒ false (legacy-load), so settings - // written before this wave decode with the whole fleet OFF, leaving existing settings unchanged. - fullPotentialEnabled = defaults.object(forKey: Keys.fullPotentialEnabled) as? Bool ?? Defaults.fullPotentialEnabled - cpuLaneEnabled = defaults.object(forKey: Keys.cpuLaneEnabled) as? Bool ?? Defaults.cpuLaneEnabled - batchedRuntimeEnabled = defaults.object(forKey: Keys.batchedRuntimeEnabled) as? Bool ?? Defaults.batchedRuntimeEnabled - mediaGenEnabled = defaults.object(forKey: Keys.mediaGenEnabled) as? Bool ?? Defaults.mediaGenEnabled - backgroundAutonomyEnabled = defaults.object(forKey: Keys.backgroundAutonomyEnabled) as? Bool ?? Defaults.backgroundAutonomyEnabled - fleetCloudEscalationEnabled = defaults.object(forKey: Keys.fleetCloudEscalationEnabled) as? Bool ?? Defaults.fleetCloudEscalationEnabled - videoProvider = (defaults.object(forKey: Keys.videoProvider) as? String) - .flatMap(VideoProvider.init(rawValue:)) ?? Defaults.videoProvider - mediaVideoBudgetPerDay = defaults.object(forKey: Keys.mediaVideoBudgetPerDay) as? Int ?? Defaults.mediaVideoBudgetPerDay - aiCommandLanguages = defaults.object(forKey: Keys.aiCommandLanguages) as? [String: String] ?? Defaults.aiCommandLanguages - aiReasoningEnabled = defaults.object(forKey: Keys.aiReasoningEnabled) as? Bool ?? Defaults.aiReasoningEnabled - agentContextPreset = AgentContextPreset(rawValue: defaults.string(forKey: Keys.agentContextPreset) ?? "") ?? Defaults.agentContextPreset - agentContextTokens = defaults.object(forKey: Keys.agentContextTokens) as? Int ?? Defaults.agentContextTokens - agentCompactKV = defaults.object(forKey: Keys.agentCompactKV) as? Bool ?? Defaults.agentCompactKV - notchTuning = NotchTuning(rawValue: defaults.string(forKey: Keys.notchTuning) ?? "") ?? Defaults.notchTuning - agentMaxParkedSessions = defaults.object(forKey: Keys.agentMaxParkedSessions) as? Int ?? Defaults.agentMaxParkedSessions - aiIdleEvictMinutes = defaults.object(forKey: Keys.aiIdleEvictMinutes) as? Int ?? Defaults.aiIdleEvictMinutes - voiceConversationEnabled = defaults.object(forKey: Keys.voiceConversationEnabled) as? Bool ?? Defaults.voiceConversationEnabled - voicePTTKeyCode = defaults.object(forKey: Keys.voicePTTKeyCode) as? Int ?? Defaults.voicePTTKeyCode - computerUseEnabled = defaults.object(forKey: Keys.computerUseEnabled) as? Bool ?? Defaults.computerUseEnabled - agentStepTimeoutSeconds = defaults.object(forKey: Keys.agentStepTimeoutSeconds) as? Int ?? Defaults.agentStepTimeoutSeconds - agentTurnDeadlineSeconds = defaults.object(forKey: Keys.agentTurnDeadlineSeconds) as? Int ?? Defaults.agentTurnDeadlineSeconds - agentParkIdleTimeout = defaults.object(forKey: Keys.agentParkIdleTimeout) as? TimeInterval ?? Defaults.agentParkIdleTimeout - agentParkAutoDismissCountdown = defaults.object(forKey: Keys.agentParkAutoDismissCountdown) as? TimeInterval ?? Defaults.agentParkAutoDismissCountdown - agentOverscrollParkThreshold = defaults.object(forKey: Keys.agentOverscrollParkThreshold) as? Double ?? Defaults.agentOverscrollParkThreshold - agentNotchRevealDwell = defaults.object(forKey: Keys.agentNotchRevealDwell) as? TimeInterval ?? Defaults.agentNotchRevealDwell - flickVelocityThreshold = defaults.object(forKey: Keys.flickVelocityThreshold) as? Double ?? Defaults.flickVelocityThreshold - flickLiftWindow = defaults.object(forKey: Keys.flickLiftWindow) as? Double ?? Defaults.flickLiftWindow - agentWhitelistPaths = defaults.object(forKey: Keys.agentWhitelistPaths) as? [String] ?? Defaults.agentWhitelistPaths - agentWhitelistCommands = defaults.object(forKey: Keys.agentWhitelistCommands) as? [String] ?? Defaults.agentWhitelistCommands - filesBandEnabled = defaults.object(forKey: Keys.filesBandEnabled) as? Bool ?? Defaults.filesBandEnabled - filesRoots = defaults.object(forKey: Keys.filesRoots) as? [String] ?? Defaults.filesRoots - filesRememberedLocations = defaults.object(forKey: Keys.filesRememberedLocations) as? [String: String] ?? Defaults.filesRememberedLocations - filesRememberLocation = defaults.object(forKey: Keys.filesRememberLocation) as? Bool ?? Defaults.filesRememberLocation - filesColumnWidth = defaults.object(forKey: Keys.filesColumnWidth) as? Double ?? Defaults.filesColumnWidth - filesDensity = (defaults.object(forKey: Keys.filesDensity) as? String).flatMap(FilesDensity.init(rawValue:)) ?? Defaults.filesDensity - filesBandTint = defaults.object(forKey: Keys.filesBandTint) as? String ?? Defaults.filesBandTint - filesIconStyle = (defaults.object(forKey: Keys.filesIconStyle) as? String).flatMap(FilesIconStyle.init(rawValue:)) ?? Defaults.filesIconStyle - filesSortField = (defaults.object(forKey: Keys.filesSortField) as? String).flatMap(FilesSortField.init(rawValue:)) ?? Defaults.filesSortField - filesSortDirection = (defaults.object(forKey: Keys.filesSortDirection) as? String).flatMap(FilesSortDirection.init(rawValue:)) ?? Defaults.filesSortDirection - filesDefaultOpen = (defaults.object(forKey: Keys.filesDefaultOpen) as? String).flatMap(FilesDefaultOpen.init(rawValue:)) ?? Defaults.filesDefaultOpen - filesRowMetadata = (defaults.object(forKey: Keys.filesRowMetadata) as? Int).map(FilesRowMetadata.init(rawValue:)) ?? Defaults.filesRowMetadata - filesLiftAction = (defaults.object(forKey: Keys.filesLiftAction) as? String).flatMap(FilesLiftAction.init(rawValue:)) ?? Defaults.filesLiftAction - filesActionMenu = AppSettings.loadCodable(FilesActionMenu.self, defaults, Keys.filesActionMenu) ?? Defaults.filesActionMenu - filesToolsDisabled = defaults.object(forKey: Keys.filesToolsDisabled) as? [String] ?? Defaults.filesToolsDisabled keyboardLanguageEnabled = defaults.object(forKey: Keys.keyboardLanguageEnabled) as? Bool ?? Defaults.keyboardLanguageEnabled keyboardLanguageDefaultSourceID = defaults.object(forKey: Keys.keyboardLanguageDefaultSourceID) as? String ?? Defaults.keyboardLanguageDefaultSourceID keyboardLanguagePerSiteEnabled = defaults.object(forKey: Keys.keyboardLanguagePerSiteEnabled) as? Bool ?? Defaults.keyboardLanguagePerSiteEnabled @@ -774,58 +357,11 @@ final class AppSettings: ObservableObject { clipboardPollInterval = Defaults.clipboardPollInterval clipboardEdgeAcceleration = Defaults.clipboardEdgeAcceleration clipboardPinDistance = Defaults.clipboardPinDistance - // Files appearance/behavior tunables reset; `filesBandEnabled` (the opt-in), the configured roots, - // and the remembered-location map are a user choice (like the clipboard opt-in/exclusion list) and - // are intentionally NOT reset. - filesColumnWidth = Defaults.filesColumnWidth - filesDensity = Defaults.filesDensity - filesBandTint = Defaults.filesBandTint - filesIconStyle = Defaults.filesIconStyle - filesSortField = Defaults.filesSortField - filesSortDirection = Defaults.filesSortDirection - filesDefaultOpen = Defaults.filesDefaultOpen - filesRowMetadata = Defaults.filesRowMetadata - filesLiftAction = Defaults.filesLiftAction // back to deliver (the default contextual-delivery lift) - filesActionMenu = Defaults.filesActionMenu // per-type menus back to the specified defaults - filesToolsDisabled = Defaults.filesToolsDisabled // all detected terminals/editors enabled again - filesRememberLocation = Defaults.filesRememberLocation // a behavior tunable (back to default ON); the remembered map itself is preserved above - // Agent context tuning is a behavior tunable (like the clipboard/files appearance tunables), so it - // resets here back to Balanced / 8-bit-KV-off. The AI opt-in + model pin below are NOT reset. - agentContextPreset = Defaults.agentContextPreset - agentContextTokens = Defaults.agentContextTokens - agentCompactKV = Defaults.agentCompactKV - notchTuning = Defaults.notchTuning - agentMaxParkedSessions = Defaults.agentMaxParkedSessions - aiIdleEvictMinutes = Defaults.aiIdleEvictMinutes - // Voice/computer-use are OPT-INS (like the AI master), so resetToDefaults returns them to - // off; the budget tunables reset like other behavior tunables. - voiceConversationEnabled = Defaults.voiceConversationEnabled - voicePTTKeyCode = Defaults.voicePTTKeyCode - computerUseEnabled = Defaults.computerUseEnabled - agentStepTimeoutSeconds = Defaults.agentStepTimeoutSeconds - agentTurnDeadlineSeconds = Defaults.agentTurnDeadlineSeconds - agentParkIdleTimeout = Defaults.agentParkIdleTimeout - agentParkAutoDismissCountdown = Defaults.agentParkAutoDismissCountdown - agentOverscrollParkThreshold = Defaults.agentOverscrollParkThreshold - agentNotchRevealDwell = Defaults.agentNotchRevealDwell - flickVelocityThreshold = Defaults.flickVelocityThreshold - flickLiftWindow = Defaults.flickLiftWindow - // `aiCommandsEnabled` (a consent-gated opt-in that allows a multi-gigabyte download) and the - // selected-model pin are a deliberate user choice, so they're intentionally NOT reset — mirrors - // the launcher / clipboard opt-in handling. - // The background-autonomy whitelist (`agentWhitelistPaths`/`agentWhitelistCommands`) is the - // security boundary — a deliberate trust choice — so it is intentionally NOT reset here either - // (same handling as the other AI opt-ins; a reset never silently widens or narrows trust). - // `keyboardLanguageEnabled` and the global-default source id are likewise an opt-in user choice - // (the learned per-app map is a separate store), so they're intentionally NOT reset either. + // `keyboardLanguageEnabled` and the global-default source id are an opt-in user choice + // (the learned per-app map is a separate store), so they're intentionally NOT reset. // The per-site sub-toggle and the Apple Events ("Allow browser control") opt-in are the same: // consent-gated user choices (the latter governs a per-browser permission), NOT reset here. // `showDockPreviews` is likewise an opt-in user choice (no tunables of its own), NOT reset here. - // The Release Full Potential master + five sub-flags (`fullPotentialEnabled`, `cpuLaneEnabled`, - // `batchedRuntimeEnabled`, `mediaGenEnabled`, `backgroundAutonomyEnabled`, - // `fleetCloudEscalationEnabled`) are consent-gated opt-ins (some allow a multi-GB media download or - // real cloud spend), so they JOIN the AI opt-in preserve-set — intentionally NOT reset here. A - // tunables reset must never silently re-arm or disarm a fleet the user deliberately configured. } private func persist(_ value: Double, _ key: String) { defaults.set(value, forKey: key) } @@ -885,7 +421,7 @@ final class AppSettings: ObservableObject { static let manageVerticalGesture = false // opt-in; relocates Mission Control to four fingers static let enableLauncher = false // opt-in; frees four-finger native gestures static let launcherActivationThreshold = 0.01 // same feather-light trigger as the switcher - static let launcherStepDistance = 0.04 // one item per ~4% travel; also Files depth/highlight + static let launcherStepDistance = 0.04 // one item per ~4% travel static let launcherContextStepDistance = 0.09 // ~2.2× the item step; deliberate band switching static let dwellToArmDuration = 0.3 // quick tick; the charge stays readable static let showDiagnostics = false // troubleshooting tools hidden from the menu by default @@ -900,54 +436,6 @@ final class AppSettings: ObservableObject { static let clipboardEdgeAcceleration = 1.0 // edge-scroll acceleration sensitivity static let clipboardPinDistance = 0.08 // deliberate horizontal flick to pin (≈2 item steps); left-exit is a single step static let clipboardExcludedApps: [String] = [] - static let aiCommandsEnabled = false // opt-in; gates the AI band + model download/residency - static let aiSelectedModelID: String? = nil // nil = registry default model - static let aiSelectedChatModelID: String? = nil // nil = fleet/registry chat default - static let aiEnabledCapabilityModelIDs: [String] = [] // no capability models enabled by default - static let fullPotentialEnabled = false // master gate; OFF → V2.5 ships calm (addendum §D1) - static let cpuLaneEnabled = false // sub-flag; OFF (heat/battery) - static let batchedRuntimeEnabled = false // sub-flag; OFF (RAM + latency) - static let mediaGenEnabled = false // sub-flag; OFF (evicts chat; minutes/clip; GB of weights) - static let backgroundAutonomyEnabled = false // sub-flag; OFF (unattended action, audited) - static let fleetCloudEscalationEnabled = false // sub-flag; OFF ($ + network + data off-device) - static let videoProvider: VideoProvider = .cloud // honest default: hosted API, nothing downloaded - static let mediaVideoBudgetPerDay = 3 // conservative cloud-video cap (rolling 24h); 0 disables - static let aiCommandLanguages: [String: String] = [:] // per-command remembered runtime language - static let aiReasoningEnabled = true // let the model think (filtered out of the result); gated by the AI opt-in - static let agentContextPreset: AgentContextPreset = .balanced // comfortable mid context, the default - static let agentContextTokens = 8_192 // Balanced; clamped to the model max at use - static let agentCompactKV = false // 8-bit KV off by default - static let notchTuning: NotchTuning = .balanced // notch chat dial: thinking on · base context - static let agentMaxParkedSessions = 6 // soft cap; evicts the least-recently-updated idle one - static let aiIdleEvictMinutes = 60 // model-weight idle TTL; 0 = never (keep-forever) - static let voiceConversationEnabled = false // push-to-talk voice; opt-in, macOS 26 + mic grant - static let voicePTTKeyCode = 61 // Right Option - static let computerUseEnabled = false // AX read/act tools; opt-in, existing grants only - static let agentStepTimeoutSeconds = 30 // non-gated tool step wall clock (design D8) - static let agentTurnDeadlineSeconds = 180 // per-turn active wall clock (approval time exempt) - static let agentParkIdleTimeout: TimeInterval = 30 * 60 // RETIRED (D1): legacy summarize-and-sleep - static let agentParkAutoDismissCountdown: TimeInterval = 0 // 0 = never — expiry is opt-in - static let agentOverscrollParkThreshold = 0.22 // above canvasResolveThreshold (0.12) - static let agentNotchRevealDwell: TimeInterval = 0.3 // dwell behind the notch before the dock reveals - static let flickVelocityThreshold = 0.8 // peak normalized vel/sec; below = reading-scroll (D4, run-verify) - static let flickLiftWindow = 0.12 // seconds from last fast frame to lift (D4, run-verify) - static let agentWhitelistPaths: [String] = [] // trust nothing arbitrary on a fresh install - static let agentWhitelistCommands: [String] = [] // trust no command pattern on a fresh install - static let filesBandEnabled = false // opt-in; injects the local-only Files band (no re-login, no new permission) - static let filesRoots: [String] = [] // configured local root folders (the Hub seeds a default set) - static let filesRememberedLocations: [String: String] = [:] // root path → last deepest path - static let filesRememberLocation = true // reopen displaying the last folder (restored at open), not the roots list - static let filesColumnWidth = 260.0 // points; current-list column width (bounded overlay) - static let filesDensity: FilesDensity = .comfortable - static let filesBandTint = "#3B82C4" // a calm blue, distinct from the clipboard band's amber - static let filesIconStyle: FilesIconStyle = .icon // cheap type icon by default (no QuickLook churn) - static let filesSortField: FilesSortField = .name - static let filesSortDirection: FilesSortDirection = .ascending - static let filesDefaultOpen: FilesDefaultOpen = .defaultApp - static let filesRowMetadata: FilesRowMetadata = .date // show the modified date beside the name - static let filesLiftAction: FilesLiftAction = .deliver // lift delivers the entry to the front app by default - static let filesActionMenu = FilesActionMenu.default // per-type menus exactly as specified - static let filesToolsDisabled: [String] = [] // all detected terminals/editors enabled static let keyboardLanguageEnabled = false // opt-in; gates per-app input-source learn/apply (no re-login) static let keyboardLanguageDefaultSourceID = "" // "" = no global default (pure learn-as-you-go) // Per-host memory inside browsers rides along by default when the keyboard-language master @@ -1001,54 +489,6 @@ final class AppSettings: ObservableObject { static let clipboardEdgeAcceleration = "clipboardEdgeAcceleration" static let clipboardPinDistance = "clipboardPinDistance" static let clipboardExcludedApps = "clipboardExcludedApps" - static let aiCommandsEnabled = "aiCommandsEnabled" - static let aiSelectedModelID = "aiSelectedModelID" - static let aiSelectedChatModelID = "aiSelectedChatModelID" - static let aiEnabledCapabilityModelIDs = "aiEnabledCapabilityModelIDs" - static let fullPotentialEnabled = "fullPotentialEnabled" - static let cpuLaneEnabled = "cpuLaneEnabled" - static let batchedRuntimeEnabled = "batchedRuntimeEnabled" - static let mediaGenEnabled = "mediaGenEnabled" - static let backgroundAutonomyEnabled = "backgroundAutonomyEnabled" - static let fleetCloudEscalationEnabled = "fleetCloudEscalationEnabled" - static let videoProvider = "videoProvider" - static let mediaVideoBudgetPerDay = "mediaVideoBudgetPerDay" - static let aiCommandLanguages = "aiCommandLanguages" - static let aiReasoningEnabled = "aiReasoningEnabled" - static let agentContextPreset = "agentContextPreset" - static let agentContextTokens = "agentContextTokens" - static let agentCompactKV = "agentCompactKV" - static let notchTuning = "notchTuning" - static let agentMaxParkedSessions = "agentMaxParkedSessions" - static let aiIdleEvictMinutes = "aiIdleEvictMinutes" - static let voiceConversationEnabled = "voiceConversationEnabled" - static let voicePTTKeyCode = "voicePTTKeyCode" - static let computerUseEnabled = "computerUseEnabled" - static let agentStepTimeoutSeconds = "agentStepTimeoutSeconds" - static let agentTurnDeadlineSeconds = "agentTurnDeadlineSeconds" - static let agentParkIdleTimeout = "agentParkIdleTimeout" - static let agentParkAutoDismissCountdown = "agentParkAutoDismissCountdown" - static let agentOverscrollParkThreshold = "agentOverscrollParkThreshold" - static let agentNotchRevealDwell = "agentNotchRevealDwell" - static let flickVelocityThreshold = "flickVelocityThreshold" - static let flickLiftWindow = "flickLiftWindow" - static let agentWhitelistPaths = "agentWhitelistPaths" - static let agentWhitelistCommands = "agentWhitelistCommands" - static let filesBandEnabled = "filesBandEnabled" - static let filesRoots = "filesRoots" - static let filesRememberedLocations = "filesRememberedLocations" - static let filesRememberLocation = "filesRememberLocation" - static let filesColumnWidth = "filesColumnWidth" - static let filesDensity = "filesDensity" - static let filesBandTint = "filesBandTint" - static let filesIconStyle = "filesIconStyle" - static let filesSortField = "filesSortField" - static let filesSortDirection = "filesSortDirection" - static let filesDefaultOpen = "filesDefaultOpen" - static let filesRowMetadata = "filesRowMetadata" - static let filesLiftAction = "filesLiftAction" - static let filesActionMenu = "filesActionMenu" - static let filesToolsDisabled = "filesToolsDisabled" static let keyboardLanguageEnabled = "keyboardLanguageEnabled" static let keyboardLanguageDefaultSourceID = "keyboardLanguageDefaultSourceID" static let keyboardLanguagePerSiteEnabled = "keyboardLanguagePerSiteEnabled" diff --git a/Sources/ThreeFingerSwitcher/Settings/ModelManagementView.swift b/Sources/ThreeFingerSwitcher/Settings/ModelManagementView.swift deleted file mode 100644 index ba1a2d5..0000000 --- a/Sources/ThreeFingerSwitcher/Settings/ModelManagementView.swift +++ /dev/null @@ -1,181 +0,0 @@ -import SwiftUI -import AppKit - -/// The on-device model-management surface (spec: "AI model management settings"). Binds to an -/// injected `ModelManager` (`@ObservedObject`) and renders its observable `ModelLifecycleState`: -/// the selected model's identity + size, the download status/progress, and the available actions -/// (download / retry, evict). Pure presentation — every action calls back into the manager, which -/// owns the lifecycle; the view re-renders as `@Published state` advances. -/// -/// The view takes the resolved `ModelDescriptor` it represents (the selected model) so the call site -/// — Settings, with `AppSettings.aiSelectedModelID` resolved against `ModelCatalog` — decides which -/// model is shown; the view just reflects the manager's state for it. -struct ModelManagementView: View { - @ObservedObject var manager: ModelManager - - /// The model this surface represents (selected model). Defaults to the registry default. - var descriptor: ModelDescriptor - - /// Begin (or retry) the download. The call site supplies it so the view stays free of the async - /// orchestration / error handling; it just triggers the action and reflects the resulting state. - var onDownload: () -> Void = {} - - /// Whether the raw "Show details" disclosure is expanded for a failure (collapsed by default — - /// detail is opt-in, never shown inline; design D4). - @State private var showingDetails = false - - /// Drives the in-window delete confirmation (no app-modal NSAlert — that would freeze the window). - @State private var confirmingDelete = false - - var body: some View { - VStack(alignment: .leading, spacing: 8) { - HStack(spacing: 8) { - Image(systemName: "brain.head.profile").foregroundStyle(.secondary) - VStack(alignment: .leading, spacing: 1) { - Text(descriptor.displayName).font(.headline) - Text(sizeLabel).font(.caption).foregroundStyle(.secondary) - } - Spacer() - } - - statusRow - - actionRow - } - .padding(.vertical, 2) - } - - // MARK: Status - - @ViewBuilder - private var statusRow: some View { - switch manager.state { - case .notDownloaded: - label("Not downloaded", systemImage: "arrow.down.circle", color: .secondary) - case let .downloading(progress): - VStack(alignment: .leading, spacing: 4) { - label("Downloading… \(Int(progress * 100))%", systemImage: "arrow.down.circle.fill", color: .accentColor) - ProgressView(value: progress) - } - case .verifying: - label("Verifying…", systemImage: "checkmark.shield", color: .accentColor) - case .ready: - label("Downloaded (not loaded)", systemImage: "internaldrive", color: .secondary) - case .loading: - label("Loading into memory…", systemImage: "memorychip", color: .accentColor) - case .loaded: - label("Loaded and ready", systemImage: "checkmark.circle.fill", color: .green) - case let .failed(reason, details): - VStack(alignment: .leading, spacing: 6) { - label("Failed", systemImage: "exclamationmark.triangle.fill", color: .red) - // The concise headline is primary: capped + middle-truncating + selectable, so an - // unexpectedly long message degrades gracefully instead of overflowing the fixed frame. - Text(reason) - .font(.caption) - .foregroundStyle(.secondary) - .lineLimit(3) - .truncationMode(.middle) - .fixedSize(horizontal: false, vertical: true) - .textSelection(.enabled) - if let details, !details.isEmpty { - detailsDisclosure(details) - } - } - } - } - - /// A collapsed "Show details" disclosure for the raw technical text behind a failure: bounded - /// (scrolls past ~120pt) so even a giant dump can't grow the row, with a "Copy details" action. - @ViewBuilder - private func detailsDisclosure(_ details: String) -> some View { - DisclosureGroup(isExpanded: $showingDetails) { - VStack(alignment: .leading, spacing: 6) { - ScrollView { - Text(details) - .font(.system(.caption2, design: .monospaced)) - .foregroundStyle(.secondary) - .textSelection(.enabled) - .frame(maxWidth: .infinity, alignment: .leading) - } - .frame(maxHeight: 120) - Button { - let pb = NSPasteboard.general - pb.clearContents() - pb.setString(details, forType: .string) - } label: { - Label("Copy details", systemImage: "doc.on.doc") - } - .controlSize(.small) - } - .padding(.top, 4) - } label: { - Text("Show details").font(.caption).foregroundStyle(.secondary) - } - } - - // MARK: Actions - - @ViewBuilder - private var actionRow: some View { - HStack { - switch manager.state { - case .notDownloaded: - Button("Download") { onDownload() } - case .failed: - Button("Retry download") { onDownload() } - case .downloading, .verifying, .loading: - ProgressView().controlSize(.small) - case .ready: - Text("Loads on demand when a command runs.") - .font(.caption).foregroundStyle(.secondary) - case .loaded: - Button("Evict from memory") { manager.evict() } - .help("Unloads the model from memory; the next command reloads it on demand.") - } - Spacer() - // Delete the selected model's weights from disk whenever it is present (`.ready`/`.loaded`). - if isOnDisk { deleteButton } - } - .confirmationDialog("Delete \(descriptor.displayName)?", - isPresented: $confirmingDelete, titleVisibility: .visible) { - Button("Delete model", role: .destructive) { manager.deleteFromDisk(descriptor) } - Button("Cancel", role: .cancel) { } - } message: { - Text("Removes the downloaded weights (\(sizeLabel)) from disk. You can re-download it later.") - } - } - - /// "Delete" affordance for the selected model — confirms in-window before removing the weights. - private var deleteButton: some View { - Button("Delete", role: .destructive) { confirmingDelete = true } - .help("Delete this model's weights from disk (frees \(sizeLabel)).") - } - - /// Whether the selected model's weights are on disk, so a Delete affordance is meaningful. `state` - /// is kept in sync with the selected model by the call site (`ModelManager.showStatus(for:)`), so - /// `.ready` or `.loaded` means present. - private var isOnDisk: Bool { - switch manager.state { - case .ready, .loaded: return true - default: return false - } - } - - // MARK: Helpers - - private var sizeLabel: String { - let formatter = ByteCountFormatter() - formatter.allowedUnits = [.useGB, .useMB] - formatter.countStyle = .file - return formatter.string(fromByteCount: descriptor.sizeBytes) - } - - private func label(_ text: String, systemImage: String, color: Color) -> some View { - Label { - Text(text) - } icon: { - Image(systemName: systemImage).foregroundStyle(color) - } - .font(.subheadline) - } -} diff --git a/Sources/ThreeFingerSwitcher/TouchInput/KeyboardSwitcherTap.swift b/Sources/ThreeFingerSwitcher/TouchInput/KeyboardSwitcherTap.swift index dd05720..a8c248c 100644 --- a/Sources/ThreeFingerSwitcher/TouchInput/KeyboardSwitcherTap.swift +++ b/Sources/ThreeFingerSwitcher/TouchInput/KeyboardSwitcherTap.swift @@ -92,12 +92,6 @@ final class KeyboardSwitcherTap { if let tap { CGEvent.tapEnable(tap: tap, enable: true) } return pass } - // Agent-posted synthetic keystrokes (`add-voice-computer-use-agent`, spec: "Agent typing - // never triggers the app's own recognizers"): tagged at the source, passed through untouched - // — the agent's typing must reach the target app and never trip our own ⌘-Tab machinery. - if AgentActionArbiter.isSyntheticAgentEvent(event) { - return pass - } switch type { case .flagsChanged: // Track ⌘ and drive the session's arm/commit edges; NEVER consume a modifier change (other diff --git a/Sources/ThreeFingerSwitcherApp/main.swift b/Sources/ThreeFingerSwitcherApp/main.swift index 21d363d..bb78733 100644 --- a/Sources/ThreeFingerSwitcherApp/main.swift +++ b/Sources/ThreeFingerSwitcherApp/main.swift @@ -1,28 +1,7 @@ import ThreeFingerSwitcherCore -import GemmaRuntime // Thin executable: all app logic lives in the ThreeFingerSwitcherCore library so that a // test target can `@testable import ThreeFingerSwitcherCore`. (A test target cannot // @testable-import an executable module that contains top-level code.) -// Wire the REAL in-process Gemma 4 (MLX) runtime into Core's model seam BEFORE app startup, so the -// lazily-built `ModelManager` resolves to it (Core itself stays MLX-free; see AIRuntimeInjection). -// Top-level executable code runs on the main thread, so asserting MainActor isolation is valid here -// (same pattern as runThreeFingerSwitcher) and lets us set the @MainActor-isolated factory. -MainActor.assumeIsolated { - // The factory carries the ALREADY-RESOLVED Full-Potential gates (AppCoordinator computes them via - // `FullPotentialGate.isUnlocked`): the CPU ternary lane is installed only when `.cpuLane` is unlocked - // and the multi-stream batched runtime only when `.batchedRuntime` is unlocked (§D1 / §7.1). Both OFF - // (the default) → today's single-GPU-lane, single-session build (the calm panic-off). - AIRuntimeInjection.modelManagerFactory = { optedIn, cpuLaneUnlocked, batchedRuntimeUnlocked in - GemmaRuntime.makeModelManager(optedIn: optedIn, - cpuLaneUnlocked: cpuLaneUnlocked, - batchedRuntimeUnlocked: batchedRuntimeUnlocked) - } - // The image `MediaRuntime` (the `generate_image` backend) — same seam pattern. The runtime's - // diffusion pipeline is unbuilt until Wave 2, so a `generate_image` route surfaces a clean, bounded - // `MediaError` (model not installed / not ready), never a blank PNG or the model's raw tool-call text. - AIRuntimeInjection.imageRuntimeFactory = { GemmaRuntime.makeImageRuntime(imageModelID: $0) } -} - runThreeFingerSwitcher() diff --git a/Tests/ThreeFingerSwitcherTests/AICommandCatalogTests.swift b/Tests/ThreeFingerSwitcherTests/AICommandCatalogTests.swift deleted file mode 100644 index 043856d..0000000 --- a/Tests/ThreeFingerSwitcherTests/AICommandCatalogTests.swift +++ /dev/null @@ -1,114 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Invariant tests for the shipped AI command catalog (task §2.9 + review hardening). These guard the -/// catalog as the single source of truth: every preset is fireable, every category is represented, the -/// curated `seeded()` subset resolves by name, and category-wide output/input conventions hold (e.g. -/// vision presets are preview-only, `{lang}` presets declare a language parameter). A failure here is a -/// real catalog inconsistency, not a unit under test. -final class AICommandCatalogTests: XCTestCase { - - // MARK: - Fireability - - func testEveryPresetIsFireable() { - for entry in AICommandCatalog.entries { - XCTAssertFalse(entry.command.name.isEmpty, - "every preset has a non-empty name") - XCTAssertFalse(entry.command.promptTemplate.isEmpty, - "preset \(entry.command.name) has a non-empty prompt template") - } - } - - // MARK: - Coverage - - func testAllCategoriesPresent() { - for category in AICommandCatalog.Category.allCases { - XCTAssertFalse(AICommandCatalog.commands(in: category).isEmpty, - "category \(category.title) has at least one preset") - } - XCTAssertEqual(AICommandCatalog.Category.allCases.count, 9, - "all 9 categories are declared") - XCTAssertGreaterThanOrEqual(AICommandCatalog.entries.count, 50, - "the catalog ships a substantial set of presets") - } - - // MARK: - copy(of:) - - func testCopyMintsFreshIdWithoutMutatingOriginal() { - let original = AICommandCatalog.entries[0].command - let first = AICommandCatalog.copy(of: original) - let second = AICommandCatalog.copy(of: original) - - XCTAssertNotEqual(first.id, second.id, "each copy mints a distinct id") - XCTAssertNotEqual(first.id, original.id, "the first copy differs from the original's id") - XCTAssertNotEqual(second.id, original.id, "the second copy differs from the original's id") - - for copy in [first, second] { - XCTAssertEqual(copy.name, original.name, "copy preserves the name") - XCTAssertEqual(copy.promptTemplate, original.promptTemplate, "copy preserves the template") - XCTAssertEqual(copy.inputs, original.inputs, "copy preserves the input capability set") - XCTAssertEqual(copy.outputs, original.outputs, "copy preserves the output capability set") - } - } - - // MARK: - Category conventions - - func testVisionPresetsArePreviewOnlyImageInputs() { - let vision = AICommandCatalog.commands(in: .vision) - XCTAssertFalse(vision.isEmpty, "the vision category ships presets") - for command in vision { - XCTAssertTrue(command.inputs.contains(.screenRegion) || command.inputs.contains(.clipboardImage), - "vision preset \(command.name) reads an image source (screen region or clipboard image)") - XCTAssertEqual(command.requiredCapabilities, [.vision], - "vision preset \(command.name) requires a vision-capable model") - XCTAssertEqual(command.outputs, [.previewOnly], - "vision preset \(command.name) is preview-only") - } - } - - func testVisionCategoryShipsAClipboardImagePreset() { - let vision = AICommandCatalog.commands(in: .vision) - let clipboardImage = vision.filter { $0.inputs.contains(.clipboardImage) } - XCTAssertFalse(clipboardImage.isEmpty, - "the Vision category ships at least one clipboard-image preset (analyze a copied image)") - for command in clipboardImage { - XCTAssertFalse(command.name.isEmpty, "preset has a name") - XCTAssertFalse(command.promptTemplate.isEmpty, "preset \(command.name) has a prompt template") - XCTAssertEqual(command.outputs, [.previewOnly], "clipboard-image preset \(command.name) is preview-only") - } - } - - func testLangPresetsDeclareLanguageParameter() { - let langPresets = AICommandCatalog.entries - .map(\.command) - .filter { $0.promptTemplate.contains("{lang}") } - XCTAssertFalse(langPresets.isEmpty, "at least one preset uses the {lang} token") - for command in langPresets { - // `languageDefault` is non-nil for any `.languageChoice` (human OR programming language), - // so this guards every `{lang}` preset without pattern-matching a specific case. - XCTAssertNotNil(command.runtimeParameter?.languageDefault, - "preset \(command.name) uses {lang} so it declares a language-choice parameter") - } - } - - func testCommentaryPresetsArePreviewOnly() { - let understand = AICommandCatalog.commands(in: .understand) - XCTAssertFalse(understand.isEmpty, "the understand category ships presets") - for command in understand { - XCTAssertEqual(command.outputs, [.previewOnly], - "understand preset \(command.name) is preview-only") - } - } - - // MARK: - seeded() - - func testSeededResolvesAllCuratedNames() { - let expected = ["Fix Grammar", "Make Concise", "Improve Writing", "Translate", - "Explain", "Summarize", "Draft a Reply", "Add to Calendar"] - let seeded = AICommandCatalog.seeded() - XCTAssertEqual(seeded.count, expected.count, - "seeded() resolves every curated name (a typo'd name would silently drop)") - XCTAssertEqual(seeded.map(\.name), expected, - "seeded() emits the curated commands in the curated order") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/AICommandExecutorTests.swift b/Tests/ThreeFingerSwitcherTests/AICommandExecutorTests.swift deleted file mode 100644 index c45ae1e..0000000 --- a/Tests/ThreeFingerSwitcherTests/AICommandExecutorTests.swift +++ /dev/null @@ -1,954 +0,0 @@ -import XCTest -import Combine -@testable import ThreeFingerSwitcherCore - -/// Tests for the command executor (spec: "Command input acquisition" / "In-place output routing"; -/// tasks phase 9) against the stub runtime + a fake selection provider + a fake task dispatcher: the -/// full in-place pipeline, selection→clipboard fallback, the no-input path, previewOnly writing -/// nothing, replaceSelection routing to the writer, cancellation, and a task path routing through the -/// dispatcher with the stored `confirmBeforeRun` honored. -@MainActor -final class AICommandExecutorTests: XCTestCase { - - // MARK: - Fakes - - /// A scriptable `SelectionProviding`: records every write and serves canned reads. `replaceLands` / - /// `pasteLands` script whether a write actually LANDED (so the non-landed → `.failed` honesty is - /// testable); `screenCaptureOutcome` overrides the derived capture result (e.g. `.permissionDenied`). - private final class FakeSelectionProvider: SelectionProviding { - var selectedText: String? - var clipboardText: String? - var clipboardImageData: Data? - var replaceLands = true - var pasteLands = true - - private(set) var replacedWith: [String] = [] - private(set) var pastedAtCursor: [String] = [] - - init(selectedText: String? = nil, clipboardText: String? = nil) { - self.selectedText = selectedText - self.clipboardText = clipboardText - } - - func readSelectedText() async -> String? { selectedText } - func readClipboardText() -> String? { clipboardText } - func readClipboardImage() -> Data? { clipboardImageData } - // Screen-region capture is no longer on this seam: the picker captures the region and the - // capture outcome is handed to `executor.fire(_:screenCapture:)`. - - @discardableResult - func replaceSelection(_ text: String) async -> Bool { replacedWith.append(text); return replaceLands } - @discardableResult - func pasteAtCursor(_ text: String) async -> Bool { pastedAtCursor.append(text); return pasteLands } - } - - /// A fake `TaskDispatching` for the executor's new two-stage seam: `prepare` records the requested - /// (kind, prompt, source) and returns a scripted review; `execute` records the executed review's - /// preview title so a test can assert the side effect fired exactly once, only on commit. - private final class FakeTaskDispatcher: TaskDispatching { - private(set) var prepared: [(kind: TaskKind, prompt: String, source: TaskSource, reasoning: Bool)] = [] - private(set) var executed: [TaskReview] = [] - /// The review `prepare` returns. Defaults to a ready `.action` so the happy path lands in - /// `.reviewingAction` / `.committed`. - var reviewToReturn: TaskReview = .action(title: "Task", - fields: [ReviewField("Field", "Value")], - payload: .openTool(tool: "x", - action: ParsedOpenTool(applicable: true, - reason: nil, - payload: "p"))) - var executeError: Error? - - func prepare(_ kind: TaskKind, resolvedPrompt: String, source: TaskSource, - reasoning: Bool) async -> TaskReview { - prepared.append((kind, resolvedPrompt, source, reasoning)) - return reviewToReturn - } - - func execute(_ review: TaskReview) async throws { - if let error = executeError { throw error } - executed.append(review) - } - } - - // MARK: - ModelManager helper - - /// A `ModelManager` that is opted-in, verified, and whose `runtimeFactory` returns `runtime`, so - /// `runtime(requiring:)` resolves to exactly that scripted stub. - private func loadedManager(runtime: StubLLMRuntime, - capabilities: Set = [.text, .vision]) async throws -> ModelManager { - let payload = Data("weights".utf8) - let registry = ModelCatalog( - models: [ModelDescriptor( - id: "test-model", - displayName: "Test Model", - sizeBytes: Int64(payload.count), - integritySHA: ModelManager.sha256Hex(payload), - downloadURL: URL(string: "https://models.invalid/test-model")!, - capabilities: capabilities, - quantization: .qat4bit - )], - defaultModelID: "test-model" - ) - let root = URL(fileURLWithPath: NSTemporaryDirectory()) - .appendingPathComponent("tfs-executor-tests-\(UUID().uuidString)", isDirectory: true) - let manager = ModelManager( - registry: registry, - downloader: FakeDownloader(payload: payload), - optedIn: true, - storageRoot: root, - runtimeFactory: { _ in runtime } - ) - try await manager.downloadAndVerify(registry.models[0]) - return manager - } - - private final class FakeDownloader: ModelDownloading, @unchecked Sendable { - let payload: Data - init(payload: Data) { self.payload = payload } - func download(_ descriptor: ModelDescriptor, to destination: URL, - progress: @Sendable (Double) -> Void) async throws -> Data { - progress(1.0); return payload - } - } - - /// A runtime that records the last `LLMRequest` it was handed, so a test can assert the executor - /// propagated `reasoning` onto the text-path request. Echoes the prompt as a single token. - private final class CapturingLLMRuntime: LLMRuntime, @unchecked Sendable { - let capabilities: Set = [.text, .vision] - private(set) var lastRequest: LLMRequest? - - func generate(_ request: LLMRequest) -> AsyncThrowingStream { - lastRequest = request - return AsyncThrowingStream { continuation in - continuation.yield(Token(request.prompt, isFinal: true)) - continuation.finish() - } - } - - func structured( - _ request: LLMRequest, schema: StructuredSchema, as type: T.Type - ) async throws -> StructuredOutcome { - lastRequest = request - throw RuntimeError.couldNotProduceValid(attempts: 1) - } - } - - /// A `ModelManager` opted-in + verified whose `runtimeFactory` returns `runtime` — variant of - /// `loadedManager(runtime:)` that accepts any `LLMRuntime` (used by the reasoning-propagation test). - private func loadedManager(anyRuntime runtime: LLMRuntime) async throws -> ModelManager { - let payload = Data("weights".utf8) - let registry = ModelCatalog( - models: [ModelDescriptor( - id: "test-model", - displayName: "Test Model", - sizeBytes: Int64(payload.count), - integritySHA: ModelManager.sha256Hex(payload), - downloadURL: URL(string: "https://models.invalid/test-model")!, - capabilities: [.text, .vision], - quantization: .qat4bit - )], - defaultModelID: "test-model" - ) - let root = URL(fileURLWithPath: NSTemporaryDirectory()) - .appendingPathComponent("tfs-executor-tests-\(UUID().uuidString)", isDirectory: true) - let manager = ModelManager( - registry: registry, - downloader: FakeDownloader(payload: payload), - optedIn: true, - storageRoot: root, - runtimeFactory: { _ in runtime } - ) - try await manager.downloadAndVerify(registry.models[0]) - return manager - } - - // MARK: - Reasoning flag propagation (text path) - - func testReasoningFlagPropagatesToTextRequest() async throws { - let runtime = CapturingLLMRuntime() - let manager = try await loadedManager(anyRuntime: runtime) - let selection = FakeSelectionProvider(selectedText: "input") - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher(), - reasoning: { true }) - - let command = AICommand(name: "Echo", icon: .emoji("🔁"), input: .selection, - promptTemplate: "{input}", output: .previewOnly) - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - XCTAssertEqual(runtime.lastRequest?.reasoning, true, - "the executor sets request.reasoning from the injected closure") - } - - // MARK: - Per-command reasoning override (text path): the command's override wins over the global - - /// A per-command `.off` override beats a global default of TRUE: the text request reasons FALSE. - func testCommandReasoningOffOverridesGlobalTrue() async throws { - let runtime = CapturingLLMRuntime() - let manager = try await loadedManager(anyRuntime: runtime) - let selection = FakeSelectionProvider(selectedText: "input") - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher(), - reasoning: { true }) // global default ON - - let command = AICommand(name: "Echo", icon: .emoji("🔁"), input: .selection, - promptTemplate: "{input}", output: .previewOnly, reasoning: .off) - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - XCTAssertEqual(runtime.lastRequest?.reasoning, false, - "a per-command .off override forces reasoning off even when the global default is on") - } - - /// A per-command `.on` override beats a global default of FALSE: the text request reasons TRUE. - func testCommandReasoningOnOverridesGlobalFalse() async throws { - let runtime = CapturingLLMRuntime() - let manager = try await loadedManager(anyRuntime: runtime) - let selection = FakeSelectionProvider(selectedText: "input") - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher(), - reasoning: { false }) // global default OFF - - let command = AICommand(name: "Echo", icon: .emoji("🔁"), input: .selection, - promptTemplate: "{input}", output: .previewOnly, reasoning: .on) - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - XCTAssertEqual(runtime.lastRequest?.reasoning, true, - "a per-command .on override forces reasoning on even when the global default is off") - } - - /// An absent per-command override (`nil`) follows the global default on the text request. - func testCommandReasoningNilFollowsGlobalDefault() async throws { - let runtime = CapturingLLMRuntime() - let manager = try await loadedManager(anyRuntime: runtime) - let selection = FakeSelectionProvider(selectedText: "input") - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher(), - reasoning: { true }) // global default ON - - let command = AICommand(name: "Echo", icon: .emoji("🔁"), input: .selection, - promptTemplate: "{input}", output: .previewOnly) // no override - XCTAssertNil(command.reasoning, "a fresh command has no reasoning override") - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - XCTAssertEqual(runtime.lastRequest?.reasoning, true, - "an absent override follows the injected global default") - } - - // MARK: - Per-command reasoning override (task path): the executor passes the resolved value - - /// The executor resolves reasoning per command and passes it into `dispatcher.prepare(..., reasoning:)`: - /// a task command with `.on` + a global default of FALSE prepares with reasoning TRUE. - func testTaskPathReceivesPerCommandResolvedReasoning() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["unused for tasks"], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "lunch with sam tomorrow") - let dispatcher = FakeTaskDispatcher() - let executor = AICommandExecutor(modelManager: manager, selection: selection, dispatcher: dispatcher, - reasoning: { false }) // global default OFF - - let command = AICommand(name: "Add to Calendar", icon: .emoji("📅"), input: .selection, - promptTemplate: "{input}", output: .runTask(.addToCalendar), - confirmBeforeRun: false, reasoning: .on) - executor.fire(command) - await waitUntil { executor.state == .committed } - let prep = try XCTUnwrap(dispatcher.prepared.first) - XCTAssertTrue(prep.reasoning, - "a per-command .on override is passed into prepare even when the global default is off") - } - - /// Spin the run loop until `predicate` holds or a deadline elapses (the executor streams off a - /// detached Task; this lets the test observe terminal/observable states without sleeping fixed). - private func waitUntil(_ predicate: @MainActor () -> Bool, - timeout: TimeInterval = 2.0, - file: StaticString = #filePath, line: UInt = #line) async { - let deadline = Date().addingTimeInterval(timeout) - while !predicate() && Date() < deadline { - try? await Task.sleep(nanoseconds: 2_000_000) // 2 ms - } - XCTAssertTrue(predicate(), "condition not met within \(timeout)s", file: file, line: line) - } - - // MARK: - In-place pipeline: replaceSelection routes to the writer - - func testReplaceSelectionPipelineRoutesToWriter() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["Fixed ", "text"], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "teh txt") - let dispatcher = FakeTaskDispatcher() - let executor = AICommandExecutor(modelManager: manager, selection: selection, dispatcher: dispatcher) - - let command = AICommand(name: "Fix", icon: .emoji("✅"), input: .selection, - promptTemplate: "Fix: {input}", output: .replaceSelection) - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - XCTAssertEqual(executor.state, .ready(result: "Fixed text")) - - try await executor.commit() - XCTAssertEqual(executor.state, .committed) - XCTAssertEqual(selection.replacedWith, ["Fixed text"], "the result is written via replaceSelection") - XCTAssertTrue(dispatcher.prepared.isEmpty, "an in-place output never prepares a task") - XCTAssertTrue(dispatcher.executed.isEmpty, "an in-place output never executes a task") - } - - // MARK: - Selection → clipboard fallback - - func testEmptySelectionFallsBackToClipboard() async throws { - let stub = StubLLMRuntime(scriptedTokens: [], interTokenDelayNanos: 0) // echoes the prompt - let manager = try await loadedManager(runtime: stub) - // No selection, but clipboard has text — the executor must use the clipboard. - let selection = FakeSelectionProvider(selectedText: nil, clipboardText: "from clipboard") - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher()) - - let command = AICommand(name: "Echo", icon: .emoji("🔁"), input: .selection, - promptTemplate: "{input}", output: .previewOnly) - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - // The stub echoes its prompt, which is the resolved template = the acquired input. - XCTAssertEqual(executor.state, .ready(result: "from clipboard"), - "an empty selection falls back to the clipboard text") - } - - // MARK: - Contextual commit (change `ai-action-context-resolution`) - - /// The core behavior: a default all-on command REPLACES when there is a live selection and PASTES when - /// there is not — one command, resolved from the environment (the user's "it knows what to do"). - func testDefaultCommandReplacesWhenSelectingAndPastesOtherwise() async throws { - // (a) With a live selection → commit is a replace. - do { - let stub = StubLLMRuntime(scriptedTokens: ["done"], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "sel", clipboardText: "clip") - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher()) - let command = AICommand(name: "Smart", icon: .emoji("✨"), - inputs: AICommand.defaultInputs, promptTemplate: "{input}", - outputs: AICommand.defaultOutputs) - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - try await executor.commit() - XCTAssertEqual(selection.replacedWith, ["done"], "a live selection commits as a replace") - XCTAssertTrue(selection.pastedAtCursor.isEmpty, "it does not also paste") - } - // (b) With no selection (clipboard only) → commit is a paste at the cursor. - do { - let stub = StubLLMRuntime(scriptedTokens: ["done"], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: nil, clipboardText: "clip") - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher()) - let command = AICommand(name: "Smart", icon: .emoji("✨"), - inputs: AICommand.defaultInputs, promptTemplate: "{input}", - outputs: AICommand.defaultOutputs) - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - try await executor.commit() - XCTAssertEqual(selection.pastedAtCursor, ["done"], "no selection commits as a paste at the cursor") - XCTAssertTrue(selection.replacedWith.isEmpty, "it does not replace the (absent) selection") - } - } - - /// A command that disables the selection channel uses the clipboard even when a selection exists. - func testDisabledSelectionChannelIsSkippedForClipboard() async throws { - let stub = StubLLMRuntime(scriptedTokens: [], interTokenDelayNanos: 0) // echoes the prompt - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "ignored selection", clipboardText: "from clipboard") - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher()) - let command = AICommand(name: "ClipOnly", icon: .emoji("📋"), - inputs: [.clipboard], promptTemplate: "{input}", outputs: [.pasteAtCursor]) - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - XCTAssertEqual(executor.state, .ready(result: "from clipboard"), - "a disabled selection channel is skipped even when a selection exists") - try await executor.commit() - XCTAssertEqual(selection.pastedAtCursor, ["from clipboard"], "the clipboard input commits as a paste") - XCTAssertTrue(selection.replacedWith.isEmpty) - } - - // MARK: - Selection read ordering + reuse (change: read before the canvas takes key focus) - - /// The interaction hook fires once, AFTER the executor has acquired its input — so the controller only - /// makes the canvas key-interactive after the selection has been read (never stealing the read). - func testOnReadyForInteractionFiresAfterInputAcquired() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["x"], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "hi") - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher()) - var readyCalls = 0 - executor.onReadyForInteraction = { readyCalls += 1 } - let command = AICommand(name: "C", icon: .emoji("✨"), inputs: [.selection], - promptTemplate: "{input}", outputs: [.previewOnly]) - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - XCTAssertEqual(readyCalls, 1, "the interaction hook fires exactly once, after input acquisition") - } - - /// A language re-run REUSES the originally-acquired source instead of re-reading the selection (which - /// would fail once the canvas panel has taken key focus) — the retained-input half of the fix. - func testSetLanguageReusesRetainedInputNotAFreshRead() async throws { - let stub = StubLLMRuntime(scriptedTokens: [], interTokenDelayNanos: 0) // echoes the prompt - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "original") - var saved: [UUID: String] = [:] - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher(), - loadLanguage: { saved[$0] }, saveLanguage: { saved[$0] = $1 }) - let command = AICommand(name: "Translate", icon: .emoji("🌐"), inputs: [.selection], - promptTemplate: "{input}", outputs: [.previewOnly], - runtimeParameter: .language(default: "English")) - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - XCTAssertEqual(executor.state, .ready(result: "original")) - // The selection CHANGES after the first fire (focus moved / the panel took key). A re-translate - // must reuse the retained source, not re-read the now-different selection. - selection.selectedText = "CHANGED" - executor.setLanguage("Hebrew") - await waitUntil { if case .ready = executor.state { return true }; return false } - XCTAssertEqual(executor.state, .ready(result: "original"), - "a language re-run reuses the originally-acquired input, not a fresh (changed) selection") - } - - // MARK: - No input - - func testNoInputSurfacesAndDoesNotInvokeModel() async throws { - // A generate that would fail loudly if reached, proving the model is NOT invoked. - let stub = StubLLMRuntime(scriptedTokens: ["SHOULD NOT APPEAR"], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: nil, clipboardText: nil) // nothing anywhere - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher()) - - let command = AICommand(name: "Fix", icon: .emoji("✅"), input: .selection, - promptTemplate: "Fix: {input}", output: .replaceSelection) - executor.fire(command) - await waitUntil { executor.state == .noInput } - XCTAssertEqual(executor.state, .noInput) - XCTAssertTrue(selection.replacedWith.isEmpty, "no model run means no output write") - } - - // MARK: - Clipboard image (vision input) - - func testClipboardImageFeedsVisionRequest() async throws { - let runtime = CapturingLLMRuntime() - let manager = try await loadedManager(anyRuntime: runtime) - let imageBytes = Data([0x89, 0x50, 0x4E, 0x47, 0x01, 0x02]) // stand-in PNG bytes - let selection = FakeSelectionProvider() - selection.clipboardImageData = imageBytes - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher()) - - let command = AICommand(name: "Describe Clipboard Image", icon: .emoji("🖼"), input: .clipboardImage, - promptTemplate: "What is in this image?", output: .previewOnly) - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - XCTAssertEqual(runtime.lastRequest?.image, imageBytes, - "the clipboard image is carried on the request as the vision image input") - XCTAssertEqual(runtime.lastRequest?.requiresVision, true, "the request is a vision request") - } - - func testClipboardImageWithNoImageSurfacesNoInputAndDoesNotInvokeModel() async throws { - let runtime = CapturingLLMRuntime() - let manager = try await loadedManager(anyRuntime: runtime) - let selection = FakeSelectionProvider() // clipboardImageData is nil — nothing on the clipboard - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher()) - - let command = AICommand(name: "Describe Clipboard Image", icon: .emoji("🖼"), input: .clipboardImage, - promptTemplate: "What is in this image?", output: .previewOnly) - executor.fire(command) - await waitUntil { executor.state == .noInput } - XCTAssertEqual(executor.state, .noInput, "no clipboard image → a clean no-input state") - XCTAssertNil(runtime.lastRequest, "the model is not invoked when there is no image") - } - - // MARK: - previewOnly writes nothing - - func testPreviewOnlyCommitsWithoutWriting() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["a summary"], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "long text") - let dispatcher = FakeTaskDispatcher() - let executor = AICommandExecutor(modelManager: manager, selection: selection, dispatcher: dispatcher) - - let command = AICommand(name: "Summarize", icon: .emoji("📝"), input: .selection, - promptTemplate: "{input}", output: .previewOnly) - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - try await executor.commit() - - XCTAssertEqual(executor.state, .committed) - XCTAssertTrue(selection.replacedWith.isEmpty, "previewOnly never replaces selection") - XCTAssertTrue(selection.pastedAtCursor.isEmpty, "previewOnly never pastes") - XCTAssertTrue(dispatcher.prepared.isEmpty, "previewOnly never prepares a task") - XCTAssertTrue(dispatcher.executed.isEmpty, "previewOnly never executes a task") - } - - // MARK: - Cancellation (horizontal discard swipe) - - func testCancellationStopsGeneration() async throws { - // A slow stream so cancel lands mid-flight. - let stub = StubLLMRuntime(scriptedTokens: Array(repeating: "x", count: 50), - interTokenDelayNanos: 5_000_000) // 5 ms each - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "input") - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher()) - - let command = AICommand(name: "Slow", icon: .emoji("🐌"), input: .selection, - promptTemplate: "{input}", output: .previewOnly) - executor.fire(command) - await waitUntil { if case .streaming = executor.state { return true }; return false } - executor.cancel() - XCTAssertEqual(executor.state, .idle, "cancel resets to idle and writes nothing") - - // Deterministic proof generation truly stopped: the runtime observed and honored - // cancellation (no wall-clock sleep — waitUntil polls a condition with a failure-only deadline). - await waitUntil { stub.observedCancellation } - XCTAssertTrue(stub.observedCancellation, "the runtime observed and honored cancellation") - XCTAssertEqual(executor.state, .idle, "state stays idle after cancel (no ready appears)") - } - - // MARK: - Show the model's thinking: channel split (thinking vs response) - - /// The runtime streams `.thinking`-channel tokens BEFORE the `.response` tokens. The executor must - /// accumulate the reasoning into the observable `thinking` (so the canvas's collapsible section can - /// render it live) while STREAMING/COMMITTING only the response — the thinking must never appear in - /// the ready result nor in what commit routes to the app. - func testThinkingStreamsSeparatelyAndOnlyResponseCommits() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["Fixed ", "text"], - scriptedThinking: ["Let me think… ", "checking grammar"], - interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "teh txt") - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher()) - - let command = AICommand(name: "Fix", icon: .emoji("✅"), input: .selection, - promptTemplate: "Fix: {input}", output: .replaceSelection) - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - - // The reasoning accumulated into `thinking`... - XCTAssertEqual(executor.thinking, "Let me think… checking grammar", - "every .thinking-channel token accumulates into the observable thinking") - // ...but the result is RESPONSE-ONLY (no thinking text leaked in). - XCTAssertEqual(executor.state, .ready(result: "Fixed text"), - "the ready result is the response only — thinking never appears in it") - XCTAssertFalse(executor.state == .ready(result: "Let me think… checking grammarFixed text"), - "thinking is not prepended to the committed result") - - // And commit routes the RESPONSE only to the app — the thinking is never written. - try await executor.commit() - XCTAssertEqual(executor.state, .committed) - XCTAssertEqual(selection.replacedWith, ["Fixed text"], - "commit writes the response only; the thinking is never routed to the app") - } - - /// A fresh fire and a cancel must each CLEAR the previously-streamed thinking, so a re-run/discard - /// never shows stale reasoning. - func testThinkingResetsOnNewFireAndOnCancel() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["done"], - scriptedThinking: ["reasoning A"], - interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "input") - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher()) - let command = AICommand(name: "Echo", icon: .emoji("🔁"), input: .selection, - promptTemplate: "{input}", output: .previewOnly) - - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - XCTAssertEqual(executor.thinking, "reasoning A", "the first run accumulates its reasoning") - - // A cancel (horizontal discard) clears the thinking immediately. - executor.cancel() - XCTAssertEqual(executor.thinking, "", "cancel clears the streamed thinking") - - // A fresh fire also starts with empty thinking before its own reasoning streams in. - let stub2 = StubLLMRuntime(scriptedTokens: ["done2"], - scriptedThinking: ["reasoning B"], - interTokenDelayNanos: 0) - let manager2 = try await loadedManager(runtime: stub2) - let executor2 = AICommandExecutor(modelManager: manager2, selection: selection, - dispatcher: FakeTaskDispatcher()) - executor2.fire(command) - await waitUntil { if case .ready = executor2.state { return true }; return false } - XCTAssertEqual(executor2.thinking, "reasoning B", - "a fresh fire shows only its own reasoning, never the prior run's") - } - - // MARK: - Commit-vs-ignore decision (down-swipe gate) - - func testStateIsCommittableOnlyForReadyResults() { - // `resolveCanvasCommit` (the DOWN-swipe commit path) commits only a committable state; a DOWN - // swipe in any other state is IGNORED — the user waits, and only a horizontal discard swipe - // cancels generation. (`.reviewingAction` is also committable — exercised by the task-path tests.) - XCTAssertTrue(AICommandExecutor.State.ready(result: "x").isCommittable) - XCTAssertFalse(AICommandExecutor.State.idle.isCommittable) - XCTAssertFalse(AICommandExecutor.State.loadingModel.isCommittable) - XCTAssertFalse(AICommandExecutor.State.noInput.isCommittable) - XCTAssertFalse(AICommandExecutor.State.streaming(partial: "half").isCommittable, - "a DOWN swipe while still streaming must NOT commit — it is ignored (the user waits); only a horizontal swipe discards") - XCTAssertFalse(AICommandExecutor.State.declined(reason: "not a meeting").isCommittable) - XCTAssertFalse(AICommandExecutor.State.failed(message: "boom").isCommittable) - XCTAssertFalse(AICommandExecutor.State.committed.isCommittable) - } - - // MARK: - Whitespace-only input counts as empty - - func testWhitespaceOnlyInputSurfacesNoInput() async throws { - // Regression: a whitespace-only selection AND clipboard count as empty → .noInput, and the - // model is never run on effectively-empty content. - let stub = StubLLMRuntime(scriptedTokens: ["SHOULD NOT APPEAR"], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: " ", clipboardText: "\n\t ") - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher()) - let command = AICommand(name: "Blank", icon: .emoji("⬜"), input: .selection, - promptTemplate: "{input}", output: .replaceSelection) - executor.fire(command) - await waitUntil { executor.state == .noInput } - XCTAssertEqual(executor.state, .noInput, "whitespace-only selection+clipboard surfaces no-input") - XCTAssertTrue(selection.replacedWith.isEmpty, "the model is never run on whitespace-only input") - } - - // MARK: - Task path: review SKIPPED when confirmBeforeRun is OFF (stored value honored) - - func testTaskOutputWithConfirmOffExecutesWithoutReviewGate() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["unused for tasks"], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "lunch with sam tomorrow") - let dispatcher = FakeTaskDispatcher() - let executor = AICommandExecutor(modelManager: manager, selection: selection, dispatcher: dispatcher) - - // A trusted side-effecting command: the user disabled confirmation. It must be honored — the - // side effect commits directly (prepare → execute), with NO armed-confirmation review gate. - let command = AICommand(name: "Add to Calendar", icon: .emoji("📅"), input: .selection, - promptTemplate: "{input}", output: .runTask(.addToCalendar), - confirmBeforeRun: false) - XCTAssertFalse(command.confirmBeforeRun) - - executor.fire(command) - // With review OFF, the executor commits the side effect itself — no separate commit() call. - await waitUntil { executor.state == .committed } - XCTAssertEqual(executor.state, .committed) - - XCTAssertEqual(dispatcher.prepared.count, 1, "the task output prepares through the dispatcher") - let prep = try XCTUnwrap(dispatcher.prepared.first) - XCTAssertEqual(prep.kind, .addToCalendar, "the calendar task kind is routed to prepare") - XCTAssertEqual(prep.prompt, "lunch with sam tomorrow", "the resolved prompt is handed to prepare") - XCTAssertEqual(dispatcher.executed.count, 1, "the side effect fires exactly once (review skipped)") - XCTAssertTrue(selection.replacedWith.isEmpty, "a task output writes nothing in-place") - } - - // MARK: - Task path: review SHOWN when confirmBeforeRun is ON; fires only on commit - - func testSendToOutputShowsReviewThenFiresOnCommit() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["unused"], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "note this") - let dispatcher = FakeTaskDispatcher() - let executor = AICommandExecutor(modelManager: manager, selection: selection, dispatcher: dispatcher) - - let command = AICommand(name: "Send", icon: .emoji("📤"), input: .selection, - promptTemplate: "{input}", output: .sendTo(.shortcut(name: "Log"))) - XCTAssertTrue(command.confirmBeforeRun, "send-to defaults confirm ON") - - executor.fire(command) - // Review ON → land in the armed-confirmation state; NOTHING has fired yet. - await waitUntil { if case .reviewingAction = executor.state { return true }; return false } - XCTAssertEqual(dispatcher.prepared.first?.kind, .sendTo(.shortcut(name: "Log")), - "a sendTo output maps to the sendTo task kind in prepare") - XCTAssertTrue(dispatcher.executed.isEmpty, "no side effect fires while only reviewing") - - // The commit (down-swipe) fires the reviewed side effect. - try await executor.commit() - XCTAssertEqual(executor.state, .committed) - XCTAssertEqual(dispatcher.executed.count, 1, "the reviewed side effect fires on commit") - } - - // MARK: - Task path: discard while reviewing fires NO side effect - - func testDiscardWhileReviewingFiresNoSideEffect() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["unused"], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "note this") - let dispatcher = FakeTaskDispatcher() - let executor = AICommandExecutor(modelManager: manager, selection: selection, dispatcher: dispatcher) - - let command = AICommand(name: "Send", icon: .emoji("📤"), input: .selection, - promptTemplate: "{input}", output: .sendTo(.shortcut(name: "Log"))) - executor.fire(command) - await waitUntil { if case .reviewingAction = executor.state { return true }; return false } - - executor.cancel() // discard before committing - XCTAssertEqual(executor.state, .idle, "discard resets to idle") - XCTAssertTrue(dispatcher.executed.isEmpty, "discarding a reviewed task fires no side effect") - } - - // MARK: - Task path: a declined review surfaces .declined and fires nothing - - func testTaskDeclineSurfacesDeclinedAndFiresNothing() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["unused"], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "not a meeting at all") - let dispatcher = FakeTaskDispatcher() - dispatcher.reviewToReturn = .declined(reason: "This text is not a meeting") - let executor = AICommandExecutor(modelManager: manager, selection: selection, dispatcher: dispatcher) - - let command = AICommand(name: "Add to Calendar", icon: .emoji("📅"), input: .selection, - promptTemplate: "{input}", output: .runTask(.addToCalendar)) - executor.fire(command) - await waitUntil { if case .declined = executor.state { return true }; return false } - XCTAssertEqual(executor.state, .declined(reason: "This text is not a meeting")) - XCTAssertTrue(dispatcher.executed.isEmpty, "a declined task fires no side effect") - } - - // MARK: - Task path: an unavailable review surfaces .failed (no malformed side effect) - - func testTaskUnavailableSurfacesFailedAndFiresNothing() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["unused"], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "lunch") - let dispatcher = FakeTaskDispatcher() - dispatcher.reviewToReturn = .unavailable(reason: "Couldn't produce a valid action.") - let executor = AICommandExecutor(modelManager: manager, selection: selection, dispatcher: dispatcher) - - let command = AICommand(name: "Add to Calendar", icon: .emoji("📅"), input: .selection, - promptTemplate: "{input}", output: .runTask(.addToCalendar)) - executor.fire(command) - await waitUntil { if case .failed = executor.state { return true }; return false } - XCTAssertEqual(executor.state, .failed(message: "Couldn't produce a valid action.")) - XCTAssertTrue(dispatcher.executed.isEmpty, "an unavailable task dispatches no side effect") - } - - // MARK: - Task path: a throwing commit surfaces .failed with a human message (Fix 3 + Fix 2) - - func testReviewedCommitThatThrowsSurfacesFailedWithHumanMessage() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["unused"], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "lunch with sam tomorrow") - let dispatcher = FakeTaskDispatcher() - // prepare returns the default ready `.action`; execute THROWS the calendar-denied error (the - // default-calendar path is armed-confirmation, so this lands on commit). - dispatcher.executeError = TaskError.calendarPermissionDenied - let executor = AICommandExecutor(modelManager: manager, selection: selection, dispatcher: dispatcher) - - // confirmBeforeRun ON (default) → drive to the armed-confirmation review first. - let command = AICommand(name: "Add to Calendar", icon: .emoji("📅"), input: .selection, - promptTemplate: "{input}", output: .runTask(.addToCalendar)) - XCTAssertTrue(command.confirmBeforeRun, "calendar defaults confirm ON") - executor.fire(command) - await waitUntil { if case .reviewingAction = executor.state { return true }; return false } - XCTAssertTrue(dispatcher.executed.isEmpty, "nothing fires while only reviewing") - - // commit() must rethrow the error AND leave the executor in .failed with a readable message. - var thrown: Error? - do { - try await executor.commit() - XCTFail("commit should rethrow the side-effect error") - } catch { - thrown = error - } - XCTAssertEqual(thrown as? TaskError, .calendarPermissionDenied, "commit rethrows the task error") - - guard case let .failed(message) = executor.state else { - return XCTFail("a throwing commit surfaces .failed, got \(executor.state)") - } - XCTAssertTrue(message.contains("Calendar"), - "the message is human-facing and mentions Calendar") - XCTAssertNotEqual(message, "calendarPermissionDenied", - "NOT the raw enum case name (Fix 2 surfaces the LocalizedError description)") - } - - // MARK: - Runtime language parameter: persisted default + in-canvas re-run - - func testActiveLanguageResolvesDeclaredDefaultAtColdStart() async throws { - let stub = StubLLMRuntime(scriptedTokens: [], interTokenDelayNanos: 0) // echoes the prompt - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "hello") - var store: [UUID: String] = [:] - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher(), - loadLanguage: { store[$0] }, saveLanguage: { store[$0] = $1 }) - let command = AICommand(name: "Translate", icon: .emoji("🌍"), input: .selection, - promptTemplate: "Translate to {lang}:\n{input}", output: .previewOnly, - runtimeParameter: .language(default: "English")) - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - XCTAssertEqual(executor.activeLanguage, "English", "cold start uses the declared default") - XCTAssertEqual(executor.state, .ready(result: "Translate to English:\nhello"), - "{lang} resolves to the active language in the streamed prompt") - } - - func testSetLanguagePersistsAndRetranslatesInPlace() async throws { - let stub = StubLLMRuntime(scriptedTokens: [], interTokenDelayNanos: 0) // echoes the prompt - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "hello") - var store: [UUID: String] = [:] - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher(), - loadLanguage: { store[$0] }, saveLanguage: { store[$0] = $1 }) - let command = AICommand(name: "Translate", icon: .emoji("🌍"), input: .selection, - promptTemplate: "Translate to {lang}:\n{input}", output: .previewOnly, - runtimeParameter: .language(default: "English")) - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - - // Repick a language in the canvas: re-runs in place AND persists for the next run. - executor.setLanguage("Hebrew") - await waitUntil { executor.state == .ready(result: "Translate to Hebrew:\nhello") } - XCTAssertEqual(executor.activeLanguage, "Hebrew") - XCTAssertEqual(store[command.id], "Hebrew", "the choice is persisted per command") - - // A fresh fire now defaults to the remembered language (the next-run default). - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - XCTAssertEqual(executor.activeLanguage, "Hebrew", "next run defaults to the remembered language") - XCTAssertEqual(executor.state, .ready(result: "Translate to Hebrew:\nhello")) - } - - func testSetLanguageIsIgnoredWithoutARuntimeParameter() async throws { - let stub = StubLLMRuntime(scriptedTokens: [], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "hello") - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher()) - let command = AICommand(name: "Echo", icon: .emoji("🔁"), input: .selection, - promptTemplate: "{input}", output: .previewOnly) // no runtimeParameter - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - XCTAssertNil(executor.activeLanguage, "a command with no runtime parameter has no active language") - executor.setLanguage("Hebrew") // must be a no-op - XCTAssertNil(executor.activeLanguage, "setLanguage is ignored without a language parameter") - XCTAssertEqual(executor.state, .ready(result: "hello"), "the result is unchanged") - } - - // MARK: - Screen-region capture outcomes (picker pre-supplies the capture) - - func testScreenRegionWithUnavailableCaptureSurfacesNoInput() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["nope"], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub, capabilities: [.text, .vision]) - let selection = FakeSelectionProvider() - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher()) - - let command = AICommand(name: "Describe", icon: .emoji("👁"), input: .screenRegion, - promptTemplate: "What's here?", output: .previewOnly) - executor.fire(command, screenCapture: .unavailable) // picker captured nothing - await waitUntil { executor.state == .noInput } - XCTAssertEqual(executor.state, .noInput, "an unavailable capture is no-input") - } - - func testScreenRegionWithNoSuppliedCaptureSurfacesNoInput() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["nope"], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub, capabilities: [.text, .vision]) - let selection = FakeSelectionProvider() - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher()) - - let command = AICommand(name: "Describe", icon: .emoji("👁"), input: .screenRegion, - promptTemplate: "What's here?", output: .previewOnly) - executor.fire(command) // no capture supplied at all (defensive) → no-input, no model run - await waitUntil { executor.state == .noInput } - XCTAssertEqual(executor.state, .noInput, "a screen-region fire with no supplied capture is no-input") - } - - func testScreenRegionWithCapturedImageStreamsResult() async throws { - let runtime = CapturingLLMRuntime() - let manager = try await loadedManager(anyRuntime: runtime) - let png = Data([0x89, 0x50, 0x4E, 0x47, 0x09]) - let executor = AICommandExecutor(modelManager: manager, selection: FakeSelectionProvider(), - dispatcher: FakeTaskDispatcher()) - - let command = AICommand(name: "Describe", icon: .emoji("👁"), input: .screenRegion, - promptTemplate: "What's here?", output: .previewOnly) - executor.fire(command, screenCapture: .captured(png)) - await waitUntil { if case .ready = executor.state { return true }; return false } - XCTAssertEqual(runtime.lastRequest?.image, png, "the picker's captured region is the vision image") - XCTAssertEqual(runtime.lastRequest?.requiresVision, true) - } - - // MARK: - Honesty (D5): a non-landed replaceSelection surfaces .failed, not .committed - - func testNonLandedReplaceSelectionSurfacesFailed() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["Fixed text"], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "teh txt") - selection.replaceLands = false // the write does NOT actually land in the app - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher()) - - let command = AICommand(name: "Fix", icon: .emoji("✅"), input: .selection, - promptTemplate: "Fix: {input}", output: .replaceSelection) - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - try await executor.commit() - - guard case let .failed(message) = executor.state else { - return XCTFail("a write that didn't land must surface .failed, got \(executor.state)") - } - XCTAssertFalse(message.isEmpty, "the failure carries a clean message") - XCTAssertTrue(selection.replacedWith == ["Fixed text"], "the write was attempted") - } - - func testNonLandedPasteSurfacesFailed() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["summary"], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "long text") - selection.pasteLands = false - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher()) - let command = AICommand(name: "Paste", icon: .emoji("📋"), input: .selection, - promptTemplate: "{input}", output: .pasteAtCursor) - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - try await executor.commit() - guard case .failed = executor.state else { - return XCTFail("a paste that didn't land must surface .failed, got \(executor.state)") - } - } - - // MARK: - Honesty (D5): a sink throw surfaces a clean task-failed message (never a raw dump) - - func testSinkFailureSurfacesCleanTaskFailedMessage() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["unused"], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub) - let selection = FakeSelectionProvider(selectedText: "note this") - let dispatcher = FakeTaskDispatcher() - dispatcher.executeError = TaskError.sinkFailed("Could not save the note to “Roadmap”.") - let executor = AICommandExecutor(modelManager: manager, selection: selection, dispatcher: dispatcher) - - let command = AICommand(name: "Save", icon: .emoji("💾"), input: .selection, - promptTemplate: "{input}", output: .runTask(.saveToProject(project: "Roadmap"))) - executor.fire(command) - await waitUntil { if case .reviewingAction = executor.state { return true }; return false } - _ = try? await executor.commit() - guard case let .failed(message) = executor.state else { - return XCTFail("a thrown sink failure must surface .failed, got \(executor.state)") - } - XCTAssertEqual(message, "Could not save the note to “Roadmap”.", - "the clean sinkFailed message is shown verbatim, not a raw error dump") - } - - // MARK: - Honesty (D5): a Screen-Recording permission gap names the permission, not "no input" - - func testScreenRecordingPermissionGapSurfacesFailedNamingThePermission() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["nope"], interTokenDelayNanos: 0) - let manager = try await loadedManager(runtime: stub, capabilities: [.text, .vision]) - let selection = FakeSelectionProvider() - let executor = AICommandExecutor(modelManager: manager, selection: selection, - dispatcher: FakeTaskDispatcher()) - let command = AICommand(name: "Describe", icon: .emoji("👁"), input: .screenRegion, - promptTemplate: "What's here?", output: .previewOnly) - executor.fire(command, screenCapture: .permissionDenied) // picker hit a Screen-Recording gap - await waitUntil { if case .failed = executor.state { return true }; return false } - guard case let .failed(message) = executor.state else { - return XCTFail("a permission gap must surface .failed (not .noInput), got \(executor.state)") - } - XCTAssertTrue(message.contains("Screen Recording"), "the message names the missing permission") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/AICommandFoldInTests.swift b/Tests/ThreeFingerSwitcherTests/AICommandFoldInTests.swift deleted file mode 100644 index 2a918fc..0000000 --- a/Tests/ThreeFingerSwitcherTests/AICommandFoldInTests.swift +++ /dev/null @@ -1,183 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for the configuration-hub AI fold-in: AI commands are now PERSISTED band items inside the -/// `Favorites` record (not a separate `AICommandStore`). Covers: the `Favorites` Codable round-trip -/// with `.aiCommand` items, the fresh-install seed including an editable "AI" band, and the one-time, -/// idempotent migration that imports a legacy `aiCommands` record into a normal "AI" band. -@MainActor -final class AICommandFoldInTests: XCTestCase { - - // MARK: - Helpers - - /// A throwaway, isolated defaults suite (removed at teardown). - private func makeDefaults(_ label: String = #function) -> UserDefaults { - let suite = "AICommandFoldInTests.\(label).\(UUID().uuidString)" - addTeardownBlock { UserDefaults().removePersistentDomain(forName: suite) } - return UserDefaults(suiteName: suite)! - } - - private func command(_ name: String, id: UUID = UUID()) -> AICommand { - AICommand(id: id, name: name, icon: .sfSymbol("sparkles"), - tint: ItemColor(red: 0.5, green: 0.4, blue: 0.9), - input: .selection, promptTemplate: "{input}", output: .previewOnly) - } - - /// The legacy on-disk shape the former `AICommandStore` persisted under "aiCommands". - private struct LegacyStored: Codable { var schemaVersion: Int; var commands: [AICommand] } - - private func writeLegacyCommands(_ commands: [AICommand], to defaults: UserDefaults) { - let data = try! JSONEncoder().encode(LegacyStored(schemaVersion: 1, commands: commands)) - defaults.set(data, forKey: "aiCommands") - } - - /// Persist a pre-fold-in (schemaVersion 1) Favorites record under "favorites". - private func writeV1Favorites(_ bands: [ContextBand], to defaults: UserDefaults) { - var fav = Favorites(bands: bands, homeBandID: bands.first?.id, homeColumn: 0) - fav.schemaVersion = 1 - let data = try! JSONEncoder().encode(fav) - defaults.set(data, forKey: "favorites") - } - - // MARK: - Codable round-trip with .aiCommand items - - func testFavoritesRoundTripWithAICommandItem() throws { - let cmd = AICommand(name: "Fix Grammar", icon: .sfSymbol("text.badge.checkmark"), - tint: ItemColor(red: 0.2, green: 0.7, blue: 0.4), - input: .selection, - promptTemplate: "Fix: {input}", - output: .runTask(.addToCalendar), - confirmBeforeRun: true) - let item = AIBand.item(for: cmd) - let band = ContextBand(name: "Work", color: ItemColor(red: 0, green: 0, blue: 1), items: [item]) - let fav = Favorites(bands: [band]) - - let data = try JSONEncoder().encode(fav) - let decoded = try JSONDecoder().decode(Favorites.self, from: data) - - guard case let .aiCommand(roundTripped)? = decoded.bands.first?.items.first?.kind else { - return XCTFail("the .aiCommand item must survive a Favorites encode/decode") - } - XCTAssertEqual(roundTripped, cmd, "the embedded AICommand round-trips intact") - XCTAssertEqual(decoded.bands.first?.items.first?.id, cmd.id, "item id mirrors the command id") - } - - // MARK: - Fresh-install seed - - func testFreshSeedIncludesEditableAIBand() { - let store = FavoritesStore(defaults: makeDefaults()) - guard let ai = store.favorites.bands.first(where: { AIBand.isAIBand($0) }) else { - return XCTFail("a fresh install seeds an \"AI\" band") - } - XCTAssertEqual(ai.name, "AI") - XCTAssertEqual(ai.items.count, AIBand.seeded().count, "seeded with the starter command set") - XCTAssertTrue(ai.items.allSatisfy { if case .aiCommand = $0.kind { return true } else { return false } }, - "every seeded AI item is an .aiCommand") - // It's a NORMAL band: editable/removable like any other. - store.removeBand(ai.id) - XCTAssertFalse(store.favorites.bands.contains { AIBand.isAIBand($0) }, - "the seeded AI band is a normal band the user can delete") - } - - func testSchemaVersionIsCurrentAfterSeed() { - let store = FavoritesStore(defaults: makeDefaults()) - XCTAssertEqual(store.favorites.schemaVersion, Favorites.currentSchemaVersion) - } - - // MARK: - Migration: legacy commands fold into a normal AI band - - func testMigrationImportsLegacyCommandsPreservingIDsAndOrder() { - let defaults = makeDefaults() - let a = command("Alpha"), b = command("Bravo"), c = command("Charlie") - let dev = ContextBand(name: "Dev", color: ItemColor(red: 0, green: 0, blue: 1)) - writeV1Favorites([dev], to: defaults) - writeLegacyCommands([a, b, c], to: defaults) - - let store = FavoritesStore(defaults: defaults) - - guard let ai = store.favorites.bands.first(where: { AIBand.isAIBand($0) }) else { - return XCTFail("migration appends an \"AI\" band") - } - XCTAssertEqual(ai.items.map(\.title), ["Alpha", "Bravo", "Charlie"], "order preserved") - XCTAssertEqual(ai.items.map(\.id), [a.id, b.id, c.id], "command ids preserved") - // The pre-existing band is untouched and the AI band is appended after it. - XCTAssertEqual(store.favorites.bands.first?.name, "Dev") - XCTAssertTrue(AIBand.isAIBand(store.favorites.bands.last!), "AI band appended last") - XCTAssertEqual(store.favorites.schemaVersion, Favorites.currentSchemaVersion, "version bumped") - XCTAssertNil(defaults.data(forKey: "aiCommands"), "legacy key retired after a successful fold-in") - } - - func testMigrationIsIdempotentAcrossReloads() { - let defaults = makeDefaults() - writeV1Favorites([ContextBand(name: "Dev", color: ItemColor(red: 0, green: 0, blue: 1))], to: defaults) - writeLegacyCommands([command("Alpha"), command("Bravo")], to: defaults) - - _ = FavoritesStore(defaults: defaults) // first launch: folds in - let reloaded = FavoritesStore(defaults: defaults) // second launch: must NOT duplicate - - let aiBands = reloaded.favorites.bands.filter { AIBand.isAIBand($0) } - XCTAssertEqual(aiBands.count, 1, "exactly one AI band after a reload (no duplicate)") - XCTAssertEqual(aiBands.first?.items.count, 2) - } - - func testMigrationNeverOptedInSeedsDefaultAIBand() { - // A user who never opted into AI (v1 record, NO aiCommands key) gets the default "AI" band - // seeded on upgrade, for discoverability (design D4). - let defaults = makeDefaults() - writeV1Favorites([ContextBand(name: "Dev", color: ItemColor(red: 0, green: 0, blue: 1))], to: defaults) - - let store = FavoritesStore(defaults: defaults) - guard let ai = store.favorites.bands.first(where: { AIBand.isAIBand($0) }) else { - return XCTFail("never-opted-in upgrade seeds the default AI band (design D4)") - } - XCTAssertEqual(ai.items.count, AIBand.seeded().count, "seeded with the starter command set") - XCTAssertEqual(store.favorites.schemaVersion, Favorites.currentSchemaVersion, "marked migrated") - // And idempotent: a reload does not add a second AI band. - let reloaded = FavoritesStore(defaults: defaults) - XCTAssertEqual(reloaded.favorites.bands.filter { AIBand.isAIBand($0) }.count, 1) - } - - func testAICommandMovesBetweenBands() { - // Spec: "An AI command moves between bands" / "movable between bands like any other item." - let store = FavoritesStore(defaults: makeDefaults()) - let dev = store.addBand(name: "Dev", color: ItemColor(red: 0, green: 0, blue: 1)) - let work = store.addBand(name: "Work", color: ItemColor(red: 0, green: 1, blue: 0)) - let cmd = command("Fix") - let item = AIBand.item(for: cmd) - store.addItem(item, toBand: dev) - XCTAssertTrue(store.favorites.bands.first { $0.id == dev }!.items.contains { $0.id == item.id }) - - store.moveItem(item.id, fromBand: dev, toBand: work) - - XCTAssertFalse(store.favorites.bands.first { $0.id == dev }!.items.contains { $0.id == item.id }, - "the item left the source band") - let moved = store.favorites.bands.first { $0.id == work }!.items.first { $0.id == item.id } - XCTAssertNotNil(moved, "the item appears in the destination band") - guard case let .aiCommand(c)? = moved?.kind else { return XCTFail("kind preserved as .aiCommand") } - XCTAssertEqual(c, cmd, "the embedded command survives the move") - } - - func testMigrationWithEmptyLegacyRecordAddsNoAIBand() { - let defaults = makeDefaults() - writeV1Favorites([ContextBand(name: "Dev", color: ItemColor(red: 0, green: 0, blue: 1))], to: defaults) - writeLegacyCommands([], to: defaults) // opted in once, but no commands - - let store = FavoritesStore(defaults: defaults) - XCTAssertFalse(store.favorites.bands.contains { AIBand.isAIBand($0) }, - "an empty legacy record imports nothing") - XCTAssertNil(defaults.data(forKey: "aiCommands"), "the empty legacy key is still retired") - } - - func testMigrationDoesNotDuplicateWhenAIBandAlreadyPresent() { - // Defensive idempotency: a v1 record that somehow already has an AI band must not get a second. - let defaults = makeDefaults() - let existingAI = AIBand.band(from: [command("Existing")]) - writeV1Favorites([existingAI], to: defaults) - writeLegacyCommands([command("Legacy")], to: defaults) - - let store = FavoritesStore(defaults: defaults) - let aiBands = store.favorites.bands.filter { AIBand.isAIBand($0) } - XCTAssertEqual(aiBands.count, 1, "never appends a second AI band") - XCTAssertEqual(aiBands.first?.items.map(\.title), ["Existing"], "keeps the existing AI band as-is") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/AICommandTests.swift b/Tests/ThreeFingerSwitcherTests/AICommandTests.swift deleted file mode 100644 index 0d6985e..0000000 --- a/Tests/ThreeFingerSwitcherTests/AICommandTests.swift +++ /dev/null @@ -1,245 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for the AI command value model (spec: "AI command value model and persistence"): Codable -/// round-trip, the confirm-defaults-on-for-side-effects-but-honored invariant (design D6), and the -/// capability derivation that drives model selection. -final class AICommandTests: XCTestCase { - - // MARK: - Codable round-trip - - func testCommandRoundTripsThroughCodable() throws { - let command = AICommand( - name: "Fix Grammar", - icon: .sfSymbol("text.badge.checkmark"), - tint: ItemColor(red: 0.25, green: 0.72, blue: 0.40), - input: .selection, - promptTemplate: "Fix:\n{input}", - output: .replaceSelection, - model: .onDevice(modelID: "gemma-4-31b"), - confirmBeforeRun: false - ) - let data = try JSONEncoder().encode(command) - let decoded = try JSONDecoder().decode(AICommand.self, from: data) - XCTAssertEqual(decoded, command, "a command survives an encode/decode round-trip unchanged") - } - - func testTaskAndDestinationOutputsRoundTrip() throws { - let cases: [OutputTarget] = [ - .replaceSelection, - .pasteAtCursor, - .previewOnly, - .runTask(.addToCalendar), - .runTask(.saveToProject(project: "Inbox")), - .runTask(.openToolWithPayload(tool: "com.example.tool")), - .runTask(.sendTo(.shortcut(name: "Log It"))), - .sendTo(.urlScheme("things:///add?title=")), - .sendTo(.shell(command: "tee -a log.txt")) - ] - for output in cases { - let command = AICommand(name: "C", icon: .emoji("✨"), input: .none, - promptTemplate: "{date}", output: output) - let data = try JSONEncoder().encode(command) - let decoded = try JSONDecoder().decode(AICommand.self, from: data) - XCTAssertEqual(decoded, command, "output \(output) (as a set) round-trips") - XCTAssertEqual(decoded.outputs, AICommand.migrate(output: output), - "output \(output) migrates to its capability set") - } - } - - // MARK: - RuntimeParameter Codable (new shape + legacy back-compat) - - func testRuntimeParameterRoundTripsBothFactories() throws { - for param: RuntimeParameter in [.language(default: "Hebrew"), .codeLanguage(default: "Rust")] { - let data = try JSONEncoder().encode(param) - let decoded = try JSONDecoder().decode(RuntimeParameter.self, from: data) - XCTAssertEqual(decoded, param, "the runtime parameter round-trips with its own option set") - } - } - - func testLegacyLanguageParameterPayloadStillDecodes() throws { - // A band persisted BEFORE the case was renamed stored `{"language": {"default": …}}` (no - // options). It must still decode — as a `.languageChoice` defaulting to the human-language list — - // so an existing install's AI band keeps loading (no migration pass runs). - let legacy = Data(#"{"language":{"default":"Spanish"}}"#.utf8) - let decoded = try JSONDecoder().decode(RuntimeParameter.self, from: legacy) - XCTAssertEqual(decoded.languageDefault, "Spanish", "the legacy default survives") - XCTAssertEqual(decoded.options, AILanguages.all, - "a legacy payload (no options) defaults to the human-language list") - } - - // MARK: - Per-command reasoning override (resolution + legacy back-compat) - - func testResolvedReasoningResolvesAgainstGlobalDefault() { - func cmd(_ r: AIReasoning?) -> AICommand { - AICommand(name: "R", icon: .emoji("🧠"), input: .selection, - promptTemplate: "{input}", output: .previewOnly, reasoning: r) - } - // .on / .off pin regardless of the global; nil follows the global default. - XCTAssertTrue(cmd(.on).resolvedReasoning(globalDefault: false), ".on forces reasoning on") - XCTAssertFalse(cmd(.off).resolvedReasoning(globalDefault: true), ".off forces reasoning off") - XCTAssertTrue(cmd(nil).resolvedReasoning(globalDefault: true), "nil follows the global (on)") - XCTAssertFalse(cmd(nil).resolvedReasoning(globalDefault: false), "nil follows the global (off)") - } - - func testLegacyCommandPayloadWithoutReasoningDecodesToNil() throws { - // A command persisted before `reasoning` existed has no key; synthesized Codable's - // decodeIfPresent must yield nil (⇒ follow the global default), not fail to decode. - let original = AICommand(name: "Fix", icon: .sfSymbol("checkmark"), input: .selection, - promptTemplate: "{input}", output: .replaceSelection) - var json = try JSONSerialization.jsonObject(with: try JSONEncoder().encode(original)) as! [String: Any] - json.removeValue(forKey: "reasoning") // simulate a pre-feature payload - let stripped = try JSONSerialization.data(withJSONObject: json) - let decoded = try JSONDecoder().decode(AICommand.self, from: stripped) - XCTAssertNil(decoded.reasoning, "a legacy command without the key decodes to nil (follows global)") - } - - // MARK: - confirmBeforeRun default + honored - - func testConfirmDefaultsOnForSideEffectingOutputs() { - let task = AICommand(name: "Cal", icon: .emoji("📅"), input: .selection, - promptTemplate: "{input}", output: .runTask(.addToCalendar)) - XCTAssertTrue(task.confirmBeforeRun, "a task output defaults confirmBeforeRun ON") - - let sendTo = AICommand(name: "Send", icon: .emoji("📤"), input: .selection, - promptTemplate: "{input}", output: .sendTo(.shortcut(name: "X"))) - XCTAssertTrue(sendTo.confirmBeforeRun, "a send-to output defaults confirmBeforeRun ON") - } - - func testConfirmDefaultsOffForInPlaceOutputs() { - for output: OutputTarget in [.replaceSelection, .pasteAtCursor, .previewOnly] { - let command = AICommand(name: "C", icon: .emoji("✨"), input: .selection, - promptTemplate: "{input}", output: output) - XCTAssertFalse(command.confirmBeforeRun, - "in-place output \(output) defaults confirmBeforeRun OFF") - } - } - - func testExplicitConfirmFalseIsHonoredNotOverriddenForSideEffectingOutput() throws { - // A user disables confirmation on a trusted side-effecting command. - let command = AICommand(name: "Cal", icon: .emoji("📅"), input: .selection, - promptTemplate: "{input}", output: .runTask(.addToCalendar), - confirmBeforeRun: false) - XCTAssertFalse(command.confirmBeforeRun, - "an explicit false is taken verbatim at creation, not re-defaulted to true") - - // ...and the stored value survives persistence (never recomputed on decode). - let data = try JSONEncoder().encode(command) - let decoded = try JSONDecoder().decode(AICommand.self, from: data) - XCTAssertFalse(decoded.confirmBeforeRun, "the stored false is honored after a round-trip") - } - - func testDefaultConfirmHelperMatchesSideEffectClassification() { - XCTAssertTrue(AICommand.defaultConfirmBeforeRun(for: [.runTask(.addToCalendar)])) - XCTAssertTrue(AICommand.defaultConfirmBeforeRun(for: [.sendTo(.shortcut(name: "X"))])) - // A set containing ANY side-effecting member confirms; a purely in-place set does not. - XCTAssertTrue(AICommand.defaultConfirmBeforeRun(for: [.previewOnly, .runTask(.newContact)])) - XCTAssertFalse(AICommand.defaultConfirmBeforeRun(for: [.replaceSelection, .pasteAtCursor, .previewOnly])) - XCTAssertFalse(AICommand.defaultConfirmBeforeRun(for: [.pasteAtCursor])) - } - - // MARK: - Capability sets: defaults, migration, resolution (change `ai-action-context-resolution`) - - func testDefaultCapabilitySetsAreAllOn() { - XCTAssertEqual(AICommand.defaultInputs, [.selection, .clipboard, .clipboardImage], - "a new command's inputs default to the ambient cascade") - XCTAssertEqual(AICommand.defaultOutputs, [.replaceSelection, .pasteAtCursor, .previewOnly], - "a new command's in-place outputs default all-on") - } - - func testLegacyScalarMigrationMapping() { - // The pure migration table (design D5): behavior-preserving, with the selection→clipboard fallback - // made explicit and replaceSelection gaining pasteAtCursor. - XCTAssertEqual(AICommand.migrate(input: .selection), [.selection, .clipboard]) - XCTAssertEqual(AICommand.migrate(input: .clipboard), [.clipboard]) - XCTAssertEqual(AICommand.migrate(input: .clipboardImage), [.clipboardImage]) - XCTAssertEqual(AICommand.migrate(input: .screenRegion), [.screenRegion]) - XCTAssertEqual(AICommand.migrate(input: .none), []) - XCTAssertEqual(AICommand.migrate(output: .replaceSelection), [.replaceSelection, .pasteAtCursor]) - XCTAssertEqual(AICommand.migrate(output: .pasteAtCursor), [.pasteAtCursor]) - XCTAssertEqual(AICommand.migrate(output: .previewOnly), [.previewOnly]) - XCTAssertEqual(AICommand.migrate(output: .runTask(.addToCalendar)), [.runTask(.addToCalendar)]) - } - - func testLegacyScalarInputOutputMigrateOnDecode() throws { - // A command persisted with the OLD single `input`/`output` scalar keys must decode into the sets. - // Rewrite a modern command's JSON to that legacy shape, then decode. - let modern = AICommand(name: "Fix", icon: .sfSymbol("checkmark"), - inputs: [.selection], promptTemplate: "{input}", outputs: [.previewOnly]) - var dict = try JSONSerialization.jsonObject(with: JSONEncoder().encode(modern)) as! [String: Any] - dict.removeValue(forKey: "inputs") - dict.removeValue(forKey: "outputs") - dict["input"] = try JSONSerialization.jsonObject( - with: JSONEncoder().encode(InputSource.selection), options: [.fragmentsAllowed]) - dict["output"] = try JSONSerialization.jsonObject( - with: JSONEncoder().encode(OutputTarget.replaceSelection)) - let data = try JSONSerialization.data(withJSONObject: dict) - let decoded = try JSONDecoder().decode(AICommand.self, from: data) - XCTAssertEqual(decoded.inputs, [.selection, .clipboard], "legacy .selection → {selection, clipboard}") - XCTAssertEqual(decoded.outputs, [.replaceSelection, .pasteAtCursor], - "legacy .replaceSelection → {replaceSelection, pasteAtCursor}") - } - - func testInPlaceCommitPlanResolvesFromChannelAndOutputs() { - let all = AICommand.defaultOutputs // {replace, paste, preview} - // A selection source replaces; a clipboard source pastes (the core "it knows what to do"). - XCTAssertEqual(AICommand.inPlaceCommitPlan(resolvedWasSelection: true, outputs: all), .replaceSelection) - XCTAssertEqual(AICommand.inPlaceCommitPlan(resolvedWasSelection: false, outputs: all), .pasteAtCursor) - // Replace disabled → a selection still pastes. - XCTAssertEqual(AICommand.inPlaceCommitPlan(resolvedWasSelection: true, - outputs: [.pasteAtCursor, .previewOnly]), .pasteAtCursor) - // Preview-only (read-only understanding command) writes nothing even with a selection. - XCTAssertEqual(AICommand.inPlaceCommitPlan(resolvedWasSelection: true, outputs: [.previewOnly]), .preview) - // Only replace enabled, no selection → still replace (SelectionService pastes when not settable). - XCTAssertEqual(AICommand.inPlaceCommitPlan(resolvedWasSelection: false, outputs: [.replaceSelection]), - .replaceSelection) - // Nothing enabled → preview (safe default). - XCTAssertEqual(AICommand.inPlaceCommitPlan(resolvedWasSelection: true, outputs: []), .preview) - } - - func testRequiredCapabilitiesIsUnionOverInputs() { - let mixed = AICommand(name: "M", icon: .emoji("🖼"), inputs: [.selection, .clipboardImage], - promptTemplate: "{input}", outputs: [.previewOnly]) - XCTAssertEqual(mixed.requiredCapabilities, [.text, .vision], - "a command mixing text + image inputs unions both capabilities") - let standalone = AICommand(name: "S", icon: .emoji("✨"), inputs: [], - promptTemplate: "{date}", outputs: [.previewOnly]) - XCTAssertEqual(standalone.requiredCapabilities, [.text], "an empty input set needs only text") - } - - func testSideEffectAccessor() { - let task = AICommand(name: "Cal", icon: .emoji("📅"), inputs: [.selection], - promptTemplate: "{input}", outputs: [.runTask(.addToCalendar)]) - XCTAssertEqual(task.sideEffect, .runTask(.addToCalendar)) - XCTAssertTrue(task.isSideEffecting) - let inPlace = AICommand(name: "Fix", icon: .emoji("✍️"), inputs: [.selection], - promptTemplate: "{input}", outputs: [.replaceSelection, .pasteAtCursor]) - XCTAssertNil(inPlace.sideEffect) - XCTAssertFalse(inPlace.isSideEffecting) - } - - // MARK: - requiredCapabilities - - func testScreenRegionRequiresVision() { - let command = AICommand(name: "Describe", icon: .emoji("👁"), input: .screenRegion, - promptTemplate: "What's here?", output: .previewOnly) - XCTAssertEqual(command.requiredCapabilities, [.vision], - "a screenRegion command needs a vision-capable model") - } - - func testClipboardImageRequiresVision() { - let command = AICommand(name: "Describe Clipboard Image", icon: .emoji("🖼"), input: .clipboardImage, - promptTemplate: "What's in this image?", output: .previewOnly) - XCTAssertEqual(command.requiredCapabilities, [.vision], - "a clipboardImage command needs a vision-capable model, statically — like screenRegion") - } - - func testTextInputsRequireOnlyText() { - for input: InputSource in [.selection, .clipboard, .none] { - let command = AICommand(name: "C", icon: .emoji("✨"), input: input, - promptTemplate: "{input}", output: .previewOnly) - XCTAssertEqual(command.requiredCapabilities, [.text], - "input \(input) requires only the text capability") - } - } -} diff --git a/Tests/ThreeFingerSwitcherTests/AIErrorTests.swift b/Tests/ThreeFingerSwitcherTests/AIErrorTests.swift deleted file mode 100644 index c250c07..0000000 --- a/Tests/ThreeFingerSwitcherTests/AIErrorTests.swift +++ /dev/null @@ -1,126 +0,0 @@ -import XCTest -import Foundation -@testable import ThreeFingerSwitcherCore - -/// Tests for the AI error taxonomy + the single central translator (change: harden-ai-error-handling): -/// the classifier over synthetic `NSError`s (offline / dropped / 5xx / auth) and over the app's own -/// `RuntimeError` / `TaskError`, asserting (a) a clean per-case headline, (b) the raw text rides on -/// `details` and NEVER on the headline, and (c) `RuntimeError` is self-describing for every case. -final class AIErrorTests: XCTestCase { - - /// A headline must read as a human sentence — never a reflected `NSError`/enum dump. - private func assertHeadlineIsClean(_ presented: AIPresentedError, - file: StaticString = #filePath, line: UInt = #line) { - XCTAssertFalse(presented.headline.isEmpty, "headline is non-empty", file: file, line: line) - for needle in ["Domain=", "Code=", "Error Domain", "UserInfo"] { - XCTAssertFalse(presented.headline.contains(needle), - "headline must not contain raw error text (\(needle)): \(presented.headline)", - file: file, line: line) - } - } - - // MARK: - 7.1 vendor/OS classifier over synthetic NSErrors - - func testOfflineNotConnectedMapsToConnectivityHeadline() { - let ns = NSError(domain: NSURLErrorDomain, code: NSURLErrorNotConnectedToInternet) // -1009 - let presented = AIError.message(for: ns) - assertHeadlineIsClean(presented) - XCTAssertEqual(presented.headline, RuntimeError.offline.errorDescription) - XCTAssertNotNil(presented.details) - XCTAssertTrue(presented.details?.contains("-1009") ?? false, - "the raw NSError text is preserved as opt-in details") - } - - func testDroppedConnectionMapsToOffline() { - let ns = NSError(domain: NSURLErrorDomain, code: NSURLErrorNetworkConnectionLost) // -1005 - XCTAssertEqual(AIError.message(for: ns).headline, RuntimeError.offline.errorDescription) - } - - func testServerError5xxMapsToServerUnavailable() { - let ns = NSError(domain: "HTTPTransport", code: 503) - let presented = AIError.message(for: ns) - assertHeadlineIsClean(presented) - XCTAssertEqual(presented.headline, RuntimeError.serverUnavailable.errorDescription) - } - - func testHTTP401_403_404MapToAccessDenied() { - for code in [401, 403, 404] { - let ns = NSError(domain: "HTTPTransport", code: code) - XCTAssertEqual(AIError.message(for: ns).headline, - RuntimeError.authOrAccessDenied.errorDescription, - "HTTP \(code) → access denied") - } - } - - func testUnknownErrorFallsBackToGenericHeadlineWithRawDetails() { - struct Weird: Error { let secret = "raw-internal-1234" } - let presented = AIError.message(for: Weird()) - XCTAssertEqual(presented.headline, AIError.unknownHeadline, "unknown → safe generic headline") - XCTAssertTrue(presented.details?.contains("Weird") ?? false, - "the raw description is preserved as details, never in the headline") - assertHeadlineIsClean(presented) - } - - // MARK: - 7.1 app taxonomy → its own clean description - - func testRuntimeErrorUsesItsLocalizedDescriptionAsHeadline() { - let presented = AIError.message(for: RuntimeError.modelMissing) - XCTAssertEqual(presented.headline, "The model is not downloaded yet.") - } - - func testModelLoadFailedCarriesDetailNotHeadline() { - let presented = AIError.message(for: RuntimeError.modelLoadFailed(detail: "MLX abort 0xDEAD")) - XCTAssertEqual(presented.headline, "The model could not be loaded.") - XCTAssertEqual(presented.details, "MLX abort 0xDEAD", - "the diagnostic detail is carried as details, kept off the headline") - XCTAssertFalse(presented.headline.contains("0xDEAD"), "raw detail never appears in the headline") - } - - func testTaskErrorUsesItsLocalizedDescriptionAsHeadline() { - let presented = AIError.message(for: TaskError.calendarPermissionDenied) - XCTAssertEqual(presented.headline, TaskError.calendarPermissionDenied.errorDescription) - XCTAssertTrue(presented.headline.contains("Calendar")) - XCTAssertNotEqual(presented.headline, "calendarPermissionDenied", "not the raw enum case name") - } - - func testCancellationGetsBenignHeadline() { - let presented = AIError.message(for: CancellationError()) - XCTAssertEqual(presented.headline, RuntimeError.cancelled.errorDescription) - } - - // MARK: - 7.2 RuntimeError: LocalizedError — every case is self-describing - - func testEveryRuntimeErrorCaseHasNonEmptyDescription() { - let cases: [RuntimeError] = [ - .unavailable(reason: "no hw"), .modelMissing, .integrityFailed, .cancelled, - .couldNotProduceValid(attempts: 3), .decodeFailed(detail: "x"), .unsupportedModality(.vision), - .offline, .serverUnavailable, .authOrAccessDenied, .modelLoadFailed(detail: nil) - ] - for c in cases { - let description = c.errorDescription - XCTAssertNotNil(description, "\(c) has a localized description") - XCTAssertFalse(description?.isEmpty ?? true, "\(c) has a non-empty description") - } - } - - func testOfflineDescriptionIsAConnectivityHint() { - let description = (RuntimeError.offline.errorDescription ?? "").lowercased() - XCTAssertTrue(description.contains("internet") || description.contains("connection"), - "the offline message hints at connectivity, got: \(description)") - } - - func testDecodeFailedHeadlineDropsRawDetail() { - // The decode detail (which can carry a raw decoder error) must not leak into the headline. - let description = RuntimeError.decodeFailed(detail: "keyNotFound rawDump").errorDescription ?? "" - XCTAssertFalse(description.contains("rawDump"), "the raw decode detail is kept off the headline") - } - - // MARK: - HTTP-status classifier shared with the runtime boundary - - func testHTTPStatusClassifierMatchesTaxonomy() { - XCTAssertEqual(AIError.runtimeError(forHTTPStatus: 403), .authOrAccessDenied) - XCTAssertEqual(AIError.runtimeError(forHTTPStatus: 500), .serverUnavailable) - XCTAssertNil(AIError.runtimeError(forHTTPStatus: 200), "a 2xx is not a failure to classify") - XCTAssertNil(AIError.runtimeError(forHTTPStatus: 0)) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/AgentConversationTests.swift b/Tests/ThreeFingerSwitcherTests/AgentConversationTests.swift deleted file mode 100644 index 1bbf2c5..0000000 --- a/Tests/ThreeFingerSwitcherTests/AgentConversationTests.swift +++ /dev/null @@ -1,195 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for the canonical conversation types (`ai-conversation-runtime`, tasks §1.3 / §3.3) — the -/// Codable round-trip + equality that every other V2 slice relies on, and the model-agnostic -/// `ChatTemplate.flatten` assembler, including the load-bearing invariant that a turn's `thinking` is -/// NEVER emitted into the assembled prompt. -final class AgentConversationTests: XCTestCase { - - // MARK: - Canonical types: Codable round-trip + equality - - func testConversationCodableRoundTripPreservesAllFields() throws { - let route = ToolRoute(tool: "add_to_calendar", argumentsJSON: "{\"title\":\"x\"}", rationale: "looks like a meeting") - let result = ToolStepResult(tool: "add_to_calendar", status: .done, summary: "Added 'x'") - let assistant = AgentMessage(id: UUID(), role: .assistant, text: "the answer", - thinking: "the reasoning", image: Data([0x89, 0x50, 0x4E, 0x47]), - toolCalls: [route], toolResult: result, - createdAt: Date(timeIntervalSince1970: 1000)) - let user = AgentMessage(role: .user, text: "the question", createdAt: Date(timeIntervalSince1970: 999)) - let original = AgentConversation(id: AgentSessionID(), - title: "A thread", - messages: [user, assistant], - createdAt: Date(timeIntervalSince1970: 998), - updatedAt: Date(timeIntervalSince1970: 1001), - compactedSummary: "earlier we discussed lunch", - skillID: "create-meeting-in-gcal") - - let data = try JSONEncoder().encode(original) - let decoded = try JSONDecoder().decode(AgentConversation.self, from: data) - XCTAssertEqual(decoded, original, "every field (incl. compactedSummary/skillID/image/toolCalls) round-trips") - XCTAssertEqual(decoded.messages[1].thinking, "the reasoning", "thinking is persisted (for display)") - XCTAssertEqual(decoded.messages[1].toolResult?.summary, "Added 'x'") - } - - func testSessionIDIsStableHashableKey() { - let id = AgentSessionID() - var byID: [AgentSessionID: String] = [:] - byID[id] = "one" - XCTAssertEqual(byID[id], "one", "the session id is a stable dictionary key") - XCTAssertEqual(id, AgentSessionID(raw: id.raw), "equality is by the raw UUID") - XCTAssertNotEqual(id, AgentSessionID(), "two fresh ids differ") - } - - func testMessageEquality() { - let shared = UUID() - let a = AgentMessage(id: shared, role: .user, text: "hi", createdAt: Date(timeIntervalSince1970: 1)) - let b = AgentMessage(id: shared, role: .user, text: "hi", createdAt: Date(timeIntervalSince1970: 1)) - let c = AgentMessage(id: shared, role: .user, text: "different", createdAt: Date(timeIntervalSince1970: 1)) - XCTAssertEqual(a, b) - XCTAssertNotEqual(a, c) - } - - // MARK: - ChatTemplate.flatten - - func testFlattenPreservesRoleOrderAndExcludesThinking() { - let messages = [ - AgentMessage(role: .system, text: "SYS"), - AgentMessage(role: .user, text: "USER-ONE"), - AgentMessage(role: .assistant, text: "ASSISTANT-ONE", thinking: "SECRET-REASONING"), - AgentMessage(role: .user, text: "USER-TWO"), - ] - let prompt = ChatTemplate.flatten(messages) - - // Role order preserved. - let sys = prompt.range(of: "System: SYS") - let u1 = prompt.range(of: "User: USER-ONE") - let a1 = prompt.range(of: "Assistant: ASSISTANT-ONE") - let u2 = prompt.range(of: "User: USER-TWO") - XCTAssertNotNil(sys); XCTAssertNotNil(u1); XCTAssertNotNil(a1); XCTAssertNotNil(u2) - XCTAssertTrue(sys!.lowerBound < u1!.lowerBound && u1!.lowerBound < a1!.lowerBound - && a1!.lowerBound < u2!.lowerBound, "messages appear in order") - - // The load-bearing invariant: a turn's thinking is NEVER in the assembled prompt. - XCTAssertFalse(prompt.contains("SECRET-REASONING"), - "thinking is structurally excluded from the assembled prompt") - - // Trailing cue invites the next turn. - XCTAssertTrue(prompt.hasSuffix("Assistant:"), "a trailing Assistant: cue invites the next turn") - } - - func testFlattenRendersToolResultSummaryNotRawText() { - let result = ToolStepResult(tool: "memory.write", status: .done, summary: "Saved a note") - let messages = [ - AgentMessage(role: .user, text: "remember this"), - AgentMessage(role: .tool, text: "RAW-SHOULD-NOT-SHOW", toolResult: result), - ] - let prompt = ChatTemplate.flatten(messages) - XCTAssertTrue(prompt.contains("Tool: Saved a note"), "a .tool message renders its result summary") - XCTAssertFalse(prompt.contains("RAW-SHOULD-NOT-SHOW"), "the .tool message's raw text is not rendered") - } - - func testFlattenEmptyListIsCueOnly() { - XCTAssertEqual(ChatTemplate.flatten([]), "Assistant:", "an empty list yields just the cue") - } - - // MARK: - Multi-image contract (design D2) - - func testMessageCarriesMultipleImagesAndImageIsFirst() { - let a = Data([0x01]); let b = Data([0x02]) - let m = AgentMessage(role: .user, text: "two pics", images: [a, b]) - XCTAssertEqual(m.images, [a, b], "a single turn carries multiple images") - XCTAssertEqual(m.image, a, "the single-image convenience is the first image") - let none = AgentMessage(role: .user, text: "no pics") - XCTAssertEqual(none.images, []) - XCTAssertNil(none.image) - } - - func testSingleImageConvenienceInitFoldsIntoImagesArray() { - let png = Data([0xAB]) - let m = AgentMessage(role: .user, text: "one pic", image: png) - XCTAssertEqual(m.images, [png], "the image: convenience init folds one image into the array") - let nilImage = AgentMessage(role: .user, text: "none", image: nil) - XCTAssertEqual(nilImage.images, [], "a nil image yields an empty array") - } - - func testMessageCodableRoundTripsMultipleImagesAndDecodesLegacySingularKey() throws { - let a = Data([0x10]); let b = Data([0x20]) - let m = AgentMessage(id: UUID(), role: .user, text: "t", images: [a, b], - createdAt: Date(timeIntervalSince1970: 1)) - let data = try JSONEncoder().encode(m) - XCTAssertEqual(try JSONDecoder().decode(AgentMessage.self, from: data).images, [a, b], - "multiple images round-trip through Codable") - - // A LEGACY blob with a singular `image` key (the pre-D2 shape) decodes into a one-element array. - let legacy = """ - {"id":"\(UUID().uuidString)","role":"user","text":"t","image":"\(a.base64EncodedString())","createdAt":1} - """ - let decoded = try JSONDecoder().decode(AgentMessage.self, from: Data(legacy.utf8)) - XCTAssertEqual(decoded.images, [a], "a legacy singular image key decodes into the images array") - } - - func testChatRequestForwardsAllImagesViaEffectiveImages() { - let a = Data([0x01]); let b = Data([0x02]) - let msgs = [AgentMessage(role: .user, text: "q", images: [a, b])] - // Request-level images take precedence when set. - let explicit = LLMChatRequest(messages: msgs, images: [Data([0x09])]) - XCTAssertEqual(explicit.effectiveImages, [Data([0x09])]) - // Otherwise the latest image-bearing message's FULL array is forwarded. - let fromMessages = LLMChatRequest(messages: msgs) - XCTAssertEqual(fromMessages.effectiveImages, [a, b], "effectiveImages forwards ALL of the turn's images") - XCTAssertEqual(fromMessages.effectiveImage, a, "effectiveImage is the first effective image") - // The single-image convenience init still works (additive default). - let single = LLMChatRequest(messages: [], image: a) - XCTAssertEqual(single.images, [a]) - } - - // MARK: - The display timeline + born-with tuning (`notch-timeline-and-tuning`) - - func testSegmentsAndBornWithTuningRoundTripThroughCodable() throws { - let segments = [TurnSegment(kind: .thinking, text: "T1"), - TurnSegment(kind: .answer, text: "R1"), - TurnSegment(kind: .thinking, text: "T2")] - let message = AgentMessage(role: .assistant, text: "R1", thinking: "T1T2", segments: segments, - createdAt: Date(timeIntervalSince1970: 1)) - let convo = AgentConversation(title: "t", messages: [message], - createdAt: Date(timeIntervalSince1970: 0), - updatedAt: Date(timeIntervalSince1970: 2), - reasoningOverride: false, contextTokens: 32_768) - let decoded = try JSONDecoder().decode(AgentConversation.self, from: JSONEncoder().encode(convo)) - XCTAssertEqual(decoded, convo, "the segment timeline + born-with tuning round-trip") - XCTAssertEqual(decoded.messages[0].segments, segments, - "segments keep their cross-channel arrival order") - XCTAssertEqual(decoded.reasoningOverride, false) - XCTAssertEqual(decoded.contextTokens, 32_768) - } - - func testPreChangeJSONDecodesWithNilSegmentsAndTuning() throws { - // A row persisted BEFORE this change carries none of the new keys — everything decodes nil - // (the decode-safe optional contract; nothing throws, nothing is dropped). - let legacy = """ - {"id":"\(UUID().uuidString)","role":"assistant","text":"answer","thinking":"old","createdAt":1} - """ - let message = try JSONDecoder().decode(AgentMessage.self, from: Data(legacy.utf8)) - XCTAssertNil(message.segments, "a pre-change message has no stored timeline") - let convo = AgentConversation(title: "t", messages: [message]) - let decoded = try JSONDecoder().decode(AgentConversation.self, from: JSONEncoder().encode(convo)) - XCTAssertNil(decoded.reasoningOverride) - XCTAssertNil(decoded.contextTokens) - } - - func testDisplaySegmentsFallsBackForPreChangeMessages() { - // Legacy flat thinking → ONE leading thinking block, then the answer (the spec's fallback). - let legacy = AgentMessage(role: .assistant, text: "answer", thinking: "old reasoning") - XCTAssertEqual(legacy.displaySegments, - [TurnSegment(kind: .thinking, text: "old reasoning"), - TurnSegment(kind: .answer, text: "answer")]) - // No thinking at all → just the answer block. - let bare = AgentMessage(role: .assistant, text: "answer") - XCTAssertEqual(bare.displaySegments, [TurnSegment(kind: .answer, text: "answer")]) - // A stored timeline wins over the flat fields. - let timeline = [TurnSegment(kind: .answer, text: "a"), TurnSegment(kind: .thinking, text: "t")] - let modern = AgentMessage(role: .assistant, text: "a", thinking: "t", segments: timeline) - XCTAssertEqual(modern.displaySegments, timeline) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/AppDataResetTests.swift b/Tests/ThreeFingerSwitcherTests/AppDataResetTests.swift index 7ed3195..1adf6d9 100644 --- a/Tests/ThreeFingerSwitcherTests/AppDataResetTests.swift +++ b/Tests/ThreeFingerSwitcherTests/AppDataResetTests.swift @@ -2,8 +2,8 @@ import XCTest @testable import ThreeFingerSwitcherCore /// Unit tests for the Danger zone's reset service (Settings/AppDataReset.swift): the pure -/// filesystem-target computation per selection (including the App-data/AI-models split), the TCC -/// service list, and the perform step against a temp directory + command spy. +/// filesystem-target computation per selection, the TCC service list, and the perform step against +/// a temp directory + command spy. @MainActor final class AppDataResetTests: XCTestCase { private let library = URL(fileURLWithPath: "/Users/test/Library", isDirectory: true) @@ -23,28 +23,12 @@ final class AppDataResetTests: XCTestCase { XCTAssertTrue(t.removeContentsExcept.isEmpty) } - func testAppDataAloneKeepsTheModels() { + func testAppDataRemovesTheWholeRoot() { let t = targets(.appData) - XCTAssertEqual(t.removeContentsExcept.count, 1) - XCTAssertEqual(t.removeContentsExcept[0].directory.path, - "/Users/test/Library/Application Support/ThreeFingerSwitcher") - XCTAssertEqual(t.removeContentsExcept[0].keep, ["models"], - "the multi-GB weights survive a settings reset unless explicitly selected") - XCTAssertEqual(paths(t.removeWhole), - ["/Users/test/Library/Saved Application State/\(bid).savedState"]) - } - - func testAIModelsAloneRemovesOnlyTheWeights() { - let t = targets(.aiModels) - XCTAssertEqual(paths(t.removeWhole), - ["/Users/test/Library/Application Support/ThreeFingerSwitcher/models"]) XCTAssertTrue(t.removeContentsExcept.isEmpty) - } - - func testAppDataPlusModelsRemovesTheWholeRoot() { - let t = targets([.appData, .aiModels]) - XCTAssertTrue(paths(t.removeWhole).contains("/Users/test/Library/Application Support/ThreeFingerSwitcher")) - XCTAssertTrue(t.removeContentsExcept.isEmpty, "no survivors when both are selected") + XCTAssertEqual(paths(t.removeWhole), + ["/Users/test/Library/Application Support/ThreeFingerSwitcher", + "/Users/test/Library/Saved Application State/\(bid).savedState"]) } func testCaches() { @@ -62,8 +46,7 @@ final class AppDataResetTests: XCTestCase { func testTCCServiceListCoversEverythingTheAppCanHold() { XCTAssertEqual(Set(AppDataReset.tccServices), - ["Accessibility", "ScreenCapture", "ListenEvent", "AppleEvents", - "Calendar", "Reminders", "AddressBook"]) + ["Accessibility", "ScreenCapture", "ListenEvent", "AppleEvents"]) } // MARK: - Perform step (temp filesystem + command spy) diff --git a/Tests/ThreeFingerSwitcherTests/AppSettingsTests.swift b/Tests/ThreeFingerSwitcherTests/AppSettingsTests.swift index ce10a4e..158dc81 100644 --- a/Tests/ThreeFingerSwitcherTests/AppSettingsTests.swift +++ b/Tests/ThreeFingerSwitcherTests/AppSettingsTests.swift @@ -38,31 +38,6 @@ final class AppSettingsTests: XCTestCase { AppSettings(defaults: defaults) } - // MARK: - Per-command runtime language persistence (spec: ai-command-band) - - /// A remembered language round-trips per command, survives a reload from the same suite, and - /// orphan entries (commands no longer present) are pruned best-effort without touching live ones. - func testPerCommandLanguagePersistenceRoundTripsAndPrunes() { - let a = UUID(), b = UUID() - let settings = makeSettings() - XCTAssertNil(settings.rememberedLanguage(for: a), "no remembered language at cold start") - - settings.rememberLanguage("Hebrew", for: a) - settings.rememberLanguage("Spanish", for: b) - XCTAssertEqual(settings.rememberedLanguage(for: a), "Hebrew") - XCTAssertEqual(settings.rememberedLanguage(for: b), "Spanish", "two commands remember independently") - - // Reload from the SAME suite: the choices persisted across instances. - let reloaded = AppSettings(defaults: defaults) - XCTAssertEqual(reloaded.rememberedLanguage(for: a), "Hebrew", "survives a reload") - XCTAssertEqual(reloaded.rememberedLanguage(for: b), "Spanish") - - // Prune keeping only `a`: b's orphan entry is dropped, a is untouched. - reloaded.pruneCommandLanguages(keeping: [a]) - XCTAssertEqual(reloaded.rememberedLanguage(for: a), "Hebrew", "a live command's entry is kept") - XCTAssertNil(reloaded.rememberedLanguage(for: b), "an orphaned entry is pruned") - } - // MARK: - First-run defaults /// On first run (empty suite) every tunable must equal its `AppSettings.Defaults` value, @@ -384,128 +359,6 @@ final class AppSettingsTests: XCTestCase { XCTAssertFalse(settings.includeNonStandardWindows, "reset restores the strict default (OFF)") } - // MARK: - AI commands opt-in - - /// The AI-commands opt-in is off on first run (it gates a multi-gigabyte model download), and the - /// selected-model pin starts nil ("registry default"). - func testAICommandsDefaultsOffAndNoSelectedModel() { - let settings = makeSettings() - XCTAssertFalse(settings.aiCommandsEnabled, "AI commands must default OFF (opt-in)") - XCTAssertEqual(settings.aiCommandsEnabled, AppSettings.Defaults.aiCommandsEnabled) - XCTAssertNil(settings.aiSelectedModelID, "no model pinned by default") - XCTAssertNil(AppSettings.Defaults.aiSelectedModelID) - } - - /// The opt-in persists across a "relaunch" (a fresh instance on the same suite) and writes through - /// to the documented raw key, both directions. - func testAICommandsEnabledPersistsAcrossInstances() { - let writer = makeSettings() - XCTAssertFalse(writer.aiCommandsEnabled) - - writer.aiCommandsEnabled = true - XCTAssertEqual(defaults.object(forKey: "aiCommandsEnabled") as? Bool, true, "writes the documented key") - - let reader = AppSettings(defaults: defaults) - XCTAssertTrue(reader.aiCommandsEnabled, "persists across instances") - - reader.aiCommandsEnabled = false - XCTAssertFalse(AppSettings(defaults: defaults).aiCommandsEnabled, "the off state persists too") - } - - /// The selected-model pin persists across instances and writes through to its key. - func testAISelectedModelIDPersistsAcrossInstances() { - let writer = makeSettings() - writer.aiSelectedModelID = "gemma-4-26b-a4b" - XCTAssertEqual(defaults.object(forKey: "aiSelectedModelID") as? String, "gemma-4-26b-a4b") - - let reader = AppSettings(defaults: defaults) - XCTAssertEqual(reader.aiSelectedModelID, "gemma-4-26b-a4b", "persists across instances") - } - - /// Older settings (no AI keys present) decode with the opt-in OFF and no pinned model, while every - /// pre-existing setting is left untouched — proving the addition is purely additive. - func testOlderSettingsDecodeWithAICommandsOffAndUntouched() { - // Arrange: simulate a pre-feature store — populate unrelated keys, but NO AI keys. - defaults.set(0.0777, forKey: "stepDistance") - defaults.set(true, forKey: "wrapAtEnds") - defaults.set(true, forKey: "keepClipboardHistory") - XCTAssertNil(defaults.object(forKey: "aiCommandsEnabled"), "precondition: no AI key on disk") - XCTAssertNil(defaults.object(forKey: "aiSelectedModelID"), "precondition: no AI model key on disk") - - // Act - let settings = AppSettings(defaults: defaults) - - // Assert: the new opt-in defaults off / nil without a stored value... - XCTAssertFalse(settings.aiCommandsEnabled) - XCTAssertNil(settings.aiSelectedModelID) - // ...and the pre-existing settings are loaded exactly as stored (not reset). - XCTAssertEqual(settings.stepDistance, 0.0777, accuracy: eps) - XCTAssertTrue(settings.wrapAtEnds) - XCTAssertTrue(settings.keepClipboardHistory) - } - - /// `resetToDefaults()` must NOT touch the AI opt-in (it's a consent-gated choice that allows a - /// multi-gigabyte download) nor the pinned model — mirrors the launcher / clipboard opt-in handling. - func testResetToDefaultsDoesNotTouchAICommands() { - let settings = makeSettings() - settings.aiCommandsEnabled = true - settings.aiSelectedModelID = "gemma-4-12b" - - settings.resetToDefaults() - - XCTAssertTrue(settings.aiCommandsEnabled, "reset must not flip the AI opt-in") - XCTAssertEqual(settings.aiSelectedModelID, "gemma-4-12b", "reset must not clear the pinned model") - } - - // MARK: - AI reasoning opt-in ("let the model think") - - /// `aiReasoningEnabled` defaults ON (the model thinks; the thinking is shown but filtered out of the - /// committed result) and matches the `Defaults` enum value. - func testAIReasoningDefaultsOn() { - let settings = makeSettings() - XCTAssertTrue(settings.aiReasoningEnabled, "reasoning must default ON") - XCTAssertEqual(settings.aiReasoningEnabled, AppSettings.Defaults.aiReasoningEnabled) - XCTAssertTrue(AppSettings.Defaults.aiReasoningEnabled, "the Defaults literal is true") - } - - /// The reasoning opt-in round-trips and persists across instances (both directions) and writes - /// through to its documented raw key. - func testAIReasoningEnabledPersistsAcrossInstances() { - let writer = makeSettings() - XCTAssertTrue(writer.aiReasoningEnabled) - - writer.aiReasoningEnabled = false - XCTAssertEqual(defaults.object(forKey: "aiReasoningEnabled") as? Bool, false, "writes the documented key") - - let reader = AppSettings(defaults: defaults) - XCTAssertFalse(reader.aiReasoningEnabled, "the off state persists across instances") - - reader.aiReasoningEnabled = true - XCTAssertTrue(AppSettings(defaults: defaults).aiReasoningEnabled, "the on state persists too") - } - - /// Older settings (no reasoning key present) decode with reasoning ON (the additive default), while - /// pre-existing settings are left untouched. - func testOlderSettingsDecodeWithReasoningOnAndUntouched() { - defaults.set(0.0777, forKey: "stepDistance") - XCTAssertNil(defaults.object(forKey: "aiReasoningEnabled"), "precondition: no reasoning key on disk") - - let settings = AppSettings(defaults: defaults) - XCTAssertTrue(settings.aiReasoningEnabled, "absent key falls back to the ON default") - XCTAssertEqual(settings.stepDistance, 0.0777, accuracy: eps, "pre-existing settings untouched") - } - - /// `resetToDefaults()` must NOT touch the reasoning opt-in — like the AI opt-in, it's a user choice - /// the tunables reset leaves alone (a disabled reasoning stays disabled across a reset). - func testResetToDefaultsDoesNotTouchAIReasoning() { - let settings = makeSettings() - settings.aiReasoningEnabled = false - - settings.resetToDefaults() - - XCTAssertFalse(settings.aiReasoningEnabled, "reset must not flip the reasoning opt-in") - } - // MARK: - Minimized windows in the switcher + minimize-all-on-down opt-ins /// Both opt-ins default OFF (preserve today's behavior: minimized windows excluded, down-swipe = App @@ -602,26 +455,6 @@ final class AppSettingsTests: XCTestCase { XCTAssertEqual(settings.stepDistance, 0.0777, accuracy: eps, "pre-existing settings untouched") } - // MARK: - Notch tuning (`notch-timeline-and-tuning`) - - /// The notch chat's thinking+context dial: defaults Balanced, round-trips its raw value, and resets - /// with the other behavior tunables. - func testNotchTuningDefaultsBalancedRoundTripsAndResets() { - let settings = makeSettings() - XCTAssertEqual(settings.notchTuning, .balanced, "default: thinking on · base context") - - settings.notchTuning = .max - let reloaded = AppSettings(defaults: defaults) - XCTAssertEqual(reloaded.notchTuning, .max, "the selected stop persists across relaunch") - - settings.resetToDefaults() - XCTAssertEqual(settings.notchTuning, .balanced, "reset returns the dial to Balanced") - - // A corrupt/unknown stored value falls back to the default rather than crashing. - defaults.set("garbage", forKey: "notchTuning") - XCTAssertEqual(AppSettings(defaults: defaults).notchTuning, .balanced) - } - // MARK: - Isolation /// Two instances on different suites must not share state, proving suite isolation. diff --git a/Tests/ThreeFingerSwitcherTests/BackgroundAutonomyTests.swift b/Tests/ThreeFingerSwitcherTests/BackgroundAutonomyTests.swift deleted file mode 100644 index 224c53e..0000000 --- a/Tests/ThreeFingerSwitcherTests/BackgroundAutonomyTests.swift +++ /dev/null @@ -1,376 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for the `ai-background-autonomy` slice (tasks §1–§5): blast-radius classification (§1), the -/// whitelist matching rules (§2), effective-tier resolution (§3), the auto-vs-escalate gate (§4), and the -/// append-only redacted audit log (§5). Everything is MLX-free Core, driven with fabricated -/// `ToolDescriptor`s — no model. `@MainActor` because `AppSettings` keys exercised here are main-actor. -@MainActor -final class BackgroundAutonomyTests: XCTestCase { - - // MARK: - Helpers - - private func descriptor(_ name: String, tier: WritePolicyTier) -> ToolDescriptor { - ToolDescriptor(name: name, summary: "tool \(name)", - argsSchema: StructuredSchema(name: name, json: "{\"type\":\"object\"}"), - writePolicy: tier) - } - - private func sid() -> AgentSessionID { AgentSessionID() } - - // MARK: - 1. Blast-radius classification - - func testContainedToolsClassifyContained() { - for name in ["memory.read", "memory.write", "save_to_project:Inbox", "retrieve", "widen_candidates"] { - XCTAssertEqual(BlastRadius.of(descriptor(name, tier: .auto)), .contained, "\(name)") - } - } - - func testExternalConfirmToolsClassifyExternal() { - for name in ["add_to_calendar", "add_to_reminders", "new_contact", "open_tool_with_payload:notes", "send_to:slack"] { - XCTAssertEqual(BlastRadius.of(descriptor(name, tier: .confirm)), .external, "\(name)") - } - } - - func testDangerousIsUnconditionalAndIgnoresName() { - // A .dangerous descriptor classifies dangerous even with a CONTAINED-looking name. - XCTAssertEqual(BlastRadius.of(descriptor("memory.write", tier: .dangerous)), .dangerous) - XCTAssertEqual(BlastRadius.of(descriptor("delete_file", tier: .dangerous)), .dangerous) - XCTAssertEqual(BlastRadius.of(descriptor("send_to:shell", tier: .dangerous)), .dangerous) - } - - func testContainedPrefixSetMatchesByPrefix() { - XCTAssertTrue(BlastRadius.isContainedName("memory.write.core")) - XCTAssertTrue(BlastRadius.isContainedName("save_to_project:Big Project")) - XCTAssertFalse(BlastRadius.isContainedName("write_memory")) // not a prefix - XCTAssertFalse(BlastRadius.isContainedName("open_tool")) - } - - // MARK: - 2. Whitelist — path matching - - func testPathMatchRespectsComponentBoundary() { - let wl = Whitelist(trustedPathPrefixes: ["/Users/me/Notes"]) - XCTAssertTrue(wl.matchesPath("/Users/me/Notes/x.md")) - XCTAssertTrue(wl.matchesPath("/Users/me/Notes")) // the prefix itself - XCTAssertFalse(wl.matchesPath("/Users/me/Notes2/x.md")) // string-prefix-only, NOT a boundary - XCTAssertFalse(wl.matchesPath("/Users/me/Other/x.md")) - } - - func testPathMatchResolvesDotDotEscape() { - let wl = Whitelist(trustedPathPrefixes: ["/Users/me/Notes"]) - // `/Users/me/Notes/../etc/passwd` standardizes to `/Users/me/etc/passwd` → does NOT match. - XCTAssertFalse(wl.matchesPath("/Users/me/Notes/../etc/passwd")) - } - - func testEmptyWhitelistMatchesNothing() { - XCTAssertFalse(Whitelist.empty.matchesPath("/Users/me/Notes/x.md")) - XCTAssertFalse(Whitelist.empty.matchesCommand("git")) - XCTAssertEqual(Whitelist.empty.trustedPathPrefixes, []) - XCTAssertEqual(Whitelist.empty.trustedCommandPatterns, []) - } - - // MARK: - 2. Whitelist — command glob - - func testCommandGlobIsAnchoredFullString() { - let wl = Whitelist(trustedCommandPatterns: ["git*"]) - XCTAssertTrue(wl.matchesCommand("git")) - XCTAssertTrue(wl.matchesCommand("git-lfs")) - XCTAssertFalse(wl.matchesCommand("forgit")) // anchored: pattern must match from the start - } - - func testCommandGlobQuestionMark() { - let wl = Whitelist(trustedCommandPatterns: ["ls?"]) - XCTAssertTrue(wl.matchesCommand("lsx")) - XCTAssertFalse(wl.matchesCommand("ls")) // `?` requires exactly one char - XCTAssertFalse(wl.matchesCommand("lsxx")) - } - - func testGlobMatchPureHelper() { - XCTAssertTrue(Whitelist.globMatch(pattern: "*", name: "anything")) - XCTAssertTrue(Whitelist.globMatch(pattern: "a*b*c", name: "axxbyyc")) - XCTAssertFalse(Whitelist.globMatch(pattern: "a*b*c", name: "axxbyy")) - XCTAssertTrue(Whitelist.globMatch(pattern: "exact", name: "exact")) - } - - // MARK: - 2. Whitelist — both-rule - - func testBothRuleRequiresCommandAndPath() { - let wl = Whitelist(trustedPathPrefixes: ["/Users/me/Repo"], trustedCommandPatterns: ["git*"]) - // Whitelisted command at a trusted path → matches. - XCTAssertTrue(wl.matchesBoth(command: "git", path: "/Users/me/Repo/file.txt")) - // Whitelisted command at an OFF-list path → does NOT match (stricter wins). - XCTAssertFalse(wl.matchesBoth(command: "git", path: "/tmp/file.txt")) - // Off-list command at a trusted path → does NOT match. - XCTAssertFalse(wl.matchesBoth(command: "rm", path: "/Users/me/Repo/file.txt")) - } - - func testWhitelistCodableRoundTrip() throws { - let wl = Whitelist(trustedPathPrefixes: ["/a/b"], trustedCommandPatterns: ["git*", "node"]) - let data = try JSONEncoder().encode(wl) - let back = try JSONDecoder().decode(Whitelist.self, from: data) - XCTAssertEqual(wl, back) - } - - // MARK: - 3. Effective-tier resolution (the full table) - - func testResolutionContainedIsAuto() { - let r = BackgroundPolicyResolver(whitelist: .empty) - XCTAssertEqual(r.effectiveTier(for: descriptor("memory.write", tier: .auto)), .auto) - XCTAssertEqual(r.effectiveTier(for: descriptor("memory.write", tier: .auto), target: PolicyTarget.none), .auto) - } - - func testResolutionDangerousNeverLowered() { - // Even a whitelisted path cannot lower a .dangerous descriptor. - let wl = Whitelist(trustedPathPrefixes: ["/Users/me/Notes"]) - let r = BackgroundPolicyResolver(whitelist: wl) - let d = descriptor("delete_file", tier: .dangerous) - XCTAssertEqual(r.effectiveTier(for: d), .dangerous) - XCTAssertEqual(r.effectiveTier(for: d, target: .path("/Users/me/Notes/x.md")), .dangerous) - } - - func testResolutionConfirmLowersOnMatch() { - let wl = Whitelist(trustedPathPrefixes: ["/Users/me/Notes"], trustedCommandPatterns: ["git*"]) - let r = BackgroundPolicyResolver(whitelist: wl) - let d = descriptor("send_to:script", tier: .confirm) - // path match → auto - XCTAssertEqual(r.effectiveTier(for: d, target: .path("/Users/me/Notes/out.md")), .auto) - // command match → auto - XCTAssertEqual(r.effectiveTier(for: d, target: .command("git")), .auto) - // both match → auto - XCTAssertEqual(r.effectiveTier(for: d, target: .both(command: "git", path: "/Users/me/Notes/out.md")), .auto) - } - - func testResolutionConfirmStaysOnNoMatch() { - let r = BackgroundPolicyResolver(whitelist: .empty) - let d = descriptor("send_to:script", tier: .confirm) - XCTAssertEqual(r.effectiveTier(for: d, target: .path("/tmp/out.md")), .confirm) - XCTAssertEqual(r.effectiveTier(for: d, target: .command("rm")), .confirm) - XCTAssertEqual(r.effectiveTier(for: d, target: PolicyTarget.none), .confirm) // .none never lowerable - XCTAssertEqual(r.effectiveTier(for: d, target: nil), .confirm) - XCTAssertEqual(r.effectiveTier(for: d), .confirm) // descriptor-only fast path - } - - func testResolverSatisfiesProtocol() { - // The descriptor-only protocol method (the routing seam) is satisfied. - let r: WritePolicyResolving = BackgroundPolicyResolver(whitelist: .empty) - XCTAssertEqual(r.effectiveTier(for: descriptor("add_to_calendar", tier: .confirm)), .confirm) - } - - // MARK: - 4. Background gate (the full decision table) - - func testGateAutoRunsWhileParkedAndActive() { - XCTAssertEqual(BackgroundGate.decide(effectiveTier: .auto, parkState: .parked), .auto) - XCTAssertEqual(BackgroundGate.decide(effectiveTier: .auto, parkState: .idle), .auto) - XCTAssertEqual(BackgroundGate.decide(effectiveTier: .auto, parkState: .active), .auto) - } - - func testGateConfirmWaitsParkedDefersForeground() { - XCTAssertEqual(BackgroundGate.decide(effectiveTier: .confirm, parkState: .parked), .waitParked) - XCTAssertEqual(BackgroundGate.decide(effectiveTier: .confirm, parkState: .idle), .waitParked) - XCTAssertEqual(BackgroundGate.decide(effectiveTier: .confirm, parkState: .active), .foreground) - } - - func testGateDangerousEscalatesWhileParked() { - let decision = BackgroundGate.decide(effectiveTier: .dangerous, parkState: .parked, tool: "delete_file") - if case let .escalate(reason) = decision { - XCTAssertTrue(reason.contains("delete_file")) - XCTAssertTrue(reason.contains("approval")) - } else { - XCTFail("expected .escalate, got \(decision)") - } - // Active → the canvas approval gate owns it (no glow). - XCTAssertEqual(BackgroundGate.decide(effectiveTier: .dangerous, parkState: .active), .foreground) - } - - func testGateNeverDoubleEscalates() { - // Already in needs-you → any tier waits (do not double-escalate / re-glow). - XCTAssertEqual(BackgroundGate.decide(effectiveTier: .dangerous, parkState: .needsYou), .waitParked) - XCTAssertEqual(BackgroundGate.decide(effectiveTier: .confirm, parkState: .needsYou), .waitParked) - XCTAssertEqual(BackgroundGate.decide(effectiveTier: .auto, parkState: .needsYou), .waitParked) - } - - // MARK: - 5. Audit record + redaction - - func testRedactionTruncatesLongArguments() { - let long = String(repeating: "x", count: 300) - let summary = AuditRedaction.summary(forRawArguments: long) - XCTAssertLessThanOrEqual(summary.count, AuditRedaction.maxSummaryLength) - XCTAssertTrue(summary.contains("\u{2026}")) // middle-truncation ellipsis - } - - func testRedactionStripsEmbeddedSecret() { - let line = "deploy --token abcdef0123456789 --target prod" - let summary = AuditRedaction.summary(forRawArguments: line) - XCTAssertFalse(summary.contains("abcdef0123456789"), "raw token leaked: \(summary)") - } - - func testRedactionPathShowsLastComponentsOnly() { - let summary = AuditRedaction.summary(for: .path("/Users/me/Secret/Tree/note.md")) - XCTAssertTrue(summary.contains("note.md")) - XCTAssertFalse(summary.contains("/Users/me/Secret")) // full tree not leaked - } - - func testRedactionCommandShowsNameNotFullLine() { - let summary = AuditRedaction.summary(for: .command("/usr/bin/git push --token SECRETTOKEN1234")) - XCTAssertTrue(summary.contains("git")) - XCTAssertFalse(summary.contains("SECRETTOKEN1234")) - } - - func testAuditRecordCodableRoundTripFailedHeadlineOnly() throws { - let rec = AuditRecord(sessionID: sid(), tool: "send_to:slack", policy: .confirm, - argumentsSummary: "$ slack → #general", - outcome: .failed(headline: "Couldn't reach Slack."), - wasBackground: true, - timestamp: Date(timeIntervalSince1970: 1000)) - let data = try JSONEncoder().encode(rec) - let back = try JSONDecoder().decode(AuditRecord.self, from: data) - XCTAssertEqual(rec, back) - if case let .failed(headline) = back.outcome { - XCTAssertEqual(headline, "Couldn't reach Slack.") - } else { - XCTFail("expected .failed outcome") - } - } - - // MARK: - 5. In-memory audit ring - - func testInMemoryRingAppendsAndReadsReverseChronological() { - let log = InMemoryAuditLog(cap: 100) - let s = sid() - for i in 0..<3 { - log.record(AuditRecord(sessionID: s, tool: "t\(i)", policy: .auto, - argumentsSummary: "a\(i)", outcome: .done, wasBackground: true, - timestamp: Date(timeIntervalSince1970: Double(i)))) - } - let recent = log.recent(limit: 10) - XCTAssertEqual(recent.map(\.tool), ["t2", "t1", "t0"]) // reverse-chronological - } - - func testInMemoryRingCapTrimsOldest() { - let log = InMemoryAuditLog(cap: 2) - let s = sid() - for i in 0..<5 { - log.record(AuditRecord(sessionID: s, tool: "t\(i)", policy: .auto, - argumentsSummary: "a", outcome: .done, wasBackground: false)) - } - let recent = log.recent(limit: 10) - XCTAssertEqual(recent.count, 2) - XCTAssertEqual(recent.map(\.tool), ["t4", "t3"]) // oldest trimmed - } - - func testRecordIsAppendOnlyAcrossOutcomes() { - // Every outcome variety writes one record (the "what did my agents do" ledger). - let log = InMemoryAuditLog(cap: 100) - let s = sid() - let outcomes: [ToolStepStatus] = [.done, .declined(reason: "skip"), .awaitingApproval, - .failed(headline: "nope")] - for (i, o) in outcomes.enumerated() { - log.record(AuditRecord(sessionID: s, tool: "t\(i)", policy: .auto, - argumentsSummary: "a", outcome: o, wasBackground: true)) - } - XCTAssertEqual(log.recent(limit: 10).count, outcomes.count) - } - - // MARK: - 5. Disk audit log (temp dir) - - func testDiskAuditLogRoundTripAndReload() { - let dir = FileManager.default.temporaryDirectory - .appendingPathComponent("tfs-audit-\(UUID().uuidString)", isDirectory: true) - let file = dir.appendingPathComponent("audit.jsonl") - let s = sid() - - let log = DiskAuditLog(fileURL: file, cap: 100) - for i in 0..<3 { - log.record(AuditRecord(sessionID: s, tool: "t\(i)", policy: .auto, - argumentsSummary: "a\(i)", outcome: .done, wasBackground: true, - timestamp: Date(timeIntervalSince1970: Double(i)))) - } - // The off-main writer is async — drain it before reloading. - let exp = expectation(description: "persisted") - DispatchQueue.global().asyncAfter(deadline: .now() + 0.4) { exp.fulfill() } - wait(for: [exp], timeout: 2.0) - - XCTAssertNil(log.lastPersistError) - - // A fresh store over the same file rebuilds the ring from disk. - let reloaded = DiskAuditLog(fileURL: file, cap: 100) - XCTAssertEqual(reloaded.recent(limit: 10).map(\.tool), ["t2", "t1", "t0"]) - - try? FileManager.default.removeItem(at: dir) - } - - func testDiskAuditLogTrimsToCapOnWrite() { - let dir = FileManager.default.temporaryDirectory - .appendingPathComponent("tfs-audit-\(UUID().uuidString)", isDirectory: true) - let file = dir.appendingPathComponent("audit.jsonl") - let s = sid() - - let log = DiskAuditLog(fileURL: file, cap: 2) - for i in 0..<5 { - log.record(AuditRecord(sessionID: s, tool: "t\(i)", policy: .auto, - argumentsSummary: "a", outcome: .done, wasBackground: false)) - } - let exp = expectation(description: "persisted") - DispatchQueue.global().asyncAfter(deadline: .now() + 0.4) { exp.fulfill() } - wait(for: [exp], timeout: 2.0) - - let reloaded = DiskAuditLog(fileURL: file, cap: 2) - XCTAssertEqual(reloaded.recent(limit: 10).count, 2) - XCTAssertEqual(reloaded.recent(limit: 10).map(\.tool), ["t4", "t3"]) - - try? FileManager.default.removeItem(at: dir) - } - - func testFailablePersistIsObservableNotThrown() { - // A persistence failure surfaces on `lastPersistError` but never throws into the caller and never - // loses the in-memory record (auditing must not break the agent). - let log = FailableInMemoryAuditLog(cap: 100) - log.failPersist = true - let s = sid() - log.record(AuditRecord(sessionID: s, tool: "t", policy: .auto, argumentsSummary: "a", - outcome: .done, wasBackground: true)) // does not throw - XCTAssertEqual(log.recent(limit: 10).count, 1) // record still present - XCTAssertNotNil(log.lastPersistError) - } - - // MARK: - 5/7. AuditError routes through the single translator - - func testAuditErrorRoutesThroughAIError() { - for err: AuditError in [.persistFailed(detail: "disk full at /x/y raw OS text"), - .storeUnavailable(detail: "EACCES raw")] { - let presented = AIError.message(for: err) - XCTAssertFalse(presented.headline.isEmpty) - XCTAssertNotEqual(presented.headline, AIError.unknownHeadline) - XCTAssertFalse(presented.headline.contains("raw"), "raw OS text leaked into headline") - XCTAssertFalse(presented.headline.contains("EACCES")) - } - } - - // MARK: - AppSettings whitelist persistence + reset - - func testWhitelistDefaultsEmptyPersistAndResetPreserves() { - let suite = "tfs-bgauto-test-\(UUID().uuidString)" - let defaults = UserDefaults(suiteName: suite)! - defer { defaults.removePersistentDomain(forName: suite) } - - let settings = AppSettings(defaults: defaults) - XCTAssertEqual(settings.agentWhitelistPaths, []) - XCTAssertEqual(settings.agentWhitelistCommands, []) - XCTAssertEqual(settings.agentWhitelist, .empty) - - settings.agentWhitelistPaths = ["/Users/me/Notes"] - settings.agentWhitelistCommands = ["git*"] - - // Persists across a fresh instance over the same defaults. - let reloaded = AppSettings(defaults: defaults) - XCTAssertEqual(reloaded.agentWhitelistPaths, ["/Users/me/Notes"]) - XCTAssertEqual(reloaded.agentWhitelistCommands, ["git*"]) - XCTAssertEqual(reloaded.agentWhitelist, - Whitelist(trustedPathPrefixes: ["/Users/me/Notes"], trustedCommandPatterns: ["git*"])) - - // A reset-to-defaults preserves the trust choice (like the other AI opt-ins). - reloaded.resetToDefaults() - XCTAssertEqual(reloaded.agentWhitelistPaths, ["/Users/me/Notes"]) - XCTAssertEqual(reloaded.agentWhitelistCommands, ["git*"]) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/BackgroundToolRunnerTests.swift b/Tests/ThreeFingerSwitcherTests/BackgroundToolRunnerTests.swift deleted file mode 100644 index b2be2a6..0000000 --- a/Tests/ThreeFingerSwitcherTests/BackgroundToolRunnerTests.swift +++ /dev/null @@ -1,169 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for the background-autonomy run path (`ai-background-autonomy` 6.1–6.4): the `BackgroundToolRunner` -/// resolves the whitelist-aware effective tier, decides auto/foreground/waitParked/escalate by park state -/// (`BackgroundGate`), runs/escalates/waits, and records an `AuditRecord` for EVERY step — plus the -/// `AgentLoop` routing each tool step through it when injected. -final class BackgroundToolRunnerTests: XCTestCase { - - // MARK: - Fakes - - private final class FakeContributor: ToolContributor, @unchecked Sendable { - let list: [ToolDescriptor] - let result: ToolStepResult - private(set) var runCalled = false - init(_ list: [ToolDescriptor], result: ToolStepResult) { self.list = list; self.result = result } - func descriptors() -> [ToolDescriptor] { list } - func canHandle(_ tool: String) -> Bool { list.contains { $0.name == tool } } - func run(_ call: RoutedCall, gate: ApprovalGate) async -> ToolStepResult { runCalled = true; return result } - } - /// Records escalations routed through the runner's `onEscalate` callback (the controller seam — - /// `refactor-park-and-background-agents` replaced the raw scheduler reference). - private final class EscalationSpy: @unchecked Sendable { - private let lock = NSLock() - private var recorded: [(AgentSessionID, String)] = [] - var escalated: [(AgentSessionID, String)] { lock.lock(); defer { lock.unlock() }; return recorded } - func record(_ id: AgentSessionID, _ reason: String) { - lock.lock(); recorded.append((id, reason)); lock.unlock() - } - } - - private func desc(_ name: String, _ policy: WritePolicyTier) -> ToolDescriptor { - ToolDescriptor(name: name, summary: "s", - argsSchema: StructuredSchema(name: name, json: "{\"type\":\"object\"}"), writePolicy: policy) - } - private func call(_ d: ToolDescriptor) -> RoutedCall { - RoutedCall(descriptor: d, route: ToolRoute(tool: d.name, argumentsJSON: "{}"), userText: "u", source: TaskSource()) - } - private func runner(parkState: ParkState, audit: AuditLog, onEscalate: EscalationSpy? = nil) -> BackgroundToolRunner { - BackgroundToolRunner(resolver: BackgroundPolicyResolver(), audit: audit, - onEscalate: onEscalate.map { spy in { spy.record($0, $1) } }, - parkStateOf: { _ in parkState }) - } - - // MARK: - Decision table + audit - - func testContainedAutoRunsInBackgroundAndAudits() async { - let audit = InMemoryAuditLog() - let d = desc("memory.write", .auto) // contained name → effective .auto - let c = FakeContributor([d], result: ToolStepResult(tool: d.name, status: .done, summary: "saved")) - let r = await runner(parkState: .parked, audit: audit) - .run(call(d), sessionID: AgentSessionID(), registry: ToolRegistry([c]), gate: AutoApproveGate()) - XCTAssertEqual(r.status, .done) - XCTAssertTrue(c.runCalled, "an auto step runs even while parked") - let rec = audit.recent(limit: 5) - XCTAssertEqual(rec.count, 1) - XCTAssertEqual(rec.first?.policy, .auto) - XCTAssertEqual(rec.first?.wasBackground, true, "applied while parked → background") - } - - func testParkedConfirmWaitsWithoutRunning() async { - let audit = InMemoryAuditLog() - let d = desc("send_to", .confirm) // external + confirm - let c = FakeContributor([d], result: ToolStepResult(tool: d.name, status: .done, summary: "sent")) - let r = await runner(parkState: .parked, audit: audit) - .run(call(d), sessionID: AgentSessionID(), registry: ToolRegistry([c]), gate: AutoApproveGate()) - XCTAssertEqual(r.status, .awaitingApproval) - XCTAssertFalse(c.runCalled, "a parked .confirm step waits — it does NOT fire the side effect") - XCTAssertEqual(audit.recent(limit: 5).first?.policy, .confirm) - } - - func testParkedDangerousEscalatesAndAudits() async { - let audit = InMemoryAuditLog() - let spy = EscalationSpy() - let d = desc("launch_claude", .dangerous) - let c = FakeContributor([d], result: ToolStepResult(tool: d.name, status: .done, summary: "x")) - let id = AgentSessionID() - let r = await runner(parkState: .parked, audit: audit, onEscalate: spy) - .run(call(d), sessionID: id, registry: ToolRegistry([c]), gate: AutoApproveGate()) - XCTAssertEqual(r.status, .awaitingApproval) - XCTAssertFalse(c.runCalled, "a parked .dangerous step never fires without foreground approval") - XCTAssertEqual(spy.escalated.count, 1, "it escalates through the controller-routed callback") - XCTAssertEqual(spy.escalated.first?.0, id) - XCTAssertEqual(audit.recent(limit: 5).first?.policy, .dangerous) - } - - func testForegroundConfirmRunsThroughTheGateAndAuditsForeground() async { - let audit = InMemoryAuditLog() - let d = desc("send_to", .confirm) - let c = FakeContributor([d], result: ToolStepResult(tool: d.name, status: .done, summary: "sent")) - let r = await runner(parkState: .active, audit: audit) - .run(call(d), sessionID: AgentSessionID(), registry: ToolRegistry([c]), gate: AutoApproveGate()) - XCTAssertEqual(r.status, .done) - XCTAssertTrue(c.runCalled, "a foreground confirm runs through the canvas approval gate") - XCTAssertEqual(audit.recent(limit: 5).first?.wasBackground, false) - } - - // MARK: - AgentLoop routes each step through the runner - - func testAgentLoopRoutesToolStepsThroughTheBackgroundRunner() async { - let audit = InMemoryAuditLog() - let d = desc("memory.write", .auto) - let contributor = FakeContributor([d], result: ToolStepResult(tool: d.name, status: .done, summary: "ran")) - let bg = BackgroundToolRunner(resolver: BackgroundPolicyResolver(), audit: audit, - parkStateOf: { _ in .parked }) - let rt = RoutingRuntime(routes: ["{\"tool\":\"memory.write\"}", "{\"tool\":\"\"}"], answer: ["done"]) - let loop = AgentLoop(runtime: rt, registry: ToolRegistry([contributor]), - candidateSource: KeywordToolCandidateSource(all: [d]), - gate: AutoApproveGate(), backgroundRunner: bg) - let result = await loop.run(context: RouteContext(messages: [AgentMessage(role: .user, text: "save it")])) - XCTAssertEqual(result.outcome, .answered(text: "done")) - XCTAssertEqual(audit.recent(limit: 5).count, 1, "the loop's tool step was routed through the runner + audited") - } - - /// `refactor-park-and-background-agents`: a parked `.confirm` step PAUSES the loop — the step is - /// neither run nor skipped, and NO final answer is fabricated over the phantom work (the old - /// `continue` on `.awaitingApproval` marched on and synthesized a completion). - func testAgentLoopPausesOnParkedConfirmWithoutFabricatingAnAnswer() async { - let audit = InMemoryAuditLog() - let d = desc("send_to", .confirm) - let contributor = FakeContributor([d], result: ToolStepResult(tool: d.name, status: .done, summary: "sent")) - let bg = BackgroundToolRunner(resolver: BackgroundPolicyResolver(), audit: audit, - parkStateOf: { _ in .parked }) - let rt = RoutingRuntime(routes: ["{\"tool\":\"send_to\"}", "{\"tool\":\"\"}"], - answer: ["should never stream"]) - let loop = AgentLoop(runtime: rt, registry: ToolRegistry([contributor]), - candidateSource: KeywordToolCandidateSource(all: [d]), - gate: AutoApproveGate(), backgroundRunner: bg) - let result = await loop.run(context: RouteContext(messages: [AgentMessage(role: .user, text: "send it")])) - XCTAssertEqual(result.outcome, .pausedAwaitingUser, "a parked confirm pauses honestly") - XCTAssertFalse(contributor.runCalled, "the pending step never fired") - XCTAssertEqual(result.steps.last?.status, .awaitingApproval, "the pending step is observable state") - } - - /// A parked `.dangerous` step pauses the loop AND escalates through the controller-routed callback. - func testAgentLoopPausesAndEscalatesOnParkedDangerous() async { - let audit = InMemoryAuditLog() - let spy = EscalationSpy() - let d = desc("launch_claude", .dangerous) - let contributor = FakeContributor([d], result: ToolStepResult(tool: d.name, status: .done, summary: "x")) - let bg = BackgroundToolRunner(resolver: BackgroundPolicyResolver(), audit: audit, - onEscalate: { spy.record($0, $1) }, - parkStateOf: { _ in .parked }) - let rt = RoutingRuntime(routes: ["{\"tool\":\"launch_claude\"}", "{\"tool\":\"\"}"], - answer: ["should never stream"]) - let loop = AgentLoop(runtime: rt, registry: ToolRegistry([contributor]), - candidateSource: KeywordToolCandidateSource(all: [d]), - gate: AutoApproveGate(), backgroundRunner: bg) - let result = await loop.run(context: RouteContext(messages: [AgentMessage(role: .user, text: "go")])) - XCTAssertEqual(result.outcome, .pausedAwaitingUser) - XCTAssertFalse(contributor.runCalled, "the dangerous step never fired in the background") - XCTAssertEqual(spy.escalated.count, 1, "the escalation reached the controller seam") - } - - /// A minimal routing runtime: structured() dequeues scripted route JSON; generate() streams an answer. - private final class RoutingRuntime: LLMRuntime, @unchecked Sendable { - let capabilities: Set = [.text] - private var routes: [String]; private let answerTokens: [String]; private let lock = NSLock() - init(routes: [String], answer: [String]) { self.routes = routes; self.answerTokens = answer } - func generate(_ request: LLMRequest) -> AsyncThrowingStream { - let toks = answerTokens - return AsyncThrowingStream { c in for (i, t) in toks.enumerated() { c.yield(Token(t, isFinal: i == toks.count - 1)) }; c.finish() } - } - func structured(_ r: LLMRequest, schema: StructuredSchema, as type: T.Type) async throws -> StructuredOutcome { - let json: String = { lock.lock(); defer { lock.unlock() }; return routes.isEmpty ? "{\"tool\":\"\"}" : routes.removeFirst() }() - return .value(try JSONDecoder().decode(T.self, from: Data(json.utf8))) - } - } -} diff --git a/Tests/ThreeFingerSwitcherTests/BatchedRuntimeTests.swift b/Tests/ThreeFingerSwitcherTests/BatchedRuntimeTests.swift deleted file mode 100644 index 478ea26..0000000 --- a/Tests/ThreeFingerSwitcherTests/BatchedRuntimeTests.swift +++ /dev/null @@ -1,233 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for the pure-Core substrate of `ai-batched-runtime-and-context` (tasks §1–§3, §5.1, §6): the -/// RAM-is-the-ceiling concurrency math, the interleaved-attention KV sum, the concrete context-budget -/// provider (the C3 tie-in to conversation-runtime compaction), context settings persistence, the -/// fixed-pattern subagent, and a stub batched runtime's de-mux. The MLX conformer (§4) is xcodebuild -/// compile-verify only; its real behavior is the user's stable-signed run-verify (task 8.3). -@MainActor -final class BatchedRuntimeTests: XCTestCase { - - // MARK: - KVCacheCost (interleaved sliding/global attention) - - func testKVInterleavedSumClampsSlidingAtWindowAndDiffersFromUniformGlobal() { - let cost = KVCacheCost(slidingLayers: 30, globalLayers: 6, slidingWindow: 1024, kvBytesPerTokenPerLayer: 1.0) - // At a context below the window, sliding and global both scale with ctx. - let short = cost.kvBytes(forContext: 512) // (30*512 + 6*512) = 18432 - XCTAssertEqual(short, 18_432) - // Beyond the window, sliding layers clamp at the window while global layers keep growing. - let long = cost.kvBytes(forContext: 8192) // (30*1024 + 6*8192) = 30720 + 49152 = 79872 - XCTAssertEqual(long, 79_872) - // Treating every layer as global would massively over-estimate (36 * 8192 = 294912). - XCTAssertLessThan(long, 294_912, "interleaved accounting is far below uniform-global") - } - - // MARK: - ConcurrencyBudget (K is RAM-derived, clamped ≥ 1) - - private func budget() -> ConcurrencyBudget { - ConcurrencyBudget( - unifiedMemoryBytes: 48 * 1_000_000_000, - weightBytes: 17 * 1_000_000_000, - reservedBytes: 6 * 1_000_000_000, - kv: KVCacheCost(slidingLayers: 30, globalLayers: 6, slidingWindow: 1024, - kvBytesPerTokenPerLayer: 20_000)) // exaggerated so K is small + visible - } - - func testMaxStreamsDecreasesMonotonicallyAsContextGrows() { - let b = budget() - let kShort = b.maxStreams(contextTokens: 1_000) - let kMid = b.maxStreams(contextTokens: 8_000) - let kLong = b.maxStreams(contextTokens: 64_000) - XCTAssertGreaterThanOrEqual(kShort, kMid) - XCTAssertGreaterThanOrEqual(kMid, kLong) - XCTAssertGreaterThan(kShort, kLong, "growing context honestly lowers the concurrent-stream count") - } - - func testMaxStreamsClampsToAtLeastOneEvenAtModelMax() { - let b = budget() - XCTAssertGreaterThanOrEqual(b.maxStreams(contextTokens: 131_072), 1, - "the foreground session always fits — never an OOM at model-max context") - } - - func testEstimatedRAMTracksStreamsAndContext() { - let b = budget() - let one = b.estimatedRAM(streams: 1, contextTokens: 4_000) - let three = b.estimatedRAM(streams: 3, contextTokens: 4_000) - XCTAssertGreaterThan(three, one, "more streams ⇒ more KV RAM") - XCTAssertGreaterThan(one, b.weightBytes, "resident RAM includes the weights read once") - } - - // MARK: - ModelDescriptor.maxContextTokens - - func testRegistryCarriesMaxContextTokens() { - let d = ModelCatalog.standard.descriptor(id: "gemma-4-31b") - XCTAssertEqual(d?.maxContextTokens, 131_072) - XCTAssertEqual(ModelCatalog.standard.descriptor(id: "gemma-4-12b")?.maxContextTokens, 32_768) - } - - // MARK: - Context preset + concrete budget provider (C3 tie-in) - - func testPresetResolvesAndClampsToModelMax() { - XCTAssertEqual(AgentContextPreset.balanced.tokens(modelMax: 131_072, custom: 0), 8_192) - XCTAssertEqual(AgentContextPreset.max.tokens(modelMax: 32_768, custom: 0), 32_768) - XCTAssertEqual(AgentContextPreset.long.tokens(modelMax: 16_000, custom: 0), 16_000, "Long clamps to a smaller model max") - XCTAssertEqual(AgentContextPreset.custom.tokens(modelMax: 10_000, custom: 999_999), 10_000, "an oversize custom clamps") - } - - func testProviderEffectiveBudgetIsClampedMinOfUserSkillAndModelMax() { - struct FixedOverride: SkillContextOverriding { let v: Int?; func contextOverride(forSkill: String?) -> Int? { v } } - // A heavy skill raises the budget above the user's value... - let raised = AgentContextBudgetProvider(userContextTokens: 8_192, modelMaxContextTokens: 131_072, - activeSkillID: "summarizer", skillOverrides: FixedOverride(v: 64_000)) - XCTAssertEqual(raised.maxContextTokens, 64_000, "max(user, skillOverride)") - // ...but everything is clamped to the model max. - let clamped = AgentContextBudgetProvider(userContextTokens: 999_999, modelMaxContextTokens: 32_768, - skillOverrides: FixedOverride(v: nil)) - XCTAssertEqual(clamped.maxContextTokens, 32_768, "∩ model max") - } - - func testGrowingTheBudgetDefersCompaction() { - // The C3 tie-in: conversation-runtime's compaction reads THIS provider, so a larger budget defers - // compaction and a smaller one triggers it sooner — they never disagree about "the budget". - let convo = AgentConversation(title: "t", - messages: (0..<20).map { AgentMessage(role: .user, text: "message number \($0) with some words") }) - let big = AgentContextBudgetProvider(userContextTokens: 100_000, modelMaxContextTokens: 131_072) - let small = AgentContextBudgetProvider(userContextTokens: 10, modelMaxContextTokens: 131_072) - XCTAssertFalse(ConversationCompactor.needsCompaction(convo, budget: big), "a large budget defers compaction") - XCTAssertTrue(ConversationCompactor.needsCompaction(convo, budget: small), "a small budget triggers it sooner") - } - - // MARK: - AppSettings persistence (defaults / clamp-at-use / reset / legacy) - - private func freshSettings() -> AppSettings { - AppSettings(defaults: UserDefaults(suiteName: "tfs-batched-\(UUID().uuidString)")!) - } - - func testContextSettingsDefaultsAndLegacyDecode() { - let s = freshSettings() // empty suite = legacy (no keys written) - XCTAssertEqual(s.agentContextPreset, .balanced) - XCTAssertEqual(s.agentContextTokens, 8_192) - XCTAssertFalse(s.agentCompactKV) - } - - func testContextSettingsPersistAndReset() { - let suite = UserDefaults(suiteName: "tfs-batched-persist-\(UUID().uuidString)")! - let s = AppSettings(defaults: suite) - s.agentContextPreset = .max - s.agentContextTokens = 131_072 - s.agentCompactKV = true - // A fresh instance over the same suite reads the persisted values. - let reloaded = AppSettings(defaults: suite) - XCTAssertEqual(reloaded.agentContextPreset, .max) - XCTAssertEqual(reloaded.agentContextTokens, 131_072) - XCTAssertTrue(reloaded.agentCompactKV) - // Reset-to-defaults returns them to Balanced / off (a behavior tunable). - reloaded.resetToDefaults() - XCTAssertEqual(reloaded.agentContextPreset, .balanced) - XCTAssertEqual(reloaded.agentContextTokens, 8_192) - XCTAssertFalse(reloaded.agentCompactKV) - } - - // MARK: - Subagent (fixed-pattern context hygiene) - - func testSubagentRunsInFreshContextAndIsBounded() { - let sub = Subagent(name: "summarize_docs", systemPrompt: "You summarize.", maxTurns: 0) - XCTAssertEqual(sub.maxTurns, 1, "maxTurns is bounded ≥ 1") - let convo = sub.freshConversation(input: "the long document") - XCTAssertEqual(convo.messages.map(\.role), [.system, .user], "a fresh, isolated conversation — no orchestrator history") - XCTAssertEqual(convo.messages[1].text, "the long document") - XCTAssertEqual(convo.skillID, "summarize_docs") - XCTAssertNotEqual(sub.freshConversation(input: "a").id, sub.freshConversation(input: "a").id, - "each run gets a distinct session id") - } - - func testSubagentExposesARoutableToolDescriptor() { - let sub = Subagent(name: "research", systemPrompt: "…") - let d = sub.toolDescriptor - XCTAssertEqual(d.name, "subagent:research") - XCTAssertEqual(d.writePolicy, .auto, "a subagent is read-only to the orchestrator's world") - } - - // MARK: - Stub batched runtime de-mux - - func testBatchStepDeMuxesTokensToTheRightSession() async throws { - let a = AgentSessionID(), b = AgentSessionID() - let stub = StubBatchedRuntime(maxConcurrentStreams: 3, - perStream: [a: ["a1", "a2"], b: ["b1", "b2", "b3"]]) - var byID: [AgentSessionID: [String]] = [:] - for try await (id, token) in stub.batchStep([a: req(), b: req()]) { - byID[id, default: []].append(token.text) - } - XCTAssertEqual(byID[a], ["a1", "a2"], "stream A's tokens de-mux to A") - XCTAssertEqual(byID[b], ["b1", "b2", "b3"], "stream B's tokens de-mux to B") - } - - private func req() -> LLMChatRequest { LLMChatRequest(messages: [AgentMessage(role: .user, text: "hi")]) } - - // MARK: - AgentContextCostModel (the Hub cost surface, task 5.3) - - func testCostModelBackgroundStreamsDropAsContextGrows() { - let mem = Int64(48) * 1_000_000_000 - let weights = Int64(17) * 1_000_000_000 - let small = AgentContextCostModel(contextTokens: 4_000, compactKV: false, weightBytes: weights, unifiedMemoryBytes: mem) - let large = AgentContextCostModel(contextTokens: 100_000, compactKV: false, weightBytes: weights, unifiedMemoryBytes: mem) - XCTAssertGreaterThanOrEqual(small.maxStreams, large.maxStreams, "more context → no more streams") - XCTAssertGreaterThanOrEqual(small.backgroundStreams, large.backgroundStreams) - XCTAssertGreaterThanOrEqual(large.maxStreams, 1, "foreground always fits (K ≥ 1)") - } - - func testCostModelCompactKVAffordsAtLeastAsManyStreams() { - let mem = Int64(48) * 1_000_000_000 - let weights = Int64(17) * 1_000_000_000 - let bf16 = AgentContextCostModel(contextTokens: 64_000, compactKV: false, weightBytes: weights, unifiedMemoryBytes: mem) - let kv8 = AgentContextCostModel(contextTokens: 64_000, compactKV: true, weightBytes: weights, unifiedMemoryBytes: mem) - XCTAssertGreaterThanOrEqual(kv8.maxStreams, bf16.maxStreams, "8-bit KV halves per-token cost → ≥ streams") - XCTAssertLessThanOrEqual(kv8.estimatedRAMBytes, bf16.estimatedRAMBytes + weights, "compact KV is not more RAM per stream") - } - - func testCostModelSpeedNoteBucketsByContext() { - let weights = Int64(17) * 1_000_000_000 - XCTAssertEqual(AgentContextCostModel(contextTokens: 8_000, compactKV: false, weightBytes: weights).speedNote, "fastest per-token speed") - XCTAssertEqual(AgentContextCostModel(contextTokens: 32_000, compactKV: false, weightBytes: weights).speedNote, "moderate per-token speed") - XCTAssertEqual(AgentContextCostModel(contextTokens: 100_000, compactKV: false, weightBytes: weights).speedNote, "slower per-token speed") - } - - func testCostModelBackgroundTextHonestAtKEqualsOne() { - // A tiny memory budget so only the foreground fits → "no background sessions" honestly. - let cost = AgentContextCostModel(contextTokens: 100_000, compactKV: false, - weightBytes: Int64(45) * 1_000_000_000, - unifiedMemoryBytes: Int64(48) * 1_000_000_000) - XCTAssertEqual(cost.maxStreams, 1) - XCTAssertEqual(cost.backgroundStreams, 0) - XCTAssertEqual(cost.backgroundText, "no background sessions") - } - - /// A deterministic `BatchedLLMRuntime` (test-only, task 1.4): scripts per-stream token sequences and - /// de-muxes them by `AgentSessionID`. - private final class StubBatchedRuntime: BatchedLLMRuntime, @unchecked Sendable { - let capabilities: Set = [.text] - let maxConcurrentStreams: Int - private let perStream: [AgentSessionID: [String]] - init(maxConcurrentStreams: Int, perStream: [AgentSessionID: [String]]) { - self.maxConcurrentStreams = maxConcurrentStreams - self.perStream = perStream - } - func generate(_ request: LLMRequest) -> AsyncThrowingStream { - AsyncThrowingStream { c in c.yield(Token(request.prompt, isFinal: true)); c.finish() } - } - func structured(_ request: LLMRequest, schema: StructuredSchema, - as type: T.Type) async throws -> StructuredOutcome { - throw RuntimeError.couldNotProduceValid(attempts: 1) - } - func batchStep(_ requests: [AgentSessionID: LLMChatRequest]) -> AsyncThrowingStream<(AgentSessionID, Token), Error> { - let scripts = requests.keys.map { ($0, perStream[$0] ?? ["?"]) } - return AsyncThrowingStream { c in - for (id, toks) in scripts { - for (i, t) in toks.enumerated() { c.yield((id, Token(t, isFinal: i == toks.count - 1))) } - } - c.finish() - } - } - } -} diff --git a/Tests/ThreeFingerSwitcherTests/BidiTextDirectionTests.swift b/Tests/ThreeFingerSwitcherTests/BidiTextDirectionTests.swift deleted file mode 100644 index fa8f56b..0000000 --- a/Tests/ThreeFingerSwitcherTests/BidiTextDirectionTests.swift +++ /dev/null @@ -1,90 +0,0 @@ -import XCTest -import SwiftUI -@testable import ThreeFingerSwitcherCore - -/// Tests for the preview canvas's bidi base-direction heuristic (spec launcher-overlay: "Bidirectional -/// (RTL/LTR) text rendering"). Each line's base direction is decided by its **first strong character** -/// (the first word/char), is **stable** once that char is present (later content never re-decides it), -/// and RTL detection covers **all** RTL scripts. The heuristic is pure; the actual right-aligned -/// `NSTextView` rendering is confirmed on a signed build. -final class BidiTextDirectionTests: XCTestCase { - - // Sample strong characters (kept as scalars so the source stays ASCII-safe and unambiguous). - private let hebrew = "\u{05E9}\u{05DC}\u{05D5}\u{05DD}" // שלום - private let hebrewWord = "\u{05DE}\u{05D9}\u{05DC}\u{05D4}" // מילה - private let arabic = "\u{0645}\u{0631}\u{062D}\u{0628}\u{0627}" // مرحبا - private let syriac = "\u{0710}\u{0712}\u{0713}\u{0714}" // Syriac - private let thaana = "\u{0780}\u{0781}\u{0782}\u{0783}" // Thaana - private let nko = "\u{07C1}\u{07C2}\u{07C3}\u{07C4}" // N'Ko - - // MARK: - First strong char decides the line - - func testFirstStrongCharDecidesTheLine() { - XCTAssertEqual(firstStrongDirection(hebrew), .rightToLeft, "a Hebrew-first line is right-to-left") - XCTAssertEqual(firstStrongDirection("Hello " + hebrew), .leftToRight, - "a line that STARTS with Latin is left-to-right (the first word decides)") - XCTAssertEqual(firstStrongDirection(hebrew + " Hello"), .rightToLeft, - "a line that starts with Hebrew is right-to-left even with trailing Latin") - } - - func testLeadingNeutralsAreSkippedToTheFirstStrongChar() { - XCTAssertEqual(firstStrongDirection(" \t " + hebrew), .rightToLeft, "leading whitespace is skipped") - XCTAssertEqual(firstStrongDirection("3. " + hebrew), .rightToLeft, "a leading number/period is skipped") - XCTAssertEqual(firstStrongDirection("- " + arabic), .rightToLeft, "a leading bullet marker is skipped") - XCTAssertEqual(firstStrongDirection("\u{2022} " + hebrew), .rightToLeft, "a leading bullet glyph is skipped") - } - - // MARK: - STABLE: later characters never re-decide the side - - func testDirectionIsStableAsMoreContentIsAppended() { - // Once the first strong char is present, appending opposite-direction content does NOT flip it — - // the line's alignment is fixed by how it starts (the user's rule). - XCTAssertEqual(firstStrongDirection(hebrew), .rightToLeft) - XCTAssertEqual(firstStrongDirection(hebrew + " hello world foo bar baz qux"), .rightToLeft, - "a Hebrew-first line stays RTL no matter how much Latin follows") - XCTAssertEqual(firstStrongDirection("Hi"), .leftToRight) - XCTAssertEqual(firstStrongDirection("Hi " + hebrew + hebrew + hebrew + hebrew), .leftToRight, - "a Latin-first line stays LTR no matter how much Hebrew follows") - } - - func testStreamingOnlyAdoptsASideOnceTheFirstStrongCharArrives() { - // A paragraph that streams neutrals first is LTR (default) until its first strong char arrives, - // then locks to that side and stays there. - XCTAssertEqual(firstStrongDirection(""), .leftToRight, "empty ⇒ LTR default") - XCTAssertEqual(firstStrongDirection("- "), .leftToRight, "neutral-only prefix ⇒ LTR default") - XCTAssertEqual(firstStrongDirection("- " + hebrew), .rightToLeft, "adopts RTL when the first strong char arrives") - XCTAssertEqual(firstStrongDirection("- " + hebrew + " and more"), .rightToLeft, "then stays RTL") - } - - // MARK: - Neutral-only ⇒ LTR - - func testNeutralOnlyIsLeftToRight() { - XCTAssertEqual(firstStrongDirection(" \n\t "), .leftToRight, "whitespace-only ⇒ LTR") - XCTAssertEqual(firstStrongDirection("123 456-789 (00%)"), .leftToRight, "digits/punctuation only ⇒ LTR") - XCTAssertEqual(firstStrongDirection("https://example.com/path?q=1"), .leftToRight, "a bare URL ⇒ LTR") - } - - // MARK: - All RTL scripts, not just Hebrew/Arabic - - func testAllRTLScriptsResolveRTL() { - XCTAssertEqual(firstStrongDirection(arabic), .rightToLeft, "Arabic is RTL") - XCTAssertEqual(firstStrongDirection(syriac), .rightToLeft, "Syriac is RTL") - XCTAssertEqual(firstStrongDirection(thaana), .rightToLeft, "Thaana is RTL") - XCTAssertEqual(firstStrongDirection(nko), .rightToLeft, "N'Ko is RTL") - } - - // MARK: - Strong-class helpers (regression guard: Hebrew letters are alphabetic — must not count LTR) - - func testStrongClassHelpers() { - let alef: UInt32 = 0x05D0 // א - XCTAssertTrue(isStrongRTL(alef), "a Hebrew letter is strong RTL") - XCTAssertFalse(isStrongLTR(alef), "a Hebrew letter must NOT also count as strong LTR (it is alphabetic)") - let a: UInt32 = 0x0041 // A - XCTAssertTrue(isStrongLTR(a)) - XCTAssertFalse(isStrongRTL(a)) - for neutral: UInt32 in [0x0033 /* 3 */, 0x0020 /* space */, 0x002E /* . */, 0x002D /* - */] { - XCTAssertFalse(isStrongLTR(neutral), "\(neutral) is neutral, not strong LTR") - XCTAssertFalse(isStrongRTL(neutral), "\(neutral) is neutral, not strong RTL") - } - } -} diff --git a/Tests/ThreeFingerSwitcherTests/CanvasResolveBindingTests.swift b/Tests/ThreeFingerSwitcherTests/CanvasResolveBindingTests.swift deleted file mode 100644 index ad31a3b..0000000 --- a/Tests/ThreeFingerSwitcherTests/CanvasResolveBindingTests.swift +++ /dev/null @@ -1,87 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// The AI-canvas resolve interpretation is a pure helper (`AppCoordinator.canvasResolveDecision`) so it -/// can be asserted without standing up the whole coordinator. The recognizer has already axis-locked, so -/// exactly one of `dx`/`dy` is non-zero, with the fixed sign convention `dy<0 → swipeDown`, -/// `dy>0 → swipeUp`, `dx<0 → swipeLeft`, `dx>0 → swipeRight` (`add-gesture-previews-and-bindings` §9.3). -/// -/// These cover: the default binding reproduces today's grammar (down=commit, up=ignore, both horizontals -/// discard), and a remap (assigning swipe-right to commit) moves commit to the new excursion while the -/// previously-default excursion no longer commits. The `canvasAtTop` commit guard lives at the call site -/// (binding-independent) and is verified in the coordinator-level integration, not here. -@MainActor -final class CanvasResolveBindingTests: XCTestCase { - - private func decide(_ dx: Int, _ dy: Int, _ binding: GestureBindings.CanvasBinding) - -> AppCoordinator.CanvasResolveDecision { - AppCoordinator.canvasResolveDecision(dx: dx, dy: dy, binding: binding) - } - - // MARK: - Default binding reproduces today's grammar - - /// Default (commit=down, dismiss=left, ignore=up, spare=right): down = commit, up = ignore, BOTH - /// horizontals discard (left is bound to dismiss; right is the spare horizontal → also discards). - func test_defaultBinding_reproducesTodaysGrammar() { - let c = GestureBindings.CanvasBinding.default - XCTAssertEqual(decide(0, -1, c), .commit, "down = commit") - XCTAssertEqual(decide(0, 1, c), .ignore, "up = ignore") - XCTAssertEqual(decide(-1, 0, c), .discard, "left = dismiss") - XCTAssertEqual(decide( 1, 0, c), .discard, "right (spare horizontal) also discards") - } - - // MARK: - Remap: commit follows the binding; the old default no longer commits - - /// After binding commit to swipe-right, a right excursion commits and a down excursion no longer - /// commits (down inherits dismiss's old excursion via the swap → discard). - func test_remapCommitToSwipeRight_rightCommits_downNoLongerCommits() { - // Default has dismiss=left, ignore=up, commit=down, right=spare. Assigning right→commit swaps - // right with whichever action holds it — none does (right is spare) — so commit just moves to - // right and down becomes the spare. - let c = GestureBindings.CanvasBinding.default.assigning(.swipeRight, to: .commit) - XCTAssertEqual(c.commit, .swipeRight) - - XCTAssertEqual(decide(1, 0, c), .commit, "right now commits") - XCTAssertNotEqual(decide(0, -1, c), .commit, "down no longer commits") - // Down is now the spare VERTICAL excursion → ignored (not a horizontal, so not a discard). - XCTAssertEqual(decide(0, -1, c), .ignore, "the freed down excursion (vertical spare) is ignored") - // Up is still bound to ignore. - XCTAssertEqual(decide(0, 1, c), .ignore, "up stays ignore") - // Left is still bound to dismiss. - XCTAssertEqual(decide(-1, 0, c), .discard, "left stays dismiss") - } - - /// A remap that swaps two bound actions: assign swipe-left (held by dismiss) to commit. The swap gives - /// dismiss the excursion commit used to hold (down). So left commits, down discards, up ignores. - func test_remapSwapsCommitAndDismiss() { - let c = GestureBindings.CanvasBinding.default.assigning(.swipeLeft, to: .commit) - XCTAssertEqual(c.commit, .swipeLeft) - XCTAssertEqual(c.dismiss, .swipeDown, "dismiss inherited commit's old excursion") - - XCTAssertEqual(decide(-1, 0, c), .commit, "left now commits") - XCTAssertEqual(decide(0, -1, c), .discard, "down now dismisses") - XCTAssertEqual(decide(0, 1, c), .ignore, "up stays ignore") - // Right is the spare horizontal → discard. - XCTAssertEqual(decide(1, 0, c), .discard, "right (spare horizontal) discards") - } - - // MARK: - Spare-excursion fallback is axis-aware - - /// With a binding whose spare excursion is VERTICAL, the spare is ignored (only a horizontal spare - /// discards). Bind ignore to swipe-right so the spare becomes swipe-up (vertical). - func test_verticalSpareIsIgnored_horizontalSpareDiscards() { - // Default ignore=up. Move ignore to right → up becomes the spare (vertical). - let c = GestureBindings.CanvasBinding.default.assigning(.swipeRight, to: .ignore) - XCTAssertEqual(c.ignore, .swipeRight) - XCTAssertEqual(decide(0, 1, c), .ignore, "the freed up excursion (vertical spare) is ignored") - XCTAssertEqual(decide(1, 0, c), .ignore, "right is now bound to ignore") - // Commit/dismiss unchanged. - XCTAssertEqual(decide(0, -1, c), .commit, "down stays commit") - XCTAssertEqual(decide(-1, 0, c), .discard, "left stays dismiss") - } - - /// A zero delta (no excursion) is a no-op (treated as ignore) — defensive; the recognizer never emits it. - func test_zeroDeltaIsIgnored() { - XCTAssertEqual(decide(0, 0, .default), .ignore) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/ClaudeHandoffTests.swift b/Tests/ThreeFingerSwitcherTests/ClaudeHandoffTests.swift deleted file mode 100644 index 1e86951..0000000 --- a/Tests/ThreeFingerSwitcherTests/ClaudeHandoffTests.swift +++ /dev/null @@ -1,411 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for the Claude-handoff escalation slice (`ai-claude-handoff`): the `launch_claude` descriptor -/// (always `.dangerous`), the rolling-24h budget cap (cap / window / concurrency / refund / persistence), -/// the production launcher's prompt→inner-command + error mapping, and the contributor's cost-gate state -/// machine (confirm gates, auto-under-budget runs, auto-over-budget escalates to confirm, fire-and-forget -/// fires the opener exactly once, parked escalation, redacted audit on every branch). All driven by a -/// fake `HandoffLauncher` + scripted `ApprovalGate` + recording audit/escalation — no real spawn. -final class ClaudeHandoffTests: XCTestCase { - - // MARK: - Fakes - - /// A launcher that records every `(folder, prompt)` and can be scripted to throw. - private final class FakeLauncher: HandoffLauncher, @unchecked Sendable { - private let lock = NSLock() - private(set) var calls: [(folder: URL, prompt: String)] = [] - var throwError: Error? - - func launch(folder: URL, prompt: String) async throws { - lock.lock(); calls.append((folder, prompt)); let e = throwError; lock.unlock() - if let e { throw e } - } - - var callCount: Int { lock.lock(); defer { lock.unlock() }; return calls.count } - } - - private final class ScriptedGate: ApprovalGate, @unchecked Sendable { - private let lock = NSLock() - private var decisions: [ApprovalDecision] - private(set) var asked = 0 - init(_ decisions: [ApprovalDecision]) { self.decisions = decisions } - func awaitDecision(for review: TaskReview) async -> ApprovalDecision { - lock.lock(); defer { lock.unlock() } - asked += 1 - return decisions.isEmpty ? .cancel : decisions.removeFirst() - } - } - - private final class RecordingAudit: HandoffAuditing, @unchecked Sendable { - private let lock = NSLock() - private(set) var records: [AuditRecord] = [] - func record(_ record: AuditRecord) async { - lock.lock(); records.append(record); lock.unlock() - } - var count: Int { lock.lock(); defer { lock.unlock() }; return records.count } - } - - private final class RecordingEscalation: HandoffEscalating, @unchecked Sendable { - private let lock = NSLock() - private(set) var escalations: [(AgentSessionID, String)] = [] - func escalate(_ sessionID: AgentSessionID, reason: String) async { - lock.lock(); escalations.append((sessionID, reason)); lock.unlock() - } - var count: Int { lock.lock(); defer { lock.unlock() }; return escalations.count } - } - - /// A resolver that keeps the dangerous tier dangerous (the default whitelist behaviour — handoff is - /// never whitelist-lowered to auto). - private struct KeepDangerous: WritePolicyResolving { - func effectiveTier(for descriptor: ToolDescriptor) -> WritePolicyTier { descriptor.writePolicy } - } - - /// A resolver that lowers ANY tier to auto — models a user who whitelisted handoff (so a skill's auto - /// can take effect). - private struct WhitelistAuto: WritePolicyResolving { - func effectiveTier(for descriptor: ToolDescriptor) -> WritePolicyTier { .auto } - } - - // MARK: - Helpers - - private let t0 = Date(timeIntervalSince1970: 1_700_000_000) - - private func call(folder: String? = "/tmp/proj", prompt: String? = "fix the bug") -> RoutedCall { - var props: [String: String] = [:] - if let folder { props["folder"] = folder } - if let prompt { props["prompt"] = prompt } - let json = ((try? JSONSerialization.data(withJSONObject: props)).flatMap { String(data: $0, encoding: .utf8) }) ?? "{}" - return RoutedCall(descriptor: ClaudeHandoffContributor.descriptor(), - route: ToolRoute(tool: ClaudeHandoffContributor.toolName, argumentsJSON: json), - userText: "please refactor", source: TaskSource()) - } - - // MARK: - Descriptor - - func testDescriptorIsAlwaysDangerous() { - let d = ClaudeHandoffContributor.descriptor() - XCTAssertEqual(d.name, "launch_claude") - XCTAssertEqual(d.writePolicy, .dangerous) - XCTAssertTrue(d.argsSchema.json.contains("\"prompt\"")) - XCTAssertTrue(d.argsSchema.json.contains("\"required\":[\"prompt\"]")) - } - - func testCanHandle() { - let c = makeContributor() - XCTAssertTrue(c.canHandle("launch_claude")) - XCTAssertFalse(c.canHandle("add_to_calendar")) - } - - // MARK: - ClaudeHandoffConfig (existing type, consumed) - - func testConfigDefaultsToConfirm() throws { - let cfg = ClaudeHandoffConfig() - XCTAssertEqual(cfg.confirmMode, .confirm) - // Codable round-trip; a config omitting confirmMode is invalid JSON for this struct, but a full - // round-trip preserves the mode. - let data = try JSONEncoder().encode(cfg) - let back = try JSONDecoder().decode(ClaudeHandoffConfig.self, from: data) - XCTAssertEqual(back, cfg) - } - - // MARK: - Budget - - func testBudgetAllowsUnderCapBlocksAtCap() { - var b = HandoffBudget(maxCallsPerDay: 2, maxConcurrent: 5) - XCTAssertTrue(b.allows(now: t0)) - b.record(at: t0); b.reap() - XCTAssertTrue(b.allows(now: t0)) - b.record(at: t0); b.reap() - XCTAssertFalse(b.allows(now: t0), "at the cap it must block") - } - - func testBudgetRollingWindowNotCalendarDay() { - // Two spends 25h apart: only the recent one is inside the 24h window ending at `now`. - let old = t0 - let recent = t0.addingTimeInterval(25 * 3600) - var b = HandoffBudget(maxCallsPerDay: 2, ledger: [HandoffSpend(at: old)]) - b.record(at: recent); b.reap() - // now just after `recent`: the `old` spend has rolled out of the window. - XCTAssertEqual(b.callsInLast24h(recent.addingTimeInterval(60)), 1) - XCTAssertTrue(b.allows(now: recent.addingTimeInterval(60))) - } - - func testBudgetMidnightCannotBeGamed() { - // N at 23:59 and N at 00:01 fall in the same rolling window → counted together. - let late = Date(timeIntervalSince1970: 1_700_000_000) // some "23:59" - let earlyNextDay = late.addingTimeInterval(2 * 60) // 2 minutes later "00:01" - var b = HandoffBudget(maxCallsPerDay: 2, ledger: [HandoffSpend(at: late)]) - b.record(at: earlyNextDay); b.reap() - XCTAssertEqual(b.callsInLast24h(earlyNextDay), 2) - XCTAssertFalse(b.allows(now: earlyNextDay), "no midnight reset — both count") - } - - func testBudgetConcurrencyBlocks() { - var b = HandoffBudget(maxCallsPerDay: 10, maxConcurrent: 1) - b.record(at: t0) // in-flight = 1, not reaped - XCTAssertFalse(b.allows(now: t0), "in-flight at maxConcurrent blocks") - b.reap() - XCTAssertTrue(b.allows(now: t0)) - } - - func testBudgetRefundRestoresSlot() { - var b = HandoffBudget(maxCallsPerDay: 1, maxConcurrent: 1) - b.record(at: t0) - XCTAssertFalse(b.allows(now: t0)) - b.refund(at: t0) - XCTAssertEqual(b.callsInLast24h(t0), 0) - XCTAssertEqual(b.inFlight, 0) - XCTAssertTrue(b.allows(now: t0), "a refunded launch leaves the cap unchanged") - } - - func testBudgetPersistsAcrossReload() { - let store = InMemoryHandoffLedgerStore() - let box1 = HandoffBudgetBox(maxCallsPerDay: 1, store: store) - box1.record(at: t0, skillID: nil); box1.reap() - XCTAssertFalse(box1.allows(now: t0)) - // A "relaunch": a fresh box over the same store still sees the spend in the window. - let box2 = HandoffBudgetBox(maxCallsPerDay: 1, store: store) - XCTAssertFalse(box2.allows(now: t0), "the cap survives a relaunch within the window") - } - - // MARK: - Launcher prompt mapping + error mapping - - func testInnerCommandEmptyVsNonEmpty() { - XCTAssertNil(OpenClaudeHandoffLauncher.innerCommand(forPrompt: " ")) - XCTAssertNil(OpenClaudeHandoffLauncher.innerCommand(forPrompt: "")) - let cmd = OpenClaudeHandoffLauncher.innerCommand(forPrompt: "fix it") - XCTAssertEqual(cmd, "claude 'fix it'") - } - - func testLaunchErrorMapping() { - let notFound = OpenClaudeHandoffLauncher.map(.claudeNotFound) - XCTAssertEqual(AIError.message(for: notFound).headline, - ClaudeLaunchError.claudeNotFound.errorDescription) - let writeFail = OpenClaudeHandoffLauncher.map(.scriptWriteFailed(details: "disk full")) - XCTAssertEqual(AIError.message(for: writeFail).details, "disk full", - "raw text rides only in details, not the headline") - XCTAssertFalse(AIError.message(for: writeFail).headline.contains("disk full")) - } - - func testHandoffErrorTranslatorCleanHeadlines() { - for e in [HandoffError.disabled, .overBudgetNoUser, .missingFolder] { - let m = AIError.message(for: e) - XCTAssertFalse(m.headline.isEmpty) - XCTAssertFalse(m.headline.contains("HandoffError"), "no raw enum dump in a headline") - } - } - - // MARK: - Contributor state machine - - private func makeContributor(config: ClaudeHandoffConfig = ClaudeHandoffConfig(), - budget: HandoffBudgetBox? = nil, - launcher: FakeLauncher = FakeLauncher(), - audit: RecordingAudit = RecordingAudit(), - escalation: RecordingEscalation = RecordingEscalation(), - resolver: WritePolicyResolving = WhitelistAuto(), - isParked: Bool = false, - now: Date? = nil) -> ClaudeHandoffContributor { - let clock = now ?? t0 - return ClaudeHandoffContributor( - config: config, - budget: budget ?? HandoffBudgetBox(maxCallsPerDay: 5), - launcher: launcher, - audit: audit, - resolver: resolver, - escalation: escalation, - sessionID: AgentSessionID(), - isParked: isParked, - globalDefaultPerDay: 5, - now: { clock }) - } - - func testConfirmSkillGatesAndApprovesLaunchesOnce() async { - let launcher = FakeLauncher() - let audit = RecordingAudit() - let gate = ScriptedGate([.approve]) - let c = makeContributor(config: ClaudeHandoffConfig(confirmMode: .confirm), - launcher: launcher, audit: audit) - let result = await c.run(call(), gate: gate) - XCTAssertEqual(gate.asked, 1, "a confirm skill must gate") - XCTAssertEqual(result.status, .done) - XCTAssertEqual(launcher.callCount, 1, "fire-and-forget: the opener fires exactly once") - XCTAssertEqual(audit.count, 1) - XCTAssertEqual(audit.records.first?.outcome, .done) - } - - func testConfirmSkillSkipDoesNotSpendOrLaunch() async { - let launcher = FakeLauncher() - let budget = HandoffBudgetBox(maxCallsPerDay: 5) - let gate = ScriptedGate([.skip]) - let c = makeContributor(config: ClaudeHandoffConfig(confirmMode: .confirm), - budget: budget, launcher: launcher) - let result = await c.run(call(), gate: gate) - if case .declined = result.status {} else { XCTFail("skip → declined") } - XCTAssertEqual(launcher.callCount, 0, "skip never launches") - XCTAssertEqual(budget.snapshot().callsInLast24h(t0), 0, "skip never spends") - } - - func testAutoSkillUnderBudgetRunsWithoutGate() async { - let launcher = FakeLauncher() - let gate = ScriptedGate([]) // must NOT be consulted - let c = makeContributor(config: ClaudeHandoffConfig(confirmMode: .auto), launcher: launcher) - let result = await c.run(call(), gate: gate) - XCTAssertEqual(gate.asked, 0, "an auto skill under budget does not gate") - XCTAssertEqual(result.status, .done) - XCTAssertEqual(launcher.callCount, 1) - } - - func testAutoSkillRequiresWhitelistToRunUnprompted() async { - // A user who did NOT whitelist handoff keeps it dangerous → even an auto skill must gate. - let launcher = FakeLauncher() - let gate = ScriptedGate([.approve]) - let c = makeContributor(config: ClaudeHandoffConfig(confirmMode: .auto), - launcher: launcher, resolver: KeepDangerous()) - _ = await c.run(call(), gate: gate) - XCTAssertEqual(gate.asked, 1, "a non-whitelisted auto handoff stays foreground") - } - - func testAutoOverBudgetEscalatesToConfirm() async { - // Fill the cap first, then an auto call over budget must degrade to a foreground confirm. - let budget = HandoffBudgetBox(maxCallsPerDay: 1) - budget.record(at: t0, skillID: nil); budget.reap() // cap is 1 → now at cap - let launcher = FakeLauncher() - let gate = ScriptedGate([.approve]) - let c = makeContributor(config: ClaudeHandoffConfig(confirmMode: .auto), - budget: budget, launcher: launcher) - let result = await c.run(call(), gate: gate) - XCTAssertEqual(gate.asked, 1, "auto over budget degrades to a foreground confirm, never auto-runs") - XCTAssertEqual(result.status, .done, "the user can still approve the one extra call") - } - - func testAutoOverBudgetInParkedSessionEscalatesNeedsYou() async { - let budget = HandoffBudgetBox(maxCallsPerDay: 1) - budget.record(at: t0, skillID: nil); budget.reap() - let launcher = FakeLauncher() - let escalation = RecordingEscalation() - let gate = ScriptedGate([]) // never consulted in a parked session - let c = makeContributor(config: ClaudeHandoffConfig(confirmMode: .auto), - budget: budget, launcher: launcher, - escalation: escalation, isParked: true) - let result = await c.run(call(), gate: gate) - XCTAssertEqual(result.status, .awaitingApproval) - XCTAssertEqual(escalation.count, 1, "parked + over-budget escalates to needs-you") - XCTAssertEqual(launcher.callCount, 0, "no spend / launch until the user returns") - XCTAssertEqual(budget.snapshot().callsInLast24h(t0), 1, "no extra spend recorded") - } - - func testAutoUnderBudgetRunsWhileParked() async { - let launcher = FakeLauncher() - let escalation = RecordingEscalation() - let c = makeContributor(config: ClaudeHandoffConfig(confirmMode: .auto), - launcher: launcher, escalation: escalation, isParked: true) - let result = await c.run(call(), gate: ScriptedGate([])) - XCTAssertEqual(result.status, .done, "auto + under-budget + whitelisted runs in the background") - XCTAssertEqual(escalation.count, 0) - XCTAssertEqual(launcher.callCount, 1) - } - - func testConfirmWhileParkedEscalates() async { - let launcher = FakeLauncher() - let escalation = RecordingEscalation() - let c = makeContributor(config: ClaudeHandoffConfig(confirmMode: .confirm), - launcher: launcher, escalation: escalation, isParked: true) - let result = await c.run(call(), gate: ScriptedGate([])) - XCTAssertEqual(result.status, .awaitingApproval) - XCTAssertEqual(escalation.count, 1) - XCTAssertEqual(launcher.callCount, 0) - } - - func testMissingFolderFailsCleanlyNoSpend() async { - let launcher = FakeLauncher() - let budget = HandoffBudgetBox(maxCallsPerDay: 5) - // No route folder AND no skill default folder. - let c = makeContributor(config: ClaudeHandoffConfig(confirmMode: .auto), - budget: budget, launcher: launcher) - let result = await c.run(call(folder: nil), gate: ScriptedGate([.approve])) - if case let .failed(headline) = result.status { - XCTAssertFalse(headline.isEmpty) - } else { XCTFail("missing folder → failed") } - XCTAssertEqual(launcher.callCount, 0) - XCTAssertEqual(budget.snapshot().callsInLast24h(t0), 0, "no spend on a missing-folder failure") - } - - func testEmptyPromptLaunchesBareSession() async { - let launcher = FakeLauncher() - let c = makeContributor(config: ClaudeHandoffConfig(confirmMode: .auto), launcher: launcher) - let result = await c.run(call(prompt: ""), gate: ScriptedGate([])) - XCTAssertEqual(result.status, .done) - XCTAssertEqual(launcher.callCount, 1) - XCTAssertEqual(launcher.calls.first?.prompt, "", "empty prompt → bare session") - } - - func testFailedLaunchRefundsSpend() async { - let launcher = FakeLauncher() - launcher.throwError = HandoffError.launchFailed(headline: "Couldn't open Claude Code.", details: "boom") - let budget = HandoffBudgetBox(maxCallsPerDay: 5) - let audit = RecordingAudit() - let c = makeContributor(config: ClaudeHandoffConfig(confirmMode: .auto), - budget: budget, launcher: launcher, audit: audit) - let result = await c.run(call(), gate: ScriptedGate([])) - if case let .failed(headline) = result.status { - XCTAssertFalse(headline.contains("boom"), "raw detail never in the headline") - } else { XCTFail("a failed launch → .failed") } - XCTAssertEqual(budget.snapshot().callsInLast24h(t0), 0, "a launch that didn't land didn't spend") - XCTAssertEqual(budget.snapshot().inFlight, 0) - XCTAssertEqual(audit.records.last?.outcome, .failed(headline: "Couldn't open Claude Code.")) - } - - func testDisabledWhenCapZero() async { - let launcher = FakeLauncher() - let c = ClaudeHandoffContributor( - config: ClaudeHandoffConfig(confirmMode: .auto, maxPerDay: nil), - budget: HandoffBudgetBox(maxCallsPerDay: 0), - launcher: launcher, - resolver: WhitelistAuto(), - globalDefaultPerDay: 0, // global default also 0 → disabled - now: { self.t0 }) - let result = await c.run(call(), gate: ScriptedGate([])) - if case .declined = result.status {} else { XCTFail("a 0 cap → declined (disabled)") } - XCTAssertEqual(launcher.callCount, 0) - } - - // MARK: - Audit redaction + one-per-branch - - func testAuditNeverCarriesFullPrompt() async { - let secretPrompt = "deploy with token=SUPERSECRETLONGVALUE12345 and also do a thing" - let audit = RecordingAudit() - let c = makeContributor(config: ClaudeHandoffConfig(confirmMode: .auto), audit: audit) - _ = await c.run(call(prompt: secretPrompt), gate: ScriptedGate([])) - XCTAssertEqual(audit.count, 1, "exactly one audit record per run") - let summary = audit.records.first!.argumentsSummary - XCTAssertFalse(summary.contains("SUPERSECRETLONGVALUE12345"), "the full prompt / secret is never in the summary") - XCTAssertEqual(audit.records.first!.policy, .dangerous, "audited at the dangerous tier") - XCTAssertEqual(audit.records.first!.tool, "launch_claude") - } - - func testEveryBranchAuditsExactlyOnce() async { - // done - let a1 = RecordingAudit() - _ = await makeContributor(config: ClaudeHandoffConfig(confirmMode: .auto), audit: a1).run(call(), gate: ScriptedGate([])) - XCTAssertEqual(a1.count, 1) - // declined (skip) - let a2 = RecordingAudit() - _ = await makeContributor(config: ClaudeHandoffConfig(confirmMode: .confirm), audit: a2).run(call(), gate: ScriptedGate([.skip])) - XCTAssertEqual(a2.count, 1) - // missing folder (failed) - let a3 = RecordingAudit() - _ = await makeContributor(config: ClaudeHandoffConfig(confirmMode: .auto), audit: a3).run(call(folder: nil), gate: ScriptedGate([])) - XCTAssertEqual(a3.count, 1) - } - - func testCancelEndsQuietlyNotAFailure() async { - let launcher = FakeLauncher() - let c = makeContributor(config: ClaudeHandoffConfig(confirmMode: .confirm), launcher: launcher) - let result = await c.run(call(), gate: ScriptedGate([.cancel])) - if case let .declined(reason) = result.status { - XCTAssertEqual(reason, TaskKindToolContributor.cancelledReason, "cancel is a quiet decline, not a failure") - } else { XCTFail("cancel → declined(cancelled)") } - XCTAssertEqual(launcher.callCount, 0) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/ComputeLaneTests.swift b/Tests/ThreeFingerSwitcherTests/ComputeLaneTests.swift deleted file mode 100644 index 56cdbe6..0000000 --- a/Tests/ThreeFingerSwitcherTests/ComputeLaneTests.swift +++ /dev/null @@ -1,242 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for the pure-Core substrate of `ai-compute-tiers` (tasks §1–§4, §7): the two-lane model -/// (`ComputeLane`/`AgentWorkRole`), the role→lane policy (`DefaultLaneRouting`), the lane-affinity hint, -/// the co-residency budget + cross-lane arbiter, the OFF-coercion gate decorator, the deterministic -/// `StubTernaryRuntime`, and the error mapping → `RuntimeError`/`AIError.message(for:)`. -/// -/// The native-linked `TernaryCPURuntime` (§5), the lane-keyed wiring + additive dispatch (§6), and the -/// live two-lane concurrency (§8.3) are `xcodebuild` compile-verify / user-run-verify only. -final class ComputeLaneTests: XCTestCase { - - // MARK: - §1.1 ComputeLane / AgentWorkRole construct + Codable round-trip - - func testComputeLaneRoundTripsCodable() throws { - for lane in ComputeLane.allCases { - let data = try JSONEncoder().encode(lane) - let back = try JSONDecoder().decode(ComputeLane.self, from: data) - XCTAssertEqual(lane, back) - } - XCTAssertEqual(ComputeLane.allCases.count, 2) - } - - func testAgentWorkRoleRoundTripsCodable() throws { - for role in AgentWorkRole.allCases { - let data = try JSONEncoder().encode(role) - let back = try JSONDecoder().decode(AgentWorkRole.self, from: data) - XCTAssertEqual(role, back) - } - XCTAssertEqual(AgentWorkRole.allCases.count, 6) - } - - // MARK: - §1.2 DefaultLaneRouting — exhaustive role→lane (one assertion per case) - - func testDefaultLaneRoutingIsExhaustiveAndTotal() { - let routing = DefaultLaneRouting() - XCTAssertEqual(routing.lane(for: .foregroundGeneration), .gpu) - XCTAssertEqual(routing.lane(for: .mediaDiffusion), .gpu) - XCTAssertEqual(routing.lane(for: .toolRoute), .cpuTernary) - XCTAssertEqual(routing.lane(for: .classify), .cpuTernary) - XCTAssertEqual(routing.lane(for: .memoryRetrieval), .cpuTernary) - XCTAssertEqual(routing.lane(for: .parkedSubagent), .cpuTernary) - // Total: every defined role has a lane (the switch above is exhaustive by construction). - for role in AgentWorkRole.allCases { - _ = routing.lane(for: role) // no trap / no default-needed - } - } - - func testDefaultLaneRoutingIsDeterministic() { - let routing = DefaultLaneRouting() - for role in AgentWorkRole.allCases { - XCTAssertEqual(routing.lane(for: role), routing.lane(for: role)) - } - } - - // MARK: - §1.3 LaneAffinity derived from the role - - func testLaneAffinityFollowsWorkRole() { - let routing = DefaultLaneRouting() - let parked = AgentSessionID() - let foreground = AgentSessionID() - - let parkedAffinity = LaneAffinity(sessionID: parked, role: .parkedSubagent, routing: routing) - XCTAssertEqual(parkedAffinity.lane, .cpuTernary) - XCTAssertEqual(parkedAffinity.sessionID, parked) - - let fgAffinity = LaneAffinity(sessionID: foreground, role: .foregroundGeneration, routing: routing) - XCTAssertEqual(fgAffinity.lane, .gpu) - } - - // MARK: - §2.1 LaneResidencyBudget — ternary co-resides where a 2nd chat model would not - - func testTernaryCoResidesWhereSecondChatModelWouldNot() { - // ~17 GB chat weights resident; ternary ~32× smaller ≈ 0.53 GB; KV ~0.5 GB/stream. - let gb: Int64 = 1024 * 1024 * 1024 - let budget = LaneResidencyBudget( - chatWeightBytes: 17 * gb, - kvBytesPerGPUStream: gb / 2, - ternaryResidencyBytes: 17 * gb / 32, // ~0.53 GB - reservedBytes: 4 * gb - ) - // 36 GB of unified memory is committed to the AI feature here (a realistic working budget on a - // 48 GB machine after the OS + other apps). With chat (17) + 4 streams KV (2) + reserved (4) = - // 23 GB used, ~13 GB of headroom remains. - let freeBytes = 36 * gb - - // The small ternary model (~0.53 GB) fits in that ~13 GB headroom and co-resides. - XCTAssertTrue(budget.ternaryCoResides(freeBytes: freeBytes, gpuStreams: 4, contextTokens: 8192)) - - // A SECOND full chat model (17 GB) would NOT fit in that same ~13 GB headroom — the contrast that - // makes a second LANE a co-resident win rather than an eviction. - let secondChatBytes = 17 * gb - let remaining = freeBytes - budget.gpuResidentBytes(gpuStreams: 4) - budget.reservedBytes - XCTAssertGreaterThanOrEqual(remaining, budget.ternaryResidencyBytes, - "ternary fits in the remaining headroom") - XCTAssertLessThan(remaining, secondChatBytes, - "a second full chat model would NOT fit in that same headroom") - } - - func testTernaryResidencyBytesTogglesTheBoundary() { - let gb: Int64 = 1024 * 1024 * 1024 - var budget = LaneResidencyBudget(chatWeightBytes: 17 * gb, - kvBytesPerGPUStream: gb, - ternaryResidencyBytes: gb, // 1 GB - reservedBytes: 4 * gb) - // free 48, chat 17, 25 streams * 1 GB = 25, reserved 4 → 25 - 25 - 4 ... tune to a tight boundary. - let free: Int64 = 23 * gb // chat 17 + reserved 4 = 21 used at 0 streams → 2 GB remains. - XCTAssertTrue(budget.ternaryCoResides(freeBytes: free, gpuStreams: 0, contextTokens: 1024), - "1 GB ternary fits in 2 GB headroom") - // Toggle the ternary footprint UP past the headroom → it flips to NOT co-residing. - budget.ternaryResidencyBytes = 3 * gb - XCTAssertFalse(budget.ternaryCoResides(freeBytes: free, gpuStreams: 0, contextTokens: 1024), - "3 GB ternary does NOT fit in 2 GB headroom") - // Toggle back DOWN → co-resides again. - budget.ternaryResidencyBytes = gb / 2 - XCTAssertTrue(budget.ternaryCoResides(freeBytes: free, gpuStreams: 0, contextTokens: 1024)) - } - - // MARK: - §2.2 LaneArbiter — concurrent admit, GPU never waits, CPU cap bounds, waits not fails - - private func ampleBudget() -> LaneResidencyBudget { - let gb: Int64 = 1024 * 1024 * 1024 - return LaneResidencyBudget(chatWeightBytes: 17 * gb, - kvBytesPerGPUStream: gb / 2, - ternaryResidencyBytes: gb / 2, - reservedBytes: 4 * gb) - } - - func testArbiterAdmitsGPUAndCPUConcurrentlyInOneTick() { - let arbiter = LaneArbiter(cpuLaneCap: 2, budget: ampleBudget()) - let gpu = AgentSessionID() - let cpu = AgentSessionID() - let result = arbiter.admit(now: Date(), - freeBytes: 48 * 1024 * 1024 * 1024, - gpuStreams: 1, - contextTokens: 8192, - gpuRequests: [gpu], - cpuRequests: [cpu]) - XCTAssertEqual(result.admittedGPU, [gpu]) - XCTAssertEqual(result.admittedCPU, [cpu]) - XCTAssertTrue(result.waitingCPU.isEmpty) - } - - func testArbiterNeverDefersGPUBehindCPUWork() { - // Even when CPU work cannot co-reside (tiny free memory), the GPU lane is STILL admitted in full. - let arbiter = LaneArbiter(cpuLaneCap: 4, budget: ampleBudget()) - let gpu1 = AgentSessionID(); let gpu2 = AgentSessionID() - let cpu = AgentSessionID() - let result = arbiter.admit(now: Date(), - freeBytes: 1 * 1024 * 1024 * 1024, // 1 GB — nothing co-resides - gpuStreams: 1, - contextTokens: 8192, - gpuRequests: [gpu1, gpu2], - cpuRequests: [cpu]) - XCTAssertEqual(result.admittedGPU, [gpu1, gpu2], "GPU never waits on CPU work") - XCTAssertTrue(result.admittedCPU.isEmpty) - XCTAssertEqual(result.waitingCPU, [cpu], "the unadmittable CPU burst WAITS, it does not fail") - } - - func testArbiterCPUCapBoundsAndOverCapWaitsNotFails() { - let arbiter = LaneArbiter(cpuLaneCap: 2, budget: ampleBudget()) - let a = AgentSessionID(); let b = AgentSessionID(); let c = AgentSessionID() - let result = arbiter.admit(now: Date(), - freeBytes: 48 * 1024 * 1024 * 1024, - gpuStreams: 2, - contextTokens: 8192, - gpuRequests: [], - cpuRequests: [a, b, c]) - XCTAssertEqual(result.admittedCPU, [a, b], "bounded by the CPU lane's own cap") - XCTAssertEqual(result.waitingCPU, [c], "the over-cap burst WAITS (not a failure)") - } - - func testArbiterRespectsInflightCPU() { - let arbiter = LaneArbiter(cpuLaneCap: 2, budget: ampleBudget()) - let a = AgentSessionID() - let result = arbiter.admit(now: Date(), - freeBytes: 48 * 1024 * 1024 * 1024, - gpuStreams: 1, - contextTokens: 8192, - gpuRequests: [], - cpuRequests: [a], - inflightCPU: 2) // cap already full - XCTAssertTrue(result.admittedCPU.isEmpty) - XCTAssertEqual(result.waitingCPU, [a]) - } - - func testArbiterIsDeterministicForFixedInputs() { - let arbiter = LaneArbiter(cpuLaneCap: 3, budget: ampleBudget()) - let now = Date(timeIntervalSince1970: 1_000) - let gpu = [AgentSessionID()]; let cpu = [AgentSessionID(), AgentSessionID()] - let r1 = arbiter.admit(now: now, freeBytes: 48 * 1024 * 1024 * 1024, gpuStreams: 1, - contextTokens: 4096, gpuRequests: gpu, cpuRequests: cpu) - let r2 = arbiter.admit(now: now, freeBytes: 48 * 1024 * 1024 * 1024, gpuStreams: 1, - contextTokens: 4096, gpuRequests: gpu, cpuRequests: cpu) - XCTAssertEqual(r1, r2) - } - - // MARK: - §3.1 GatedLaneRouting OFF-coercion decorator - - func testGateOffCoercesEveryRoleToGPU() { - let gate = GatedLaneRouting(fullPotentialEnabled: false, cpuLaneEnabled: false) - for role in AgentWorkRole.allCases { - XCTAssertEqual(gate.lane(for: role), .gpu, "OFF → every role coerces to .gpu") - } - XCTAssertFalse(gate.cpuLaneActive) - } - - func testGateSubFlagOffStillCoercesToGPU() { - // Master ON but the CPU-lane sub-flag OFF → still one-lane. - let gate = GatedLaneRouting(fullPotentialEnabled: true, cpuLaneEnabled: false) - XCTAssertEqual(gate.lane(for: .toolRoute), .gpu) - XCTAssertFalse(gate.cpuLaneActive) - } - - func testGateMasterOffOverridesSubFlagOn() { - // Sub-flag ON but the MASTER OFF → still one-lane (a sub-flag never overrides the master OFF). - let gate = GatedLaneRouting(fullPotentialEnabled: false, cpuLaneEnabled: true) - XCTAssertEqual(gate.lane(for: .classify), .gpu) - XCTAssertFalse(gate.cpuLaneActive) - } - - func testGateOnPreservesDefaultMapping() { - let gate = GatedLaneRouting(fullPotentialEnabled: true, cpuLaneEnabled: true) - XCTAssertTrue(gate.cpuLaneActive) - XCTAssertEqual(gate.lane(for: .foregroundGeneration), .gpu) - XCTAssertEqual(gate.lane(for: .mediaDiffusion), .gpu) - XCTAssertEqual(gate.lane(for: .toolRoute), .cpuTernary) - XCTAssertEqual(gate.lane(for: .classify), .cpuTernary) - XCTAssertEqual(gate.lane(for: .memoryRetrieval), .cpuTernary) - XCTAssertEqual(gate.lane(for: .parkedSubagent), .cpuTernary) - } - - // MARK: - §5.3 media/vision never routes to the CPU lane (verified via the policy) - - func testMediaDiffusionKeepsMediaOffTheCPULane() { - let routing = DefaultLaneRouting() - XCTAssertEqual(routing.lane(for: .mediaDiffusion), .gpu) - // And under the gate ON, media is still GPU. - let gate = GatedLaneRouting(fullPotentialEnabled: true, cpuLaneEnabled: true) - XCTAssertEqual(gate.lane(for: .mediaDiffusion), .gpu) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/ComputerUseTests.swift b/Tests/ThreeFingerSwitcherTests/ComputerUseTests.swift deleted file mode 100644 index e1d2aee..0000000 --- a/Tests/ThreeFingerSwitcherTests/ComputerUseTests.swift +++ /dev/null @@ -1,241 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// `add-voice-computer-use-agent`: the pure AX snapshot builder + the constrained-ID epoch store, -/// the loop's wall-clock budgets, and the auto-approving gate — every rule deterministic, no live -/// AX, no real model. -@MainActor -final class ComputerUseTests: XCTestCase { - - // MARK: - AXSnapshotBuilder - - private func sampleTree() -> AXNodeData { - AXNodeData(role: "AXWindow", children: [ - AXNodeData(role: "AXStaticText", value: "Hello from the terminal"), - AXNodeData(role: "AXButton", label: "Send", isPressable: true), - AXNodeData(role: "AXGroup", children: [ - AXNodeData(role: "AXTextField", label: "Message", value: "draft", isSettable: true), - AXNodeData(role: "AXStaticText", value: "footer note"), - ]), - ]) - } - - func testBuilderExtractsTextAndElements() { - let snapshot = AXSnapshotBuilder.build(pid: 42, appName: "Terminal", title: "zsh", - root: sampleTree(), epoch: 0) - XCTAssertEqual(snapshot.textBlocks, ["Hello from the terminal", "draft", "footer note"]) - XCTAssertEqual(snapshot.elements.count, 2) - XCTAssertTrue(snapshot.elements.contains { $0.label == "Send" && $0.isPressable }) - XCTAssertTrue(snapshot.elements.contains { $0.label == "Message" && $0.isSettable }) - XCTAssertFalse(snapshot.truncated) - } - - func testStableIDsAreStableAcrossRereadsAndPathSensitive() { - let first = AXSnapshotBuilder.build(pid: 1, appName: "A", title: "t", root: sampleTree(), epoch: 0) - let second = AXSnapshotBuilder.build(pid: 1, appName: "A", title: "t", root: sampleTree(), epoch: 0) - XCTAssertEqual(first.elements.map(\.id), second.elements.map(\.id), - "an unchanged window re-reads to the same ids") - // A moved element (different path) gets a DIFFERENT id — the staleness signal. - var moved = sampleTree() - moved.children.swapAt(0, 1) - let third = AXSnapshotBuilder.build(pid: 1, appName: "A", title: "t", root: moved, epoch: 0) - XCTAssertNotEqual(first.elements.map(\.id), third.elements.map(\.id)) - } - - func testDepthAndCountLimitsReportTruncationHonestly() { - // A deep chain past maxDepth. - var deep = AXNodeData(role: "AXStaticText", value: "leaf") - for _ in 0..<20 { deep = AXNodeData(role: "AXGroup", children: [deep]) } - let snapshot = AXSnapshotBuilder.build(pid: 1, appName: "A", title: "t", root: deep, epoch: 0, - limits: .init(maxDepth: 5, maxNodes: 100, maxTextBlocks: 10)) - XCTAssertTrue(snapshot.truncated, "a depth cut must be reported, never silent") - } - - func testContentHashChangesWithContent() { - let before = AXSnapshotBuilder.build(pid: 1, appName: "A", title: "t", root: sampleTree(), epoch: 0) - var changedTree = sampleTree() - changedTree.children[0].value = "Hello CHANGED" - let after = AXSnapshotBuilder.build(pid: 1, appName: "A", title: "t", root: changedTree, epoch: 0) - XCTAssertNotEqual(before.contentHash, after.contentHash) - } - - // MARK: - AXSnapshotStore (the constrained-ID epoch) - - func testStoreResolvesOnlyCurrentEpoch() throws { - let store = AXSnapshotStore() - let first = store.register(AXSnapshotBuilder.build(pid: 7, appName: "A", title: "t", - root: sampleTree(), epoch: 0)) - let sendID = try XCTUnwrap(first.elements.first(where: { $0.label == "Send" })?.id) - XCTAssertNoThrow(try store.resolve(sendID, pid: 7)) - - // A NEW snapshot with a changed layout replaces the epoch: the old id is now stale. - var moved = sampleTree() - moved.children.swapAt(0, 1) - _ = store.register(AXSnapshotBuilder.build(pid: 7, appName: "A", title: "t", - root: moved, epoch: 0)) - XCTAssertThrowsError(try store.resolve(sendID, pid: 7)) { error in - XCTAssertEqual(error as? AXActionError, .staleElement) - } - // An unknown pid is stale too (never a guess). - XCTAssertThrowsError(try store.resolve(sendID, pid: 999)) - } - - // MARK: - LoopBudget in AgentLoop - - /// A contributor whose single tool sleeps forever (cancellation-safe) — the step-timeout victim. - private struct HangingContributor: ToolContributor { - func descriptors() -> [ToolDescriptor] { - [ToolDescriptor(name: "hang_forever", summary: "hangs", - argsSchema: StructuredSchema(name: "hang", json: "{\"type\":\"object\"}"), - writePolicy: .auto, keywords: ["hang"])] - } - func canHandle(_ tool: String) -> Bool { tool == "hang_forever" } - func run(_ call: RoutedCall, gate: ApprovalGate) async -> ToolStepResult { - while !Task.isCancelled { - try? await Task.sleep(nanoseconds: 20_000_000) - } - return ToolStepResult(tool: call.descriptor.name, status: .done, summary: "woke up") - } - } - - private struct ScriptedGate: ApprovalGate { - func awaitDecision(for review: TaskReview) async -> ApprovalDecision { .approve } - } - - func testHungToolStepTimesOutAsCleanBudgetFailure() async { - // Route script: the model picks the hanging tool once. - let stub = StubLLMRuntime(capabilities: [.text]) - stub.structuredScript = .valid(json: #"{"tool":"hang_forever","argumentsJSON":"{}"}"#) - stub.scriptedTokens = ["fallback answer"] - let registry = ToolRegistry([HangingContributor()]) - let loop = AgentLoop(runtime: stub, registry: registry, - candidateSource: KeywordToolCandidateSource(all: { registry.allDescriptors() }), - gate: ScriptedGate(), - budget: LoopBudget(stepTimeout: 0.15, turnDeadline: 60)) - let result = await loop.run(context: RouteContext(messages: [ - AgentMessage(role: .user, text: "hang please")])) - // The timed-out step is the terminal failure — a clean budget headline. - guard case let .failed(headline) = result.outcome else { - return XCTFail("expected .failed, got \(result.outcome)") - } - XCTAssertTrue(headline.contains("timed out"), "budget failure, not a network headline: \(headline)") - XCTAssertEqual(result.steps.count, 1) - } - - func testTurnDeadlineTerminatesViaCapFallback() async { - let stub = StubLLMRuntime(capabilities: [.text]) - stub.scriptedTokens = ["partial summary"] - let registry = ToolRegistry([HangingContributor()]) - // A stepping clock: the FIRST read (the loop's deadline baseline) is t0; every later read is - // 11s past it — so the deadline (10s) trips at the first between-steps check. - let ticks = LockedCounter() - let t0 = Date(timeIntervalSinceReferenceDate: 0) - let loop = AgentLoop(runtime: stub, registry: registry, - candidateSource: KeywordToolCandidateSource(all: { registry.allDescriptors() }), - gate: ScriptedGate(), - budget: LoopBudget(stepTimeout: 30, turnDeadline: 10), - clock: { ticks.next() == 0 ? t0 : t0.addingTimeInterval(11) }) - let result = await loop.run(context: RouteContext(messages: [ - AgentMessage(role: .user, text: "anything")])) - guard case .capReached = result.outcome else { - return XCTFail("expected .capReached via the deadline, got \(result.outcome)") - } - } - - // MARK: - AutoApprovingGate - - private final class CountingGate: ApprovalGate, @unchecked Sendable { - private let lock = NSLock() - private var count = 0 - var decisions: Int { lock.lock(); defer { lock.unlock() }; return count } - func awaitDecision(for review: TaskReview) async -> ApprovalDecision { - lock.lock(); count += 1; lock.unlock() - return .skip - } - } - - func testAutoGateApprovesInstantlyAndNarratesUnderGrant() async { - let base = CountingGate() - let narrated = LockedStrings() - let grant = LockedBool(true) - let gate = AutoApprovingGate(base: base, - isGranted: { grant.value }, - narrate: { narrated.append($0) }) - let review = TaskReview.action(title: "Type in Chrome", - fields: [ReviewField("Text", "hello")], - payload: .openTool(tool: "t", action: ParsedOpenTool(applicable: true, reason: nil, payload: ""))) - let decision = await gate.awaitDecision(for: review) - XCTAssertEqual(decision, .approve) - XCTAssertEqual(base.decisions, 0, "the base gate is never consulted under the grant") - XCTAssertEqual(narrated.values.count, 1, "auto-approved acts are narrated — never silent") - XCTAssertTrue(narrated.values[0].contains("Type in Chrome")) - - // Revoked → transparent pass-through. - grant.value = false - let second = await gate.awaitDecision(for: review) - XCTAssertEqual(second, .skip) - XCTAssertEqual(base.decisions, 1) - } - - // MARK: - Contributor flag gating - - func testComputerUseToolsAbsentWhenDisabled() async { - let arbiter = AgentActionArbiter() - let contributor = ComputerUseToolContributor( - enabled: { false }, - resolveWindow: { _, _ in nil }, - focusWindow: { _ in false }, - performer: AXActionPerformer(eventSource: nil), - arbiter: arbiter, - narrate: { _ in }) - XCTAssertTrue(contributor.descriptors().isEmpty, "off means ABSENT from candidates") - } - - func testComputerUseSchemasHaveNoCoordinates() { - let arbiter = AgentActionArbiter() - let contributor = ComputerUseToolContributor( - enabled: { true }, - resolveWindow: { _, _ in nil }, - focusWindow: { _ in false }, - performer: AXActionPerformer(eventSource: nil), - arbiter: arbiter, - narrate: { _ in }) - for descriptor in contributor.descriptors() { - let json = descriptor.argsSchema.json.lowercased() - XCTAssertFalse(json.contains("\"x\"") || json.contains("\"y\"") || json.contains("coordinate"), - "\(descriptor.name) must expose no coordinate surface") - } - } - - // MARK: - Arbiter - - func testArbiterAbortFiresOnceWhileActingOnly() async { - let arbiter = AgentActionArbiter() - var aborts = 0 - arbiter.onAbort = { aborts += 1 } - arbiter.humanTouchDetected() - XCTAssertEqual(aborts, 0, "touch outside an acting scope is a normal gesture") - _ = await arbiter.acting { - arbiter.humanTouchDetected() - arbiter.humanTouchDetected() // debounced within one act - return true - } - XCTAssertEqual(aborts, 1) - XCTAssertFalse(arbiter.isActing) - } -} - -/// Lock-guarded fixtures for @Sendable capture in tests. -private final class LockedStrings: @unchecked Sendable { - private let lock = NSLock() - private var stored: [String] = [] - var values: [String] { lock.lock(); defer { lock.unlock() }; return stored } - func append(_ value: String) { lock.lock(); stored.append(value); lock.unlock() } -} - -private final class LockedCounter: @unchecked Sendable { - private let lock = NSLock() - private var count = -1 - /// Returns 0 on the first call, 1, 2, … after. - func next() -> Int { lock.lock(); defer { lock.unlock() }; count += 1; return count } -} diff --git a/Tests/ThreeFingerSwitcherTests/ConversationCompactionTests.swift b/Tests/ThreeFingerSwitcherTests/ConversationCompactionTests.swift deleted file mode 100644 index 1a70eec..0000000 --- a/Tests/ThreeFingerSwitcherTests/ConversationCompactionTests.swift +++ /dev/null @@ -1,131 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for context compaction (`ai-conversation-runtime`, tasks §4) — the pure windowing decision -/// (`needsCompaction`/`plan`/`applied`) against an injected fixed budget, the summarization model call, -/// the load-bearing guarantee that thinking never enters the summary input, and the end-to-end shape: -/// a compacted conversation assembles with the summary as a `system` prefix and the dropped raw turns -/// gone. -final class ConversationCompactionTests: XCTestCase { - - /// A fixed-budget provider so the windowing decision is deterministic (integration fix C3: the - /// executor reads the budget through this seam, never a concrete user slider). - private struct FixedBudget: ContextBudgetProviding { - let maxContextTokens: Int - } - - private func conversation(_ messages: [AgentMessage], summary: String? = nil) -> AgentConversation { - AgentConversation(title: "t", messages: messages, compactedSummary: summary) - } - - // MARK: - TokenEstimator - - func testEstimateGrowsWithContentAndIgnoresEmpty() { - XCTAssertEqual(TokenEstimator.estimate(""), 0, "empty text costs nothing") - let short = TokenEstimator.estimate("hello") - let long = TokenEstimator.estimate(String(repeating: "hello ", count: 100)) - XCTAssertGreaterThan(long, short, "the estimate grows with content length") - // Estimates committed text only — a message's thinking does not affect the estimate. - let withThinking = AgentMessage(role: .assistant, text: "hi", thinking: String(repeating: "x", count: 1000)) - XCTAssertEqual(TokenEstimator.estimate([withThinking]), TokenEstimator.estimate("hi"), - "the estimator reads committed text only, never thinking") - } - - // MARK: - needsCompaction (injected budget) - - func testNeedsCompactionTrueOnlyOverTheMarginAdjustedBudget() { - let small = conversation([AgentMessage(role: .user, text: "short")]) - XCTAssertFalse(ConversationCompactor.needsCompaction(small, budget: FixedBudget(maxContextTokens: 1000)), - "a short thread under budget does not compact") - - let bigText = String(repeating: "word ", count: 500) // ~625 estimated tokens - let big = conversation([AgentMessage(role: .user, text: bigText)]) - XCTAssertTrue(ConversationCompactor.needsCompaction(big, budget: FixedBudget(maxContextTokens: 100)), - "a thread whose estimate crosses the margin-adjusted budget compacts") - } - - // MARK: - plan: keep recent N, collapse the older prefix (incl. a prior summary) - - func testPlanKeepsRecentTurnsAndFoldsPriorSummary() { - let msgs = (0..<10).map { AgentMessage(role: $0 % 2 == 0 ? .user : .assistant, text: "m\($0)") } - let convo = conversation(msgs, summary: "PRIOR-SUMMARY") - let plan = ConversationCompactor.plan(convo, keepRecentTurns: 3) - - XCTAssertEqual(plan.keptTail.map(\.text), ["m7", "m8", "m9"], "the most recent 3 are kept verbatim") - XCTAssertEqual(plan.toSummarize.map(\.text), (0..<7).map { "m\($0)" }, "the older 7 are collapsed") - XCTAssertEqual(plan.priorSummary, "PRIOR-SUMMARY", "the prior summary is folded into the new summary's input") - XCTAssertFalse(plan.isEmpty) - } - - func testPlanIsEmptyWhenNothingOlderThanTheKeptTail() { - let msgs = [AgentMessage(role: .user, text: "a"), AgentMessage(role: .assistant, text: "b")] - let plan = ConversationCompactor.plan(conversation(msgs), keepRecentTurns: 6) - XCTAssertTrue(plan.isEmpty, "nothing to collapse when the thread is at/below the kept-tail size") - } - - // MARK: - summarize: a model call over committed text only - - func testSummarizeProducesScriptedSummary() async throws { - let runtime = StubLLMRuntime(scriptedTokens: ["A concise summary."], interTokenDelayNanos: 0) - let plan = ConversationCompactor.plan( - conversation((0..<8).map { AgentMessage(role: .user, text: "m\($0)") }), - keepRecentTurns: 2) - let summary = try await ConversationCompactor.summarize(plan, runtime: runtime) - XCTAssertEqual(summary, "A concise summary.", "summarize returns the model's response") - } - - func testSummarizeInputExcludesThinking() async throws { - // An echoing stub (empty scriptedTokens) returns its prompt as the "summary", so we can inspect - // exactly what was fed to the model. - let runtime = StubLLMRuntime(scriptedTokens: [], interTokenDelayNanos: 0) - let toSummarize = [ - AgentMessage(role: .user, text: "VISIBLE-USER-TEXT"), - AgentMessage(role: .assistant, text: "VISIBLE-ANSWER", thinking: "SECRET-REASONING"), - ] - let plan = CompactionPlan(toSummarize: toSummarize, keptTail: [], priorSummary: "OLD-SUMMARY") - let echoedPrompt = try await ConversationCompactor.summarize(plan, runtime: runtime) - - XCTAssertTrue(echoedPrompt.contains("VISIBLE-USER-TEXT"), "committed user text is summarized") - XCTAssertTrue(echoedPrompt.contains("VISIBLE-ANSWER"), "committed answer text is summarized") - XCTAssertTrue(echoedPrompt.contains("OLD-SUMMARY"), "a prior summary is folded into the input") - XCTAssertFalse(echoedPrompt.contains("SECRET-REASONING"), - "thinking is NEVER part of the summarization input") - } - - func testSummarizePropagatesRuntimeError() async { - let runtime = StubLLMRuntime(scriptedTurns: [.init(error: .serverUnavailable)], interTokenDelayNanos: 0) - let plan = CompactionPlan(toSummarize: [AgentMessage(role: .user, text: "x")], keptTail: [], priorSummary: nil) - do { - _ = try await ConversationCompactor.summarize(plan, runtime: runtime) - XCTFail("summarize should propagate the runtime error so the caller can fail the turn") - } catch let error as RuntimeError { - XCTAssertEqual(error, .serverUnavailable) - } catch { - XCTFail("expected a RuntimeError, got \(error)") - } - } - - // MARK: - applied + assembly: the summary becomes a system prefix, dropped turns are gone - - func testAppliedReplacesDroppedTurnsAndAssemblesAsSystemPrefix() { - let msgs = (0..<8).map { AgentMessage(role: $0 % 2 == 0 ? .user : .assistant, text: "OLD\($0)") } - + [AgentMessage(role: .user, text: "RECENT")] - let convo = conversation(msgs) - let plan = ConversationCompactor.plan(convo, keepRecentTurns: 1) - let compacted = ConversationCompactor.applied(plan, summary: "THE-SUMMARY", to: convo, - now: Date(timeIntervalSince1970: 5)) - - XCTAssertEqual(compacted.messages.map(\.text), ["RECENT"], "only the recent tail survives in messages") - XCTAssertEqual(compacted.compactedSummary, "THE-SUMMARY", "the summary becomes the prefix") - - // Assemble the way the executor does: prefix the summary as a synthetic system message. - var assembled: [AgentMessage] = [] - if let s = compacted.compactedSummary { assembled.append(AgentMessage(role: .system, text: s)) } - assembled.append(contentsOf: compacted.messages) - let prompt = ChatTemplate.flatten(assembled) - - XCTAssertTrue(prompt.contains("System: THE-SUMMARY"), "the summary rides as a system prefix") - XCTAssertTrue(prompt.contains("User: RECENT"), "the kept tail is present") - XCTAssertFalse(prompt.contains("OLD0"), "the dropped raw turns are gone from the assembled context") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/ConversationSessionTests.swift b/Tests/ThreeFingerSwitcherTests/ConversationSessionTests.swift deleted file mode 100644 index e411c0f..0000000 --- a/Tests/ThreeFingerSwitcherTests/ConversationSessionTests.swift +++ /dev/null @@ -1,520 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for the notch-session multi-turn engine (`notch-native-conversations` D1 — the executor's -/// conversational half re-homed into `NotchSessionEngine`): a new session's first typed turn streams, a -/// later turn sees the earlier turns, thinking streams live + is stored for display but NEVER re-fed, a -/// per-turn error is observable, and a mid-turn discard leaves no partial message and is not a failure. -/// The one-shot preset path is covered by `AICommandExecutorTests` and is untouched here. -@MainActor -final class ConversationSessionTests: XCTestCase { - - // MARK: - Harness - - private final class FakeSelectionProvider: SelectionProviding { - func readSelectedText() async -> String? { nil } - func readClipboardText() -> String? { nil } - func readClipboardImage() -> Data? { nil } - @discardableResult func replaceSelection(_ text: String) async -> Bool { true } - @discardableResult func pasteAtCursor(_ text: String) async -> Bool { true } - } - - private final class FakeDownloader: ModelDownloading, @unchecked Sendable { - let payload: Data - init(payload: Data) { self.payload = payload } - func download(_ descriptor: ModelDescriptor, to destination: URL, - progress: @Sendable (Double) -> Void) async throws -> Data { - progress(1.0); return payload - } - } - - private func loadedManager(runtime: LLMRuntime, - capabilities: Set = [.text, .vision]) async throws -> ModelManager { - let payload = Data("weights".utf8) - let registry = ModelCatalog( - models: [ModelDescriptor(id: "test-model", displayName: "Test Model", - sizeBytes: Int64(payload.count), - integritySHA: ModelManager.sha256Hex(payload), - downloadURL: URL(string: "https://models.invalid/test-model")!, - capabilities: capabilities, quantization: .qat4bit)], - defaultModelID: "test-model") - let root = URL(fileURLWithPath: NSTemporaryDirectory()) - .appendingPathComponent("tfs-conversation-tests-\(UUID().uuidString)", isDirectory: true) - let manager = ModelManager(registry: registry, downloader: FakeDownloader(payload: payload), - optedIn: true, storageRoot: root, runtimeFactory: { _ in runtime }) - try await manager.downloadAndVerify(registry.models[0]) - return manager - } - - private func makeEngine(_ manager: ModelManager, - reasoning: Bool = false, - budget: ContextBudgetProviding = DefaultContextBudget()) -> NotchSessionEngine { - NotchSessionEngine(modelManager: manager, selection: FakeSelectionProvider(), - reasoning: { reasoning }, budgetProvider: budget) - } - - private func waitUntil(_ predicate: @MainActor () -> Bool, timeout: TimeInterval = 2.0, - file: StaticString = #filePath, line: UInt = #line) async { - let deadline = Date().addingTimeInterval(timeout) - while !predicate() && Date() < deadline { - try? await Task.sleep(nanoseconds: 2_000_000) - } - XCTAssertTrue(predicate(), "condition not met within \(timeout)s", file: file, line: line) - } - - // MARK: - The first typed turn opens the thread (and names the session) - - func testFirstSendStreamsFirstTurnAndNamesTheSession() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["Hello ", "there"], interTokenDelayNanos: 0) - let engine = makeEngine(try await loadedManager(runtime: stub)) - - let born = engine.startNew() - XCTAssertEqual(born.title, "New chat", "a fresh session carries the placeholder title") - XCTAssertEqual(engine.state, .awaitingTurn, "a new session idles awaiting its first typed turn") - - engine.send("hi") - await waitUntil { engine.state == .awaitingTurn && engine.conversation?.messages.count == 2 } - - let convo = try XCTUnwrap(engine.conversation) - XCTAssertEqual(convo.messages.map(\.role), [.user, .assistant]) - XCTAssertEqual(convo.messages[0].text, "hi", "the typed message is turn 1 (a user message)") - XCTAssertEqual(convo.messages[1].text, "Hello there", "the assistant turn carries the response") - XCTAssertEqual(convo.title, "hi", "the first turn names the session (was the placeholder)") - XCTAssertEqual(convo.id, born.id, "the session identity is stable from birth") - } - - func testEmptySendIsANoOp() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["SHOULD NOT APPEAR"], interTokenDelayNanos: 0) - let engine = makeEngine(try await loadedManager(runtime: stub)) - - engine.startNew() - engine.send(" ") // whitespace-only, no images - try? await Task.sleep(nanoseconds: 30_000_000) - XCTAssertEqual(engine.state, .awaitingTurn, "an empty send never runs the model") - XCTAssertEqual(engine.conversation?.messages.count, 0, "no message is appended for an empty send") - } - - func testSendWithImageCarriesItOnTheTurn() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["ok"], interTokenDelayNanos: 0) - let engine = makeEngine(try await loadedManager(runtime: stub)) - let png = Data([0x89, 0x50, 0x4E, 0x47, 0x0A]) - - engine.startNew() - engine.send("what is this?", images: [png]) - await waitUntil { engine.state == .awaitingTurn && engine.conversation?.messages.count == 2 } - - let convo = try XCTUnwrap(engine.conversation) - XCTAssertEqual(convo.messages.first?.images, [png], "the turn's image rides on the user message") - XCTAssertEqual(engine.assembleRequest()?.effectiveImage, png, "assembly carries the latest image") - } - - // MARK: - A later turn sees earlier turns; thinking is never re-fed - - func testSecondTurnSeesFirstTurnAndThinkingNeverReFed() async throws { - let stub = StubLLMRuntime( - scriptedTurns: [.init(tokens: ["A1"], thinking: ["SECRET-REASONING"]), - .init(tokens: ["A2"])], - interTokenDelayNanos: 0) - let engine = makeEngine(try await loadedManager(runtime: stub), reasoning: true) - - engine.startNew() - engine.send("U1") - await waitUntil { engine.state == .awaitingTurn && engine.conversation?.messages.count == 2 } - XCTAssertEqual(engine.conversation?.messages[1].text, "A1") - XCTAssertEqual(engine.conversation?.messages[1].thinking, "SECRET-REASONING", - "the turn's reasoning is stored on the message for display") - XCTAssertEqual(engine.thinking, "SECRET-REASONING", "thinking streamed live during the turn") - - // The assembled context for the NEXT turn sees turn 1's text — but never its thinking. - let request = try XCTUnwrap(engine.assembleRequest()) - let assembled = ChatTemplate.flatten(request.messages) - XCTAssertTrue(assembled.contains("U1"), "the later turn's context contains the earlier user turn") - XCTAssertTrue(assembled.contains("A1"), "the later turn's context contains the earlier assistant turn") - XCTAssertFalse(assembled.contains("SECRET-REASONING"), - "no prior turn's thinking appears in the assembled context") - - // Send turn 2. - engine.send("U2") - await waitUntil { engine.conversation?.messages.count == 4 } - XCTAssertEqual(engine.conversation?.messages.map(\.text), ["U1", "A1", "U2", "A2"]) - XCTAssertEqual(engine.state, .awaitingTurn) - XCTAssertEqual(engine.thinking, "", - "live thinking is reset at each turn start; turn 2 had none, so it shows nothing") - } - - // MARK: - Every settled turn reports a durable snapshot (the collapse-mid-turn contract's seam) - - func testTurnSettledReportsSnapshotWithTheAppendedTurn() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["A1"], interTokenDelayNanos: 0) - let engine = makeEngine(try await loadedManager(runtime: stub)) - var settled: [(AgentConversation, NotchSessionEngine.TurnSettlement)] = [] - engine.onTurnSettled = { conversation, settlement in settled.append((conversation, settlement)) } - - engine.startNew() - engine.send("U1") - await waitUntil { engine.state == .awaitingTurn && engine.conversation?.messages.count == 2 } - - XCTAssertEqual(settled.count, 1, "exactly one settle per completed turn") - XCTAssertEqual(settled.first?.0.messages.map(\.text), ["U1", "A1"], - "the settled snapshot carries the appended assistant turn") - XCTAssertEqual(settled.first?.1, .answered, - "a settled chat turn is an ANSWER — there is no terminal task-completion notion") - } - - // MARK: - Per-turn failure is observable, history is not dropped - - func testPerTurnErrorSurfacesFailedWithoutDroppingHistory() async throws { - let stub = StubLLMRuntime( - scriptedTurns: [.init(tokens: ["A1"]), .init(error: .serverUnavailable)], - interTokenDelayNanos: 0) - let engine = makeEngine(try await loadedManager(runtime: stub)) - - engine.startNew() - engine.send("U1") - await waitUntil { engine.state == .awaitingTurn && engine.conversation?.messages.count == 2 } - - engine.send("U2") - await waitUntil { if case .failed = engine.state { return true }; return false } - - guard case let .failed(message) = engine.state else { return XCTFail("expected .failed") } - XCTAssertEqual(message, RuntimeError.serverUnavailable.errorDescription, - "the failure carries the clean translated headline, not a raw dump") - XCTAssertEqual(engine.conversation?.messages.map(\.text), ["U1", "A1", "U2"], - "the user turn stays; no partial assistant turn is appended; history is not dropped") - } - - // MARK: - Mid-turn discard leaves no partial message and is not a failure - - func testDiscardTurnLeavesNoPartialAndIsNotAFailure() async throws { - let stub = StubLLMRuntime( - scriptedTurns: [.init(tokens: ["A1"]), - .init(tokens: Array(repeating: "x", count: 50))], - interTokenDelayNanos: 5_000_000) // 5 ms/token so turn 2 streams long enough to discard - let engine = makeEngine(try await loadedManager(runtime: stub)) - - engine.startNew() - engine.send("U1") - await waitUntil { engine.state == .awaitingTurn && engine.conversation?.messages.count == 2 } - - engine.send("U2") - await waitUntil { if case .conversing = engine.state { return true }; return false } - engine.discardTurn() - - XCTAssertEqual(engine.state, .awaitingTurn, "a discarded turn returns the thread to idle, not failed") - XCTAssertEqual(engine.conversation?.messages.map(\.text), ["U1", "A1", "U2"], - "no partial assistant message is appended for the discarded turn") - await waitUntil { stub.observedCancellation } - XCTAssertTrue(stub.observedCancellation, "generation was actually cancelled") - } - - // MARK: - Bind / unbind (the expand / collapse verbs' engine half) - - func testBindIdlesAwaitingTurnAndUnbindReturnsTheSnapshot() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["A1"], interTokenDelayNanos: 0) - let engine = makeEngine(try await loadedManager(runtime: stub)) - - let now = Date() - let stored = AgentConversation( - title: "Stored", - messages: [AgentMessage(role: .user, text: "old", createdAt: now), - AgentMessage(role: .assistant, text: "answer", createdAt: now)], - createdAt: now, updatedAt: now) - engine.bind(stored) - XCTAssertEqual(engine.state, .awaitingTurn, "a bound session idles awaiting the next turn") - XCTAssertEqual(engine.conversation?.id, stored.id) - - let snapshot = engine.unbind() - XCTAssertEqual(snapshot?.id, stored.id, "unbind hands the snapshot back for persistence") - XCTAssertEqual(engine.state, .idle, "an idle unbind fully resets the engine") - XCTAssertNil(engine.conversation) - } - - func testUnbindMidTurnKeepsTheTurnRunningToSettlement() async throws { - let stub = StubLLMRuntime( - scriptedTokens: Array(repeating: "x", count: 40), - interTokenDelayNanos: 5_000_000) - let engine = makeEngine(try await loadedManager(runtime: stub)) - var settled: [(AgentConversation, NotchSessionEngine.TurnSettlement)] = [] - engine.onTurnSettled = { conversation, settlement in settled.append((conversation, settlement)) } - - engine.startNew() - engine.send("U1") - await waitUntil { if case .conversing = engine.state { return true }; return false } - - let snapshot = engine.unbind() // collapse mid-turn: does NOT cancel - XCTAssertNotNil(snapshot, "collapse still gets a snapshot to persist") - XCTAssertTrue(engine.isTurnInFlight, "the in-flight turn survives the unbind (collapse ≠ cancel)") - - await waitUntil({ !settled.isEmpty }, timeout: 4.0) - XCTAssertEqual(settled.first?.0.messages.count, 2, - "the detached turn ran to completion and reported its settled snapshot") - XCTAssertFalse(stub.observedCancellation, "the turn was never cancelled by the collapse") - } - - // MARK: - Routed turns (registry-wired — the production path the old suite never exercised, - // which is exactly how the detached-settle deletion bug stayed invisible) - - /// A scripted routing runtime: `structured()` dequeues route JSON; `generate()` streams the answer - /// (thinking chunks first, in order). - private final class ScriptedRoutingRuntime: LLMRuntime, @unchecked Sendable { - let capabilities: Set = [.text] - private var routes: [String] - private let answerTokens: [String] - private let thinkingChunks: [String] - private let lock = NSLock() - init(routes: [String], answer: [String], thinking: [String] = []) { - self.routes = routes; self.answerTokens = answer; self.thinkingChunks = thinking - } - func generate(_ request: LLMRequest) -> AsyncThrowingStream { - let thinks = thinkingChunks, toks = answerTokens - return AsyncThrowingStream { c in - for t in thinks { c.yield(Token(t, isFinal: false, channel: .thinking)) } - for (i, t) in toks.enumerated() { c.yield(Token(t, isFinal: i == toks.count - 1)) } - c.finish() - } - } - func structured(_ r: LLMRequest, schema: StructuredSchema, - as type: T.Type) async throws -> StructuredOutcome { - let json: String = { - lock.lock(); defer { lock.unlock() } - return routes.isEmpty ? "{\"tool\":\"\"}" : routes.removeFirst() - }() - return .value(try JSONDecoder().decode(T.self, from: Data(json.utf8))) - } - } - - /// A confirm-tier tool whose run awaits the injected gate — the foreground approval path. - private final class GatedContributor: ToolContributor, @unchecked Sendable { - let descriptor: ToolDescriptor - private(set) var ran = false - init(name: String) { - descriptor = ToolDescriptor(name: name, summary: "s", - argsSchema: StructuredSchema(name: name, json: "{\"type\":\"object\"}"), - writePolicy: .confirm) - } - func descriptors() -> [ToolDescriptor] { [descriptor] } - func canHandle(_ tool: String) -> Bool { tool == descriptor.name } - func run(_ call: RoutedCall, gate: ApprovalGate) async -> ToolStepResult { - switch await gate.awaitDecision(for: .unavailable(reason: "approve me")) { - case .approve: - ran = true - return ToolStepResult(tool: descriptor.name, status: .done, summary: "did it") - case .skip: - return ToolStepResult(tool: descriptor.name, status: .declined(reason: "skipped"), summary: "skipped") - case .cancel: - return ToolStepResult(tool: descriptor.name, - status: .declined(reason: TaskKindToolContributor.cancelledReason), - summary: "cancelled") - } - } - } - - private func makeRoutedEngine(_ manager: ModelManager, registry: ToolRegistry) -> NotchSessionEngine { - NotchSessionEngine(modelManager: manager, selection: FakeSelectionProvider(), - registry: registry, - candidateSource: KeywordToolCandidateSource(all: { registry.allDescriptors() })) - } - - func testRoutedPlainAnswerSettlesAnsweredWithOrderedThinking() async throws { - let rt = ScriptedRoutingRuntime(routes: ["{\"tool\":\"\"}"], answer: ["A1"], - thinking: ["T1", "T2", "T3"]) - let engine = makeRoutedEngine(try await loadedManager(runtime: rt), registry: ToolRegistry([])) - var settled: [(AgentConversation, NotchSessionEngine.TurnSettlement)] = [] - engine.onTurnSettled = { settled.append(($0, $1)) } - - engine.startNew() - engine.send("U1") - await waitUntil { engine.state == .awaitingTurn && engine.conversation?.messages.count == 2 } - - XCTAssertEqual(settled.map(\.1), [.answered], - "a routed answer settles as ANSWERED — never a terminal 'task complete'") - XCTAssertEqual(settled.first?.0.messages.last?.text, "A1") - XCTAssertEqual(engine.thinking, "T1T2T3", - "reasoning tokens land in order — the stream is turn-owned, drained before settle") - } - - func testDockedApprovalSurvivesAndApproveResumesTheSameStep() async throws { - let tool = GatedContributor(name: "confirm_tool") - let rt = ScriptedRoutingRuntime(routes: ["{\"tool\":\"confirm_tool\"}", "{\"tool\":\"\"}"], - answer: ["done"]) - let engine = makeRoutedEngine(try await loadedManager(runtime: rt), registry: ToolRegistry([tool])) - var settled: [(AgentConversation, NotchSessionEngine.TurnSettlement)] = [] - engine.onTurnSettled = { settled.append(($0, $1)) } - - engine.startNew() - engine.send("go") - await waitUntil { engine.isPausedAtApproval } - XCTAssertTrue(engine.isTurnInFlight, - "a paused approval counts as in flight — docking must keep the engine") - - let snapshot = engine.unbind() // dock while paused at the gate - XCTAssertNotNil(snapshot, "the dock still gets a snapshot to persist") - XCTAssertTrue(engine.isTurnInFlight, "the suspended gate survives the dock (no orphaned continuation)") - XCTAssertNotNil(engine.conversation, "the conversation stays bound while the turn is in flight") - - XCTAssertTrue(engine.approve(), "re-expanding re-presents the SAME pending step; approve resumes it") - await waitUntil { !settled.isEmpty } - XCTAssertTrue(tool.ran, "the approved step actually fired — the turn was never restarted") - XCTAssertEqual(settled.map(\.1), [.answered]) - XCTAssertEqual(settled.first?.0.messages.last?.text, "done") - } - - // MARK: - advance() (the background driver's verb) - - // MARK: - The timeline (`notch-timeline-and-tuning`): interleaved segments, streamed + persisted - - /// A runtime whose `generate` yields an arbitrary CHANNEL SCRIPT — interleaved thinking/response - /// chunks in exact order (the stub emits thinking-then-response only, which can't exercise a flip - /// back). `structured()` dequeues route JSON so the same type drives routed turns. - private final class ChannelScriptRuntime: LLMRuntime, @unchecked Sendable { - let capabilities: Set = [.text] - private var routes: [String] - private let script: [(TokenChannel, String)] - private let lock = NSLock() - init(script: [(TokenChannel, String)], routes: [String] = []) { - self.script = script; self.routes = routes - } - func generate(_ request: LLMRequest) -> AsyncThrowingStream { - let script = self.script - return AsyncThrowingStream { c in - for (i, entry) in script.enumerated() { - c.yield(Token(entry.1, isFinal: i == script.count - 1, channel: entry.0)) - } - c.finish() - } - } - func structured(_ r: LLMRequest, schema: StructuredSchema, - as type: T.Type) async throws -> StructuredOutcome { - let json: String = { - lock.lock(); defer { lock.unlock() } - return routes.isEmpty ? "{\"tool\":\"\"}" : routes.removeFirst() - }() - return .value(try JSONDecoder().decode(T.self, from: Data(json.utf8))) - } - } - - func testTimelineInterleavesChannelsInArrivalOrderAndPersistsOnSettle() async throws { - let rt = ChannelScriptRuntime(script: [(.thinking, "T1"), (.response, "R1"), - (.thinking, "T2"), (.response, "R2")]) - let engine = makeEngine(try await loadedManager(runtime: rt), reasoning: true) - - engine.startNew() - engine.send("U1") - await waitUntil { engine.state == .awaitingTurn && engine.conversation?.messages.count == 2 } - - let message = try XCTUnwrap(engine.conversation?.messages.last) - XCTAssertEqual(message.text, "R1R2", "the committed text is the response channel only") - XCTAssertEqual(message.thinking, "T1T2", "the flat thinking accumulation is unchanged") - XCTAssertEqual(message.segments, - [TurnSegment(kind: .thinking, text: "T1"), - TurnSegment(kind: .answer, text: "R1"), - TurnSegment(kind: .thinking, text: "T2"), - TurnSegment(kind: .answer, text: "R2")], - "the persisted timeline preserves cross-channel ARRIVAL order — a segment per flip") - XCTAssertTrue(engine.liveSegments.isEmpty, - "the live timeline hands off to the persisted message at settle") - } - - func testTimelineCoalescesSameChannelChunksIntoOneSegment() async throws { - let rt = ChannelScriptRuntime(script: [(.thinking, "Ta"), (.thinking, "Tb"), (.response, "R")]) - let engine = makeEngine(try await loadedManager(runtime: rt), reasoning: true) - - engine.startNew() - engine.send("U1") - await waitUntil { engine.state == .awaitingTurn && engine.conversation?.messages.count == 2 } - - XCTAssertEqual(engine.conversation?.messages.last?.segments, - [TurnSegment(kind: .thinking, text: "TaTb"), - TurnSegment(kind: .answer, text: "R")], - "consecutive same-channel chunks coalesce — the array grows only at channel flips") - } - - func testRoutedAnswerStreamsLiveInterleavedIntoTheTimeline() async throws { - // Thinking AFTER the answer chunk can only appear mid-timeline if the answer streamed LIVE - // (the wired `onResponseToken`); an at-settle append would leave [thinking, answer] instead. - let rt = ChannelScriptRuntime(script: [(.thinking, "Ta"), (.response, "R1"), (.thinking, "Tb")], - routes: ["{\"tool\":\"\"}"]) - let engine = makeRoutedEngine(try await loadedManager(runtime: rt), registry: ToolRegistry([])) - - engine.startNew() - engine.send("U1") - await waitUntil { engine.state == .awaitingTurn && engine.conversation?.messages.count == 2 } - - let message = try XCTUnwrap(engine.conversation?.messages.last) - XCTAssertEqual(message.text, "R1") - XCTAssertEqual(message.segments, - [TurnSegment(kind: .thinking, text: "Ta"), - TurnSegment(kind: .answer, text: "R1"), - TurnSegment(kind: .thinking, text: "Tb")], - "the routed answer streamed live BETWEEN the thinking chunks — not whole at settle") - } - - // MARK: - Born-with tuning (`notch-timeline-and-tuning` D7) - - private func makeTunedEngine(_ manager: ModelManager, - reasoning: Bool, - tuning: @escaping @MainActor () -> NotchSessionEngine.TuningSnapshot?) - -> NotchSessionEngine { - NotchSessionEngine(modelManager: manager, selection: FakeSelectionProvider(), - reasoning: { reasoning }, tuningDefault: tuning) - } - - func testTuningIsStampedAtBirthAndKeptAcrossRebind() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["A1"], interTokenDelayNanos: 0) - var dial: NotchSessionEngine.TuningSnapshot? = (reasoning: false, contextTokens: 4_096) - let engine = makeTunedEngine(try await loadedManager(runtime: stub), reasoning: true, - tuning: { dial }) - - let born = engine.startNew() - XCTAssertEqual(born.reasoningOverride, false, "the dial's reasoning is stamped at birth") - XCTAssertEqual(born.contextTokens, 4_096, "the dial's context budget is stamped at birth") - XCTAssertEqual(engine.assembleRequest()?.reasoning, false, - "the turn runs with the born-with reasoning, not the global default") - - // The dial moves AFTER birth: a stored conversation keeps the tuning it was born under. - dial = (reasoning: true, contextTokens: 32_768) - var stored = born - stored.messages = [AgentMessage(role: .user, text: "U1")] - engine.bind(stored) - XCTAssertEqual(engine.assembleRequest()?.reasoning, false, - "re-binding keeps the born-with reasoning — a later slider change never retunes it") - } - - func testPreChangeConversationFallsBackToTheGlobalReasoningDefault() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["A1"], interTokenDelayNanos: 0) - let engine = makeTunedEngine(try await loadedManager(runtime: stub), reasoning: true, - tuning: { (reasoning: false, contextTokens: 4_096) }) - - // A conversation stored BEFORE the dial existed (no born-with fields) keeps the exact legacy - // behavior: the global reasoning default re-read at bind. - let legacy = AgentConversation(title: "old", - messages: [AgentMessage(role: .user, text: "U1")]) - engine.bind(legacy) - XCTAssertEqual(engine.assembleRequest()?.reasoning, true, - "nil born-with tuning falls back to the global reasoning default") - } - - // MARK: - advance() (the background driver's verb) - - func testAdvanceRunsThePendingTurnAndNoOpsWhenNothingPends() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["A1"], interTokenDelayNanos: 0) - let engine = makeEngine(try await loadedManager(runtime: stub)) - let now = Date() - let pending = AgentConversation( - title: "t", - messages: [AgentMessage(role: .user, text: "U1", createdAt: now)], - createdAt: now, updatedAt: now) - var settled: [(AgentConversation, NotchSessionEngine.TurnSettlement)] = [] - engine.onTurnSettled = { settled.append(($0, $1)) } - - engine.bind(pending) - engine.advance() - await waitUntil { !settled.isEmpty } - XCTAssertEqual(settled.first?.0.messages.map(\.text), ["U1", "A1"], - "advance ran the pending turn without appending a new user message") - - engine.advance() // last message is now the assistant's — nothing pends - try? await Task.sleep(nanoseconds: 30_000_000) - XCTAssertEqual(settled.count, 1, "advance with nothing pending runs no turn") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/ConversationalCanvasTests.swift b/Tests/ThreeFingerSwitcherTests/ConversationalCanvasTests.swift deleted file mode 100644 index 451e3bf..0000000 --- a/Tests/ThreeFingerSwitcherTests/ConversationalCanvasTests.swift +++ /dev/null @@ -1,249 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Engine-machine tests ported from the retired conversational canvas (`notch-native-conversations`): -/// the canvas-era seed/float-up/extract/park semantics died with the revert (the launcher canvas is -/// one-shot again — see `AICommandExecutorTests`), and the multi-turn machine they exercised lives on in -/// `NotchSessionEngine`. What survives here: multi-image turns + the vision capability flip, the -/// composer-attachment model's purity, fold-staged-images-then-clear, and the tool-step approval gate -/// (now resolved by the expanded notch panel's Approve/Skip buttons). -@MainActor -final class ConversationalCanvasTests: XCTestCase { - - // MARK: - Harness - - private final class FakeSelection: SelectionProviding { - /// Scriptable live-clipboard image bytes: `attachClipboardImage()` reads via this seam. - var clipboardImage: Data? - func readSelectedText() async -> String? { nil } - func readClipboardText() -> String? { nil } - func readClipboardImage() -> Data? { clipboardImage } - @discardableResult func replaceSelection(_ text: String) async -> Bool { true } - @discardableResult func pasteAtCursor(_ text: String) async -> Bool { true } - } - private final class FakeDownloader: ModelDownloading, @unchecked Sendable { - let payload: Data; init(payload: Data) { self.payload = payload } - func download(_ d: ModelDescriptor, to dest: URL, progress: @Sendable (Double) -> Void) async throws -> Data { progress(1); return payload } - } - private func loadedManager(_ runtime: LLMRuntime) async throws -> ModelManager { - let payload = Data("w".utf8) - let registry = ModelCatalog(models: [ModelDescriptor(id: "m", displayName: "M", sizeBytes: 1, - integritySHA: ModelManager.sha256Hex(payload), downloadURL: URL(string: "https://x.invalid/m")!, - capabilities: [.text, .vision], quantization: .qat4bit)], defaultModelID: "m") - let root = URL(fileURLWithPath: NSTemporaryDirectory()).appendingPathComponent("tfs-canvas-\(UUID().uuidString)") - let m = ModelManager(registry: registry, downloader: FakeDownloader(payload: payload), optedIn: true, - storageRoot: root, runtimeFactory: { _ in runtime }) - try await m.downloadAndVerify(registry.models[0]); return m - } - private func makeEngine(_ m: ModelManager, selection: FakeSelection? = nil) -> NotchSessionEngine { - NotchSessionEngine(modelManager: m, selection: selection ?? FakeSelection()) - } - private func waitUntil(_ p: @MainActor () -> Bool, _ timeout: TimeInterval = 2, - file: StaticString = #filePath, line: UInt = #line) async { - let deadline = Date().addingTimeInterval(timeout) - while !p() && Date() < deadline { try? await Task.sleep(nanoseconds: 2_000_000) } - XCTAssertTrue(p(), "condition not met", file: file, line: line) - } - - // MARK: - Multi-image turns + the vision capability flip - - func testSendWithImagesAppendsThemAndAssemblyForwardsThem() async throws { - let stub = StubLLMRuntime(scriptedTurns: [.init(tokens: ["A1"]), .init(tokens: ["A2"])], interTokenDelayNanos: 0) - let e = makeEngine(try await loadedManager(stub)) - e.startNew() - e.send("first") - await waitUntil { e.state == .awaitingTurn && e.conversation?.messages.count == 2 } - - let png1 = Data([0x01, 0x02]); let png2 = Data([0x03, 0x04]) - e.send("more", images: [png1, png2]) - await waitUntil { e.conversation?.messages.count == 4 } - let userTurn2 = try XCTUnwrap(e.conversation?.messages[2]) - XCTAssertEqual(userTurn2.images, [png1, png2], "BOTH images ride the appended user turn (multi-image)") - XCTAssertEqual(userTurn2.text, "more") - let req = try XCTUnwrap(e.assembleRequest()) - XCTAssertEqual(req.images, [png1, png2], "assembleRequest forwards the FULL images array to LLMChatRequest") - } - - /// Vision flip: after attaching images to turn 2, the turn requests [.text, .vision]. Observed via a - /// text-only registry: a vision turn against it fails with a clean capability message. - func testAttachingImagesFlipsTheTurnToVision() async throws { - let textOnly = StubLLMRuntime(capabilities: [.text], scriptedTurns: [.init(tokens: ["A1"])], - interTokenDelayNanos: 0) - let payload = Data("w".utf8) - let registry = ModelCatalog(models: [ModelDescriptor(id: "m", displayName: "M", sizeBytes: 1, - integritySHA: ModelManager.sha256Hex(payload), downloadURL: URL(string: "https://x.invalid/m")!, - capabilities: [.text], quantization: .qat4bit)], defaultModelID: "m") - let root = URL(fileURLWithPath: NSTemporaryDirectory()).appendingPathComponent("tfs-vflip-\(UUID().uuidString)") - let m = ModelManager(registry: registry, downloader: FakeDownloader(payload: payload), optedIn: true, - storageRoot: root, runtimeFactory: { _ in textOnly }) - try await m.downloadAndVerify(registry.models[0]) - let e = NotchSessionEngine(modelManager: m, selection: FakeSelection()) - - e.startNew() - e.send("text turn") // turn 1: text only → succeeds against the text model - await waitUntil { e.state == .awaitingTurn && e.conversation?.messages.count == 2 } - - e.send("look", images: [Data([0x89, 0x50])]) // turn 2: an image → needsVision → asks [.text,.vision] - await waitUntil { if case .failed = e.state { return true }; return false } - guard case let .failed(message) = e.state else { return XCTFail("expected .failed (no vision model)") } - XCTAssertTrue(message.lowercased().contains("vision"), - "an image turn requires a vision-capable runtime; the text-only model can't serve it: \(message)") - } - - // MARK: - Composer attachment model purity (API kept; the notch attach UI is a documented future) - - func testClipboardImageAttachmentPopulatesAndClears() async throws { - let selection = FakeSelection() - let png = Data([0xAA, 0xBB]) - selection.clipboardImage = png - let e = makeEngine(try await loadedManager(StubLLMRuntime(interTokenDelayNanos: 0)), selection: selection) - - XCTAssertTrue(e.pendingAttachments.isEmpty, "starts empty") - XCTAssertTrue(e.attachClipboardImage(), "a clipboard image is staged") - XCTAssertEqual(e.pendingAttachments.images, [png]) - - // Staging a screenshot appends; removing by index empties. - e.attachScreenshot(Data([0xCC])) - XCTAssertEqual(e.pendingAttachments.images, [png, Data([0xCC])]) - e.removeAttachment(at: 0) - XCTAssertEqual(e.pendingAttachments.images, [Data([0xCC])]) - e.removeAttachment(at: 5) // out of range is a no-op - XCTAssertEqual(e.pendingAttachments.images, [Data([0xCC])]) - e.clearPendingAttachments() - XCTAssertTrue(e.pendingAttachments.isEmpty, "clear empties") - } - - func testAttachClipboardImageNoOpWhenClipboardHasNoImage() async throws { - let selection = FakeSelection() // clipboardImage == nil - let e = makeEngine(try await loadedManager(StubLLMRuntime(interTokenDelayNanos: 0)), selection: selection) - XCTAssertFalse(e.attachClipboardImage(), "no image on the clipboard → no staging, not a failure") - XCTAssertTrue(e.pendingAttachments.isEmpty) - } - - /// The staged-attachment send path: staging a clipboard image AND a screenshot then sending text - /// (the engine folds `pendingAttachments.images` when no explicit images ride the call) puts BOTH - /// images on the one appended turn, forwards them to assembly, and clears the staging. - func testSendFoldsMultipleStagedSourcesOntoTurnThenClears() async throws { - let selection = FakeSelection() - let clip = Data([0x01, 0x02]) - let shot = Data([0x03, 0x04]) - selection.clipboardImage = clip - let stub = StubLLMRuntime(scriptedTokens: ["A1"], interTokenDelayNanos: 0) - let e = makeEngine(try await loadedManager(stub), selection: selection) - e.startNew() - - XCTAssertTrue(e.attachClipboardImage()) - e.attachScreenshot(shot) - XCTAssertEqual(e.pendingAttachments.images, [clip, shot], "both sources staged for one turn") - e.send("look at these") - await waitUntil { e.state == .awaitingTurn && e.conversation?.messages.count == 2 } - - let turn = try XCTUnwrap(e.conversation?.messages.first) - XCTAssertEqual(turn.text, "look at these") - XCTAssertEqual(turn.images, [clip, shot], "BOTH staged images ride the one turn (multi-image)") - let req = try XCTUnwrap(e.assembleRequest()) - XCTAssertEqual(req.images, [clip, shot], "assembly forwards the full staged-image set to the request") - XCTAssertTrue(e.pendingAttachments.isEmpty, "the staging clears once the turn folds the images") - } - - func testSessionResetsClearStagedAttachments() async throws { - let selection = FakeSelection() - selection.clipboardImage = Data([0x99]) - let e = makeEngine(try await loadedManager(StubLLMRuntime(interTokenDelayNanos: 0)), selection: selection) - e.startNew() - XCTAssertTrue(e.attachClipboardImage()) - e.unbind() - XCTAssertTrue(e.pendingAttachments.isEmpty, "collapse clears staged-but-unsent attachments") - - // A re-bind also starts with an empty staging area. - e.bind(AgentConversation(title: "t", messages: [AgentMessage(role: .user, text: "q")])) - XCTAssertTrue(e.pendingAttachments.isEmpty, "a bound session starts with an empty composer staging") - } - - // MARK: - Tool-step approval gate (resolved by the expanded panel's Approve/Skip buttons) - - /// A contributor that always needs approval, then runs the routed call per the gate's decision. - private struct GatedContributor: ToolContributor { - let name: String - func descriptors() -> [ToolDescriptor] { - [ToolDescriptor(name: name, summary: "t", argsSchema: StructuredSchema(name: name, json: "{\"type\":\"object\"}"), - writePolicy: .confirm, keywords: [])] - } - func canHandle(_ tool: String) -> Bool { tool == name } - func run(_ call: RoutedCall, gate: ApprovalGate) async -> ToolStepResult { - let decision = await gate.awaitDecision(for: .action(title: "Do it", fields: [ReviewField("What", "thing")], - payload: .openTool(tool: name, action: ParsedOpenTool(applicable: true, reason: nil, payload: "p")))) - switch decision { - case .approve: return ToolStepResult(tool: name, status: .done, summary: "ran \(name)") - case .skip: return ToolStepResult(tool: name, status: .declined(reason: "skipped"), summary: "skipped \(name)") - case .cancel: return ToolStepResult(tool: name, status: .declined(reason: "cancelled"), summary: "cancelled") - } - } - } - - private struct RoutingStub: LLMRuntime, @unchecked Sendable { - let capabilities: Set = [.text] - let routeJSON: String - let answer: String - final class Box: @unchecked Sendable { var routed = false } - let box = Box() - func generate(_ request: LLMRequest) -> AsyncThrowingStream { - AsyncThrowingStream { c in c.yield(Token(answer, isFinal: true)); c.finish() } - } - func structured(_ request: LLMRequest, schema: StructuredSchema, as type: T.Type) async throws -> StructuredOutcome { - // First call routes to the gated tool; subsequent calls answer plainly (empty tool). - let json = box.routed ? "{\"tool\":\"\"}" : routeJSON - box.routed = true - return .value(try JSONDecoder().decode(T.self, from: Data(json.utf8))) - } - } - - private func isAwaitingApproval(_ e: NotchSessionEngine) -> Bool { - if case .awaitingApproval = e.state { return true } - return false - } - - func testApprovalGateApproveFiresTheStep() async throws { - let e = makeEngine(try await loadedManager(StubLLMRuntime(interTokenDelayNanos: 0))) - let contributor = GatedContributor(name: "doThing") - let registry = ToolRegistry([contributor]) - let source = StaticCandidateSource(contributor.descriptors()) - let gate = e.makeApprovalGate() - let loop = AgentLoop(runtime: RoutingStub(routeJSON: "{\"tool\":\"doThing\"}", answer: "Done."), - registry: registry, candidateSource: source, gate: gate, maxToolSteps: 4) - async let result = loop.run(context: RouteContext(messages: [AgentMessage(role: .user, text: "do it")])) - // The loop pauses awaiting the decision → the engine surfaces `.awaitingApproval`. - await waitUntil { self.isAwaitingApproval(e) } - XCTAssertTrue(e.approve(), "approve resolves the pending pause") - let r = await result - XCTAssertTrue(r.steps.contains { if case .done = $0.status { return true }; return false }, "approve fired the step") - } - - func testApprovalGateSkipDeclinesTheStep() async throws { - let e = makeEngine(try await loadedManager(StubLLMRuntime(interTokenDelayNanos: 0))) - let contributor = GatedContributor(name: "doThing") - let registry = ToolRegistry([contributor]) - let source = StaticCandidateSource(contributor.descriptors()) - let gate = e.makeApprovalGate() - let loop = AgentLoop(runtime: RoutingStub(routeJSON: "{\"tool\":\"doThing\"}", answer: "OK."), - registry: registry, candidateSource: source, gate: gate, maxToolSteps: 4) - async let result = loop.run(context: RouteContext(messages: [AgentMessage(role: .user, text: "do it")])) - await waitUntil { self.isAwaitingApproval(e) } - XCTAssertTrue(e.skip(), "skip resolves the pending pause") - let r = await result - XCTAssertTrue(r.steps.contains { if case .declined = $0.status { return true }; return false }, "skip declined the step") - } - - func testApproveSkipNoOpWhenNothingPending() async throws { - let e = makeEngine(try await loadedManager(StubLLMRuntime(interTokenDelayNanos: 0))) - XCTAssertFalse(e.approve(), "no pending step → approve is a no-op") - XCTAssertFalse(e.skip(), "no pending step → skip is a no-op") - } -} - -/// A minimal candidate source advertising a fixed descriptor set (the registry's tools), for the loop tests. -private struct StaticCandidateSource: ToolCandidateSource { - let all: [ToolDescriptor] - init(_ all: [ToolDescriptor]) { self.all = all } - func candidates(for context: RouteContext, limit: Int) -> [ToolDescriptor] { Array(all.prefix(limit)) } -} diff --git a/Tests/ThreeFingerSwitcherTests/EvictionPolicyTests.swift b/Tests/ThreeFingerSwitcherTests/EvictionPolicyTests.swift deleted file mode 100644 index cba5af4..0000000 --- a/Tests/ThreeFingerSwitcherTests/EvictionPolicyTests.swift +++ /dev/null @@ -1,248 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// `model-idle-ttl-and-memory-pressure`: the pure `EvictionPolicy` (every spec scenario is a case -/// here — no OS pressure source, no resident model, faked time throughout) plus the `ModelManager` -/// integration (TTL/pressure evict transitions `loaded → ready` and the next request transparently -/// re-loads through the single-flight path). -@MainActor -final class EvictionPolicyTests: XCTestCase { - - private let t0 = Date(timeIntervalSinceReferenceDate: 1_000_000) - - private func verdict(sinceActivity: TimeInterval = 0, - pressure: MemoryPressureLevel = .nominal, - quiescence: QuiescenceSnapshot = QuiescenceSnapshot(), - ttl: TimeInterval = 3600, - loadInFlight: Bool = false) -> EvictionVerdict { - EvictionPolicy.verdict(now: t0, - lastActivity: t0.addingTimeInterval(-sinceActivity), - pressure: pressure, - quiescence: quiescence, - ttl: ttl, - loadInFlight: loadInFlight) - } - - // MARK: - Pure policy - - func testIdleTTLEvictsAQuiescentSystem() { - XCTAssertEqual(verdict(sinceActivity: 3600, ttl: 3600), .evict(.idleTTL)) - } - - func testBelowTTLKeeps() { - XCTAssertEqual(verdict(sinceActivity: 3599, ttl: 3600), .keep) - } - - func testTTLZeroRestoresKeepForeverButPressureStaysArmed() { - // Quiescent forever with ttl 0 → never a TTL evict… - XCTAssertEqual(verdict(sinceActivity: 1_000_000, ttl: 0), .keep) - // …while a pressure trigger on the same inputs still fires. - XCTAssertEqual(verdict(sinceActivity: 1_000_000, pressure: .warning, ttl: 0), - .evict(.warningPressure)) - XCTAssertEqual(verdict(sinceActivity: 0, pressure: .critical, ttl: 0), - .evict(.criticalPressure)) - } - - func testWarningPressureRespectsAnOpenConversation() { - let open = QuiescenceSnapshot(foregroundSessionActive: true) - XCTAssertEqual(verdict(pressure: .warning, quiescence: open), .keep) - // And an open conversation also protects against the TTL trigger. - XCTAssertEqual(verdict(sinceActivity: 10_000, quiescence: open, ttl: 3600), .keep) - } - - func testWarningPressureEvictsOnlyWhenQuiescentANDIdlePastTheFloor() { - // Fully quiescent but RECENTLY active: chronic warning between turns must NOT thrash - // (`fix-evict-thrash-and-hot-path` — the reload-storm regression). - XCTAssertEqual(verdict(sinceActivity: 0, pressure: .warning), .keep) - XCTAssertEqual(verdict(sinceActivity: EvictionPolicy.warningIdleFloor - 1, pressure: .warning), - .keep) - // Quiescent AND genuinely idle: warning still reclaims. - XCTAssertEqual(verdict(sinceActivity: EvictionPolicy.warningIdleFloor, pressure: .warning), - .evict(.warningPressure)) - } - - func testReloadBuysAGraceWindowUnderChronicWarning() { - // The storm scenario end-to-end at the policy level: activity just stamped (a reload), the - // system back at warning — the next ticks keep the model until the floor passes. - XCTAssertEqual(verdict(sinceActivity: 60, pressure: .warning), .keep) - XCTAssertEqual(verdict(sinceActivity: 299, pressure: .warning), .keep) - // …and critical remains the immediate emergency valve. - XCTAssertEqual(verdict(sinceActivity: 0, pressure: .critical), .evict(.criticalPressure)) - } - - func testCriticalPressureEvictsBetweenTurnsEvenWithForegroundSession() { - let open = QuiescenceSnapshot(foregroundSessionActive: true) - XCTAssertEqual(verdict(pressure: .critical, quiescence: open), .evict(.criticalPressure)) - } - - func testNeverEvictMidTurnOrMidLoad() { - let midTurn = QuiescenceSnapshot(turnInFlight: true) - XCTAssertEqual(verdict(pressure: .critical, quiescence: midTurn), .keep) - XCTAssertEqual(verdict(pressure: .warning, quiescence: midTurn), .keep) - XCTAssertEqual(verdict(sinceActivity: 10_000, quiescence: midTurn), .keep) - XCTAssertEqual(verdict(pressure: .critical, loadInFlight: true), .keep) - XCTAssertEqual(verdict(sinceActivity: 10_000, ttl: 3600, loadInFlight: true), .keep) - } - - func testImminentScheduledWorkBlocksTTLAndWarning() { - let imminent = QuiescenceSnapshot(nextScheduledWork: t0.addingTimeInterval(60)) - XCTAssertEqual(verdict(sinceActivity: 10_000, quiescence: imminent), .keep) - XCTAssertEqual(verdict(pressure: .warning, quiescence: imminent), .keep) - // Past-due counts as imminent (the driver just hasn't served it yet). - let overdue = QuiescenceSnapshot(nextScheduledWork: t0.addingTimeInterval(-5)) - XCTAssertEqual(verdict(sinceActivity: 10_000, quiescence: overdue), .keep) - // Work far beyond the horizon does NOT hold the weights resident (it lazy-reloads later). - let far = QuiescenceSnapshot( - nextScheduledWork: t0.addingTimeInterval(EvictionPolicy.scheduledWorkHorizon + 60)) - XCTAssertEqual(verdict(sinceActivity: 10_000, quiescence: far), .evict(.idleTTL)) - // …but critical pressure ignores the schedule (only turn/load block it). - XCTAssertEqual(verdict(pressure: .critical, quiescence: imminent), .evict(.criticalPressure)) - } - - // MARK: - Manager integration fixtures - - private final class InstantDownloader: ModelDownloading, @unchecked Sendable { - let payload: Data - init(payload: Data) { self.payload = payload } - func download(_ descriptor: ModelDescriptor, to destination: URL, - progress: @Sendable (Double) -> Void) async throws -> Data { - progress(1.0) - return payload - } - } - - private func tempRoot() -> URL { - let url = URL(fileURLWithPath: NSTemporaryDirectory()) - .appendingPathComponent("tfs-evict-tests-\(UUID().uuidString)", isDirectory: true) - try? FileManager.default.createDirectory(at: url, withIntermediateDirectories: true) - return url - } - - private func catalog(matching payload: Data, lane: ComputeLane? = .gpu) -> ModelCatalog { - ModelCatalog( - models: [ModelDescriptor( - id: "evict-test-model", - displayName: "Evict Test Model", - sizeBytes: Int64(payload.count), - integritySHA: ModelManager.sha256Hex(payload), - downloadURL: URL(string: "https://models.invalid/evict-test")!, - capabilities: [.text], - quantization: .qat4bit, - lane: lane - )], - defaultModelID: "evict-test-model" - ) - } - - /// A loaded manager with automatic eviction installed against the given fakes. Time starts at - /// `t0`; tests advance it by mutating `clock.now`. - private final class Clock { var now: Date; init(_ d: Date) { now = d } } - - private func loadedManager(payload: Data = Data("weights".utf8), - lane: ComputeLane? = .gpu, - pressure: FakeMemoryPressureSource, - clock: Clock, - quiescence: @escaping () -> QuiescenceSnapshot = { QuiescenceSnapshot() }, - ttlMinutes: Int = 60) async throws -> ModelManager { - let manager = ModelManager(registry: catalog(matching: payload, lane: lane), - downloader: InstantDownloader(payload: payload), - optedIn: true, - storageRoot: tempRoot()) - // Install BEFORE loading so the activity stamps taken during the load use the fake clock — - // otherwise `lastActivity` is real wall-clock time and every TTL comparison goes negative. - manager.installAutomaticEviction(pressure: pressure, - quiescence: quiescence, - idleTTL: { TimeInterval(ttlMinutes * 60) }, - now: { clock.now }, - tickInterval: nil) // tests drive evaluation directly - try await manager.downloadAndVerify(manager.registry.models[0]) - _ = try await manager.loadIfNeeded() - return manager - } - - // MARK: - Manager integration - - func testTTLEvictTransitionsLoadedToReadyAndReloadsTransparently() async throws { - let clock = Clock(t0) - let pressure = FakeMemoryPressureSource() - let manager = try await loadedManager(pressure: pressure, clock: clock) - XCTAssertTrue(manager.isResident) - - // Below the TTL: the tick keeps the model resident. - clock.now = t0.addingTimeInterval(30 * 60) - manager.evaluateAutomaticEviction() - XCTAssertTrue(manager.isResident, "half the TTL is not idle enough") - - // Past the TTL while fully quiescent: evicted, weights still on disk → `.ready`. - clock.now = t0.addingTimeInterval(61 * 60) - manager.evaluateAutomaticEviction() - XCTAssertFalse(manager.isResident, "TTL must evict a quiescent system") - XCTAssertEqual(manager.state, .ready, "weights stay on disk — ready, never a failure") - - // Invisible-correct: the next request transparently re-loads (single-flight path). - let runtime = try await manager.loadIfNeeded() - XCTAssertNotNil(runtime) - XCTAssertEqual(manager.state, .loaded) - XCTAssertTrue(manager.isResident) - } - - func testPressureEventEvictsImmediatelyWithoutWaitingForTick() async throws { - let clock = Clock(t0) - let pressure = FakeMemoryPressureSource() - let manager = try await loadedManager(pressure: pressure, clock: clock) - XCTAssertTrue(manager.isResident) - // A critical event fires the policy through `onChange` — no tick needed. - pressure.report(.critical) - XCTAssertFalse(manager.isResident) - XCTAssertEqual(manager.state, .ready) - } - - func testWarningPressureKeepsWhileForegroundSessionOpen() async throws { - let clock = Clock(t0) - let pressure = FakeMemoryPressureSource() - let open = QuiescenceSnapshot(foregroundSessionActive: true) - let manager = try await loadedManager(pressure: pressure, clock: clock, - quiescence: { open }) - pressure.report(.warning) - XCTAssertTrue(manager.isResident, "warning must respect an open conversation") - pressure.report(.critical) - XCTAssertFalse(manager.isResident, "critical overrides the open session between turns") - } - - func testTurnInFlightBlocksEvenCriticalPressure() async throws { - let clock = Clock(t0) - let pressure = FakeMemoryPressureSource() - let busy = QuiescenceSnapshot(turnInFlight: true) - let manager = try await loadedManager(pressure: pressure, clock: clock, - quiescence: { busy }) - pressure.report(.critical) - XCTAssertTrue(manager.isResident, "never evict mid-turn") - } - - func testCPUTernaryLaneIsExempt() async throws { - let clock = Clock(t0) - let pressure = FakeMemoryPressureSource() - let manager = try await loadedManager(lane: .cpuTernary, pressure: pressure, clock: clock) - clock.now = t0.addingTimeInterval(10 * 3600) - manager.evaluateAutomaticEviction() - pressure.report(.critical) - XCTAssertTrue(manager.isResident, "the ternary lane's small footprint is never auto-evicted") - } - - func testActivityStampDefersTTL() async throws { - let clock = Clock(t0) - let pressure = FakeMemoryPressureSource() - let manager = try await loadedManager(pressure: pressure, clock: clock) - // A request 50 minutes in re-stamps activity… - clock.now = t0.addingTimeInterval(50 * 60) - _ = try await manager.loadIfNeeded() - // …so 61 minutes after t0 (only 11 after the stamp) the system is not idle yet. - clock.now = t0.addingTimeInterval(61 * 60) - manager.evaluateAutomaticEviction() - XCTAssertTrue(manager.isResident) - // 50 + 61 minutes: now genuinely idle past the TTL. - clock.now = t0.addingTimeInterval((50 + 61) * 60) - manager.evaluateAutomaticEviction() - XCTAssertFalse(manager.isResident) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/FileOpenServiceTests.swift b/Tests/ThreeFingerSwitcherTests/FileOpenServiceTests.swift deleted file mode 100644 index 8142c6d..0000000 --- a/Tests/ThreeFingerSwitcherTests/FileOpenServiceTests.swift +++ /dev/null @@ -1,335 +0,0 @@ -import XCTest -import Foundation -@testable import ThreeFingerSwitcherCore - -/// Tests for the Files-band open / Open-With / defusable-open service (change: files-band, tasks 3.2 / 3.3 / -/// 3.4) against a stub `FileWorkspace` that records every call and can simulate a failure: -/// - default open routes a **file** to its default app and a **folder** to a Finder-window open; -/// - Open-With lists ONLY the capable apps, with the default indicated; -/// - choosing an app opens with it; -/// - a defuse cancels a pending open so the stub records NO open, and **never terminates** anything; -/// - a failed open surfaces a clean, bounded headline carrying NO raw error text. -/// -/// `@MainActor` because `FileOpenService` / `PendingOpen` are `@MainActor` (they hold observable UI state), -/// matching `AICommandExecutorTests`. -@MainActor -final class FileOpenServiceTests: XCTestCase { - - // MARK: - Stub workspace - - /// A scriptable `FileWorkspace` that records what it was asked to open (so a defuse's "opens nothing" is - /// testable) and serves canned app associations. It has **no terminate capability at all** — there is - /// nothing for a defuse to call — which is exactly the point: defuse can only prevent a not-yet-fired open, - /// it can't kill a running app. `openError`, when set, makes every open throw (the mapped-at-the-boundary - /// `FileActionError`), exercising the `.failed` surfacing. Foundation-only (no `import AppKit`). - private final class StubFileWorkspace: FileWorkspace { - private(set) var openedDefault: [URL] = [] - private(set) var openedWith: [(file: URL, app: URL)] = [] - private let apps: [URL] - private let defaultApp: URL? - private let openError: FileActionError? - - init(apps: [URL] = [], defaultApp: URL? = nil, openError: FileActionError? = nil) { - self.apps = apps - self.defaultApp = defaultApp - self.openError = openError - } - - /// Total opens of any kind — `0` proves a defuse opened nothing. - var totalOpens: Int { openedDefault.count + openedWith.count } - - func open(_ url: URL) async throws { - if let openError { throw openError } - openedDefault.append(url) - } - - func open(_ url: URL, withApplicationAt applicationURL: URL) async throws { - if let openError { throw openError } - openedWith.append((file: url, app: applicationURL)) - } - - func urlsForApplications(toOpen url: URL) -> [URL] { apps } - func urlForApplication(toOpen url: URL) -> URL? { defaultApp } - } - - // MARK: - Fixtures - - private func fileEntry(_ path: String) -> FileEntry { - let url = URL(fileURLWithPath: path) - return FileEntry(url: url, name: url.lastPathComponent, isDirectory: false, - modificationDate: nil, kind: .text) - } - - private func folderEntry(_ path: String) -> FileEntry { - let url = URL(fileURLWithPath: path) - return FileEntry(url: url, name: url.lastPathComponent, isDirectory: true, - modificationDate: nil, kind: .folder) - } - - /// Spin the run loop until `condition` holds (the open fires on a `Task`), bounded so a never-firing open - /// fails loudly via the caller's assertion instead of hanging. Mirrors `AICommandExecutorTests.waitUntil`. - private func waitUntil(_ condition: @MainActor () -> Bool, timeout: TimeInterval = 2) async { - let deadline = Date().addingTimeInterval(timeout) - while !condition() && Date() < deadline { - try? await Task.sleep(nanoseconds: 2_000_000) // 2 ms - } - } - - /// Let any scheduled `Task` run, then return — for the "nothing opened" assertions, where we must give a - /// (possibly defused) open the chance to fire before asserting it did NOT. A short, fixed settle: a defuse - /// has no async work to do, so a few run-loop turns is plenty. - private func settle() async { - for _ in 0..<10 { - await Task.yield() - try? await Task.sleep(nanoseconds: 2_000_000) // 2 ms - } - } - - // MARK: - Default open routing (task 3.2 / 3.4) - - func testDefaultOpenRoutesAFileToItsDefaultApp() async { - let workspace = StubFileWorkspace() - let service = FileOpenService(workspace: workspace) - let file = fileEntry("/tmp/report.txt") - - service.prepareOpen(file).commit() - await waitUntil { service.state == .opened } - - XCTAssertEqual(workspace.openedDefault, [file.url], "a file opens in its default app via the workspace") - XCTAssertTrue(workspace.openedWith.isEmpty, "a default open is not an Open-With") - XCTAssertEqual(service.state, .opened) - } - - func testDefaultOpenRoutesAFolderToAFinderWindowOpen() async { - let workspace = StubFileWorkspace() - let service = FileOpenService(workspace: workspace) - let folder = folderEntry("/tmp/Projects") - - service.prepareOpen(folder).commit() - await waitUntil { service.state == .opened } - - // A folder opens through the SAME default-open seam (which, in production, opens a Finder window on - // the current Space) — the service routes both kinds to `workspace.open(_:)`, never via SpaceWindowMover. - XCTAssertEqual(workspace.openedDefault, [folder.url], "a folder opens as a Finder window via the default open") - XCTAssertTrue(workspace.openedWith.isEmpty) - } - - func testOpenTargetsTheCapturedFrontAppContextBeforeFiring() async { - let workspace = StubFileWorkspace() - var activated = 0 - let service = FileOpenService(workspace: workspace, activateFrontAppContext: { activated += 1 }) - - service.prepareOpen(fileEntry("/tmp/a.txt")).commit() - await waitUntil { service.state == .opened } - - XCTAssertEqual(activated, 1, "the open re-asserts the captured front-app context (not the frontmost at fire time)") - } - - // MARK: - Open-With enumeration (task 3.2 / 3.4) - - func testOpenWithListsOnlyCapableAppsWithTheDefaultIndicated() { - let textEdit = URL(fileURLWithPath: "/Applications/TextEdit.app") - let xcode = URL(fileURLWithPath: "/Applications/Xcode.app") - // Only these two are "capable"; the full installed-apps list is irrelevant — the service lists exactly - // what the workspace's association returns, in order, with the default flagged. - let workspace = StubFileWorkspace(apps: [textEdit, xcode], defaultApp: xcode) - let service = FileOpenService(workspace: workspace) - - let candidates = service.openWithCandidates(for: fileEntry("/tmp/a.swift")) - - XCTAssertEqual(candidates.map { $0.app.url }, [textEdit, xcode], "only the capable apps, in system order") - XCTAssertEqual(candidates.first { $0.isDefault }?.app.url, xcode, "the default app is indicated") - XCTAssertEqual(candidates.filter { $0.isDefault }.count, 1, "exactly one app is marked default") - XCTAssertEqual(candidates.first { $0.app.url == textEdit }?.isDefault, false) - // The reused `AppCandidate` derives a display name from the bundle URL. - XCTAssertEqual(candidates.first?.app.name, "TextEdit") - } - - func testOpenWithIsEmptyForAFolder() { - let finder = URL(fileURLWithPath: "/System/Library/CoreServices/Finder.app") - let workspace = StubFileWorkspace(apps: [finder], defaultApp: finder) - let service = FileOpenService(workspace: workspace) - // Open-With is a file action: a folder's default open is a Finder window, so no Open-With list. - XCTAssertTrue(service.openWithCandidates(for: folderEntry("/tmp/Projects")).isEmpty) - } - - func testOpenWithIsEmptyWhenNoApplicationHandlesTheFile() { - let service = FileOpenService(workspace: StubFileWorkspace(apps: [], defaultApp: nil)) - XCTAssertTrue(service.openWithCandidates(for: fileEntry("/tmp/thing.weird")).isEmpty) - } - - func testChoosingAnAppOpensTheFileWithIt() async { - let workspace = StubFileWorkspace() - let service = FileOpenService(workspace: workspace) - let file = fileEntry("/tmp/a.swift") - let app = URL(fileURLWithPath: "/Applications/Xcode.app") - - service.prepareOpenWith(file, appURL: app).commit() - await waitUntil { service.state == .opened } - - XCTAssertEqual(workspace.openedWith.count, 1) - XCTAssertEqual(workspace.openedWith.first?.file, file.url) - XCTAssertEqual(workspace.openedWith.first?.app, app, "the file opens with the chosen app") - XCTAssertTrue(workspace.openedDefault.isEmpty, "Open-With does not also fire the default open") - } - - // MARK: - Defusable open (task 3.3 / 3.4) - - func testDefuseBeforeCommitOpensNothing() async { - let workspace = StubFileWorkspace() - let service = FileOpenService(workspace: workspace) - - let pending = service.prepareOpen(fileEntry("/tmp/a.txt")) - XCTAssertTrue(pending.isCommittable, "a freshly prepared open is held, awaiting commit") - pending.cancel() // discard before the open ever fired - - // Give any (nonexistent) scheduled open a chance to run, then assert nothing opened. - await settle() - XCTAssertEqual(workspace.totalOpens, 0, "a defused open opens nothing") - XCTAssertEqual(service.state, .idle, "a defused open rests at idle, never .opened") - XCTAssertFalse(pending.isCommittable, "a cancelled open is resolved (a stray re-lift is a no-op)") - } - - func testDefuseViaTheServiceOpensNothing() async { - let workspace = StubFileWorkspace() - let service = FileOpenService(workspace: workspace) - - service.prepareOpen(fileEntry("/tmp/a.txt")) - service.cancelPending() // discard via the service's held pending - - await settle() - XCTAssertEqual(workspace.totalOpens, 0) - XCTAssertNil(service.pendingOpen, "the pending open is cleared after a defuse") - } - - func testDefuseDuringTheFuseWindowOpensNothing() async { - let workspace = StubFileWorkspace() - let service = FileOpenService(workspace: workspace) - - let pending = service.prepareOpen(fileEntry("/tmp/a.txt")) - pending.commit(afterFuse: .milliseconds(200)) // armed, but not yet fired - XCTAssertTrue(pending.isDefusable, "a fusing open is still defusable") - pending.cancel() // discard within the fuse window - - await settle() - XCTAssertEqual(workspace.totalOpens, 0, "defusing within the fuse window opens nothing") - XCTAssertEqual(service.state, .idle) - } - - func testFuseThatIsNotDefusedFires() async { - let workspace = StubFileWorkspace() - let service = FileOpenService(workspace: workspace) - let file = fileEntry("/tmp/a.txt") - - service.prepareOpen(file).commit(afterFuse: .milliseconds(20)) - await waitUntil { service.state == .opened } - XCTAssertEqual(workspace.openedDefault, [file.url], "an un-defused fuse fires the open") - } - - func testDefuseNeverTerminatesAnything() async { - // The stub workspace exposes NO terminate API — so the only way this test compiles is if defuse never - // tries to terminate. Defuse only prevents a not-yet-fired open; an already-running app is untouched. - let workspace = StubFileWorkspace() - let service = FileOpenService(workspace: workspace) - - let pending = service.prepareOpen(fileEntry("/tmp/already-running.txt")) - pending.cancel() - - await settle() - // No open, and (by construction) no terminate — the recorded calls are open-only. - XCTAssertEqual(workspace.openedDefault, [], "defuse opens nothing") - XCTAssertEqual(workspace.openedWith.count, 0, "defuse fires no Open-With") - } - - func testStrayReLiftAfterCommitDoesNotDoubleOpen() async { - let workspace = StubFileWorkspace() - let service = FileOpenService(workspace: workspace) - let file = fileEntry("/tmp/a.txt") - - let pending = service.prepareOpen(file) - pending.commit() - await waitUntil { service.state == .opened } - // A stray re-lift after the firing lift: a second commit AND a late cancel are both no-ops. - pending.commit() - pending.cancel() - await settle() - - XCTAssertEqual(workspace.openedDefault, [file.url], "the open fired exactly once") - XCTAssertEqual(service.state, .opened, "a late discard does not undo or kill the opened window") - } - - func testPreparingASecondOpenSupersedesAStillPendingOne() async { - let workspace = StubFileWorkspace() - let service = FileOpenService(workspace: workspace) - let first = fileEntry("/tmp/first.txt") - let second = fileEntry("/tmp/second.txt") - - let firstPending = service.prepareOpen(first) - let secondPending = service.prepareOpen(second) // supersedes the first (cancels it) - XCTAssertFalse(firstPending.isCommittable, "the superseded open is cancelled") - XCTAssertTrue(secondPending.isCommittable) - - secondPending.commit() - await waitUntil { service.state == .opened } - XCTAssertEqual(workspace.openedDefault, [second.url], "only the live (second) open fires") - } - - // MARK: - Failure surfacing: clean, bounded, no raw error text (task 3.4) - - func testAFailedOpenSurfacesACleanBoundedHeadline() async { - let rawOSText = "Error Domain=NSCocoaErrorDomain Code=257 \"You don't have permission.\" UserInfo={…}" - // The workspace boundary already mapped the OS error into the taxonomy, stashing the raw text in the - // opt-in `details` — never the headline. - let mapped = FileActionError.openFailed(name: "report.txt", details: rawOSText) - let workspace = StubFileWorkspace(openError: mapped) - let service = FileOpenService(workspace: workspace) - - service.prepareOpen(fileEntry("/tmp/report.txt")).commit() - await waitUntil { if case .failed = service.state { return true }; return false } - - guard case let .failed(headline, details) = service.state else { - return XCTFail("a failed open must surface .failed, got \(service.state)") - } - XCTAssertFalse(headline.isEmpty, "the failure carries a clean headline") - XCTAssertTrue(headline.contains("report.txt"), "the headline names the file") - assertHeadlineIsClean(headline) - // The raw OS text rides ONLY in the opt-in details (for a Show-details/Copy disclosure), never the - // headline: the details carry it verbatim, the headline (asserted clean above) does not. - XCTAssertEqual(details, rawOSText, "the raw OS text is preserved as opt-in copyable details") - XCTAssertTrue(workspace.openedDefault.isEmpty, "a failed open never falsely records a success") - } - - func testAFailedOpenWithSurfacesACleanBoundedHeadline() async { - let mapped = FileActionError.openFailed(name: "a.swift", - details: "Error Domain=NSOSStatusErrorDomain Code=-10814") - let workspace = StubFileWorkspace(openError: mapped) - let service = FileOpenService(workspace: workspace) - - service.prepareOpenWith(fileEntry("/tmp/a.swift"), - appURL: URL(fileURLWithPath: "/Applications/Xcode.app")).commit() - await waitUntil { if case .failed = service.state { return true }; return false } - - guard case let .failed(headline, details) = service.state else { - return XCTFail("a failed Open-With must surface .failed, got \(service.state)") - } - assertHeadlineIsClean(headline) - // The raw status text is opt-in details only — present (so a disclosure can show it), never inline: - // the details carry the raw text verbatim, while the headline (asserted clean above) does not. - XCTAssertEqual(details, "Error Domain=NSOSStatusErrorDomain Code=-10814", - "the raw status text is preserved as opt-in copyable details") - XCTAssertTrue(workspace.openedWith.isEmpty, "a failed Open-With never falsely records a success") - } - - /// A surfaced headline must read as a human sentence — never a reflected enum dump or raw OS text. Mirrors - /// the needle set `FilesSeamsTests` pins on `FileActionError` headlines. - private func assertHeadlineIsClean(_ headline: String, - file: StaticString = #filePath, line: UInt = #line) { - XCTAssertFalse(headline.isEmpty, "headline is non-empty", file: file, line: line) - for needle in ["Domain=", "Code=", "Error Domain", "UserInfo", "FileActionError", - "NSCocoaErrorDomain", "NSOSStatusErrorDomain"] { - XCTAssertFalse(headline.contains(needle), - "headline must not contain raw error text (\(needle)): \(headline)", - file: file, line: line) - } - } -} diff --git a/Tests/ThreeFingerSwitcherTests/FilesActionMenuTests.swift b/Tests/ThreeFingerSwitcherTests/FilesActionMenuTests.swift deleted file mode 100644 index cd1105a..0000000 --- a/Tests/ThreeFingerSwitcherTests/FilesActionMenuTests.swift +++ /dev/null @@ -1,160 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Unit tests for the pure Files action-menu model (`Files/FilesActionMenu.swift`) and the keep-both -/// Paste-into name resolver — the `files-action-menu` defaults, per-type independence, the runtime -/// visibility rules (pasteInto gating, terminal expansion), and the conflict-free rename. -final class FilesActionMenuTests: XCTestCase { - - // MARK: - Fixtures - - private func file(_ path: String) -> FileEntry { - FileEntry(url: URL(fileURLWithPath: path), name: (path as NSString).lastPathComponent, - isDirectory: false, modificationDate: nil, kind: .other) - } - private func folder(_ path: String) -> FileEntry { - FileEntry(url: URL(fileURLWithPath: path), name: (path as NSString).lastPathComponent, - isDirectory: true, modificationDate: nil, kind: .folder) - } - private func term(_ id: String, _ name: String, enabled: Bool = true) -> FilesTool { - FilesTool(bundleID: id, name: name, role: .terminal, enabled: enabled) - } - - // MARK: - Defaults match the spec exactly - - func testDefaultMenusMatchSpec() { - XCTAssertEqual(FilesActionMenu.defaultFileItems, - [.copyAsPath, .copy, .cut, .pasteInto, .openIn, .delete], - "file default: Copy as path · Copy · Cut · Paste · Open in · Delete (Delete last)") - XCTAssertEqual(FilesActionMenu.defaultFolderItems, - [.copyAsPath, .copy, .cut, .pasteInto, .openInTerminals, .openIn, .delete], - "folder default adds the terminals group before Open in; Cut after Copy, Delete last") - XCTAssertEqual(FilesActionMenu.default.fileItems, FilesActionMenu.defaultFileItems) - XCTAssertEqual(FilesActionMenu.default.folderItems, FilesActionMenu.defaultFolderItems) - } - - func testItemsForTypeSelectsTheRightList() { - let menu = FilesActionMenu.default - XCTAssertEqual(menu.items(forFolder: false), FilesActionMenu.defaultFileItems) - XCTAssertEqual(menu.items(forFolder: true), FilesActionMenu.defaultFolderItems) - } - - // MARK: - Visible rows: file vs folder, pasteInto gating, terminal expansion - - func testFileRowsOmitTerminalsAndExpandNothing() { - let rows = FilesActionMenu.default.visibleRows( - for: file("/tmp/a.txt"), pasteboardHasFile: true, - terminals: [term("com.apple.Terminal", "Terminal")], editors: []) - XCTAssertEqual(rows, [.action(.copyAsPath), .action(.copy), .action(.cut), - .action(.pasteInto), .action(.openIn), .action(.delete)]) - } - - func testFolderRowsExpandOneRowPerEnabledTerminal() { - let rows = FilesActionMenu.default.visibleRows( - for: folder("/tmp/dir"), pasteboardHasFile: true, - terminals: [term("com.apple.Terminal", "Terminal"), term("com.googlecode.iterm2", "iTerm")], - editors: []) - XCTAssertEqual(rows, [ - .action(.copyAsPath), .action(.copy), .action(.cut), .action(.pasteInto), - .tool(.openInTerminals, term("com.apple.Terminal", "Terminal")), - .tool(.openInTerminals, term("com.googlecode.iterm2", "iTerm")), - .action(.openIn), .action(.delete) - ]) - } - - func testDisabledTerminalsAreFilteredOut() { - let rows = FilesActionMenu.default.visibleRows( - for: folder("/tmp/dir"), pasteboardHasFile: false, - terminals: [term("com.apple.Terminal", "Terminal", enabled: false), - term("com.googlecode.iterm2", "iTerm")], - editors: []) - let toolRows = rows.filter { if case .tool = $0 { return true } else { return false } } - XCTAssertEqual(toolRows, [.tool(.openInTerminals, term("com.googlecode.iterm2", "iTerm"))]) - } - - func testEmptyTerminalsVanishEntirely() { - let rows = FilesActionMenu.default.visibleRows( - for: folder("/tmp/dir"), pasteboardHasFile: true, terminals: [], editors: []) - XCTAssertFalse(rows.contains { if case .tool = $0 { return true } else { return false } }, - "no terminals installed → no terminal rows") - } - - func testPasteIntoHiddenWithoutFileOnPasteboard() { - let rows = FilesActionMenu.default.visibleRows( - for: file("/tmp/a.txt"), pasteboardHasFile: false, terminals: [], editors: []) - XCTAssertFalse(rows.contains(.action(.pasteInto)), "Paste is hidden when the pasteboard holds no file") - XCTAssertEqual(rows, [.action(.copyAsPath), .action(.copy), .action(.cut), - .action(.openIn), .action(.delete)]) - } - - // MARK: - Cut / Delete (file operations) - - func testCutAndDeleteAreDefaultForBothTypesAsPlainRows() { - // Both are catalog defaults for files and folders, pass through as plain `.action` rows (no grouping - // or visibility special-casing), and Delete is ordered last (set apart from the everyday actions). - XCTAssertTrue(FilesActionMenu.defaultFileItems.contains(.cut)) - XCTAssertTrue(FilesActionMenu.defaultFileItems.contains(.delete)) - XCTAssertTrue(FilesActionMenu.defaultFolderItems.contains(.cut)) - XCTAssertTrue(FilesActionMenu.defaultFolderItems.contains(.delete)) - XCTAssertEqual(FilesActionMenu.defaultFileItems.last, .delete, "Delete is last") - XCTAssertEqual(FilesActionMenu.defaultFolderItems.last, .delete, "Delete is last") - XCTAssertTrue(FilesMenuAction.defaultCatalog.contains(.cut)) - XCTAssertTrue(FilesMenuAction.defaultCatalog.contains(.delete)) - - let fileRows = FilesActionMenu.default.visibleRows( - for: file("/tmp/a.txt"), pasteboardHasFile: false, terminals: [], editors: []) - XCTAssertTrue(fileRows.contains(.action(.cut))) - XCTAssertTrue(fileRows.contains(.action(.delete))) - } - - func testCutAndDeleteSurviveCodableRoundTrip() throws { - let menu = FilesActionMenu(fileItems: [.cut, .delete], folderItems: [.delete, .cut]) - let data = try JSONEncoder().encode(menu) - XCTAssertEqual(try JSONDecoder().decode(FilesActionMenu.self, from: data), menu) - } - - // MARK: - Per-type independence - - func testEditingFolderMenuLeavesFileMenuUntouched() { - var menu = FilesActionMenu.default - menu.folderItems = [.openIn] - XCTAssertEqual(menu.fileItems, FilesActionMenu.defaultFileItems, "file menu unaffected by folder edit") - XCTAssertEqual(menu.items(forFolder: true), [.openIn]) - } - - // MARK: - Codable round-trip (the persistence shape) - - func testCodableRoundTrip() throws { - var menu = FilesActionMenu.default - menu.fileItems = [.addToFavorites, .copyAsPath, .openIn] - let data = try JSONEncoder().encode(menu) - XCTAssertEqual(try JSONDecoder().decode(FilesActionMenu.self, from: data), menu) - } - - // MARK: - Keep-both Paste-into name resolution - - func testUniqueNameReturnsDesiredWhenFree() { - XCTAssertEqual(FilesPasteName.uniqueName(for: "report.pdf", existing: []), "report.pdf") - XCTAssertEqual(FilesPasteName.uniqueName(for: "report.pdf", existing: ["other.pdf"]), "report.pdf") - } - - func testUniqueNameFirstCollisionAppendsCopyPreservingExtension() { - XCTAssertEqual(FilesPasteName.uniqueName(for: "report.pdf", existing: ["report.pdf"]), "report copy.pdf") - } - - func testUniqueNameRepeatedCollisionsCount() { - let existing: Set = ["report.pdf", "report copy.pdf", "report copy 2.pdf"] - XCTAssertEqual(FilesPasteName.uniqueName(for: "report.pdf", existing: existing), "report copy 3.pdf") - } - - func testUniqueNameNoExtension() { - XCTAssertEqual(FilesPasteName.uniqueName(for: "Makefile", existing: ["Makefile"]), "Makefile copy") - XCTAssertEqual(FilesPasteName.uniqueName(for: "Makefile", existing: ["Makefile", "Makefile copy"]), - "Makefile copy 2") - } - - func testUniqueNameFolderName() { - // A folder (no extension) collides like an extensionless file. - XCTAssertEqual(FilesPasteName.uniqueName(for: "Projects", existing: ["Projects"]), "Projects copy") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/FilesActionSettingsTests.swift b/Tests/ThreeFingerSwitcherTests/FilesActionSettingsTests.swift deleted file mode 100644 index a2ecfb0..0000000 --- a/Tests/ThreeFingerSwitcherTests/FilesActionSettingsTests.swift +++ /dev/null @@ -1,63 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Persistence + defaults + reset for the new Files action settings (`tunable-settings`): the lift action, -/// the per-type action-menu lists, and the curated-tools allow-list — mirroring the `GestureBindings` -/// persistence tests' UserDefaults-suite pattern. -@MainActor -final class FilesActionSettingsTests: XCTestCase { - - private func makeSettings() -> (AppSettings, String, UserDefaults) { - let suite = "ThreeFingerSwitcherTests.FilesActions.\(UUID().uuidString)" - let defaults = UserDefaults(suiteName: suite)! - return (AppSettings(defaults: defaults), suite, defaults) - } - - func testDefaultsMatchSpec() { - let (settings, suite, defaults) = makeSettings() - defer { defaults.removePersistentDomain(forName: suite) } - XCTAssertEqual(settings.filesLiftAction, .deliver, "lift defaults to deliver") - XCTAssertEqual(settings.filesActionMenu, .default, "menus default to the specified lists") - XCTAssertEqual(settings.filesToolsDisabled, [], "all detected tools enabled by default") - } - - func testFilesActionSettingsPersistAcrossInstances() { - let (writer, suite, defaults) = makeSettings() - defer { defaults.removePersistentDomain(forName: suite) } - - writer.filesLiftAction = .open - writer.filesActionMenu.fileItems = [.addToFavorites, .copyAsPath, .openIn] - writer.filesToolsDisabled = ["com.googlecode.iterm2"] - - let reader = AppSettings(defaults: defaults) - XCTAssertEqual(reader.filesLiftAction, .open, "lift action survived the reload") - XCTAssertEqual(reader.filesActionMenu.fileItems, [.addToFavorites, .copyAsPath, .openIn], - "the customized file menu survived the reload") - XCTAssertEqual(reader.filesToolsDisabled, ["com.googlecode.iterm2"], "the tool curation survived") - } - - func testResetRestoresFilesActionDefaults() { - let (settings, suite, defaults) = makeSettings() - defer { defaults.removePersistentDomain(forName: suite) } - - settings.filesLiftAction = .open - settings.filesActionMenu.folderItems = [.openIn] - settings.filesToolsDisabled = ["com.apple.Terminal"] - - settings.resetToDefaults() - - XCTAssertEqual(settings.filesLiftAction, .deliver, "reset restores the deliver lift") - XCTAssertEqual(settings.filesActionMenu, .default, "reset restores the default menus") - XCTAssertEqual(settings.filesToolsDisabled, [], "reset re-enables all tools") - } - - func testMissingMenuBlobFallsBackToDefault() { - let (settings, suite, defaults) = makeSettings() - defer { defaults.removePersistentDomain(forName: suite) } - // A garbage blob under the menu key must decode to the default, not crash. - defaults.set(Data("not json".utf8), forKey: "filesActionMenu") - let reader = AppSettings(defaults: defaults) - XCTAssertEqual(reader.filesActionMenu, .default, "an unreadable menu blob falls back to the default") - _ = settings - } -} diff --git a/Tests/ThreeFingerSwitcherTests/FilesBandBuilderTests.swift b/Tests/ThreeFingerSwitcherTests/FilesBandBuilderTests.swift deleted file mode 100644 index 331bd78..0000000 --- a/Tests/ThreeFingerSwitcherTests/FilesBandBuilderTests.swift +++ /dev/null @@ -1,113 +0,0 @@ -import XCTest -import Foundation -@testable import ThreeFingerSwitcherCore - -/// Tests the synthetic **Files band** projection (change: files-band, tasks 4.1/4.2/5.1): -/// `FilesBandBuilder` turning a directory column's `[FileEntry]` into a `ContextBand` of `.fileEntry` -/// `LaunchItem`s. The load-bearing guarantee is **path-stable item identity** — a `LaunchItem.id` is a -/// `UUID` but a `FileEntry.id` is a path `String`, so the builder must derive a *deterministic* UUID from -/// the path or the selection highlight strobes on every re-list (design D2). Like the foundation tests, -/// this is AppKit-free and not `@MainActor` (the builder never touches `FileManager` or AppKit). -final class FilesBandBuilderTests: XCTestCase { - - // MARK: - Fixtures - - private func fileEntry(_ path: String, isDirectory: Bool = false, kind: FileKind = .other, - mod: Date? = nil) -> FileEntry { - FileEntry(url: URL(fileURLWithPath: path), name: URL(fileURLWithPath: path).lastPathComponent, - isDirectory: isDirectory, modificationDate: mod, kind: kind) - } - - // MARK: - Band shape & sentinel - - func testBuildCarriesTheSentinelNameAndIcon() { - let band = FilesBandBuilder.build(currentColumn: []) - XCTAssertEqual(band.id, FilesBandBuilder.bandID) - XCTAssertEqual(band.name, "Files") - XCTAssertEqual(band.icon, .sfSymbol("folder.fill")) - XCTAssertTrue(band.items.isEmpty) - } - - func testIsFilesBandMatchesOnlyTheSentinel() { - XCTAssertTrue(FilesBandBuilder.isFilesBand(FilesBandBuilder.build(currentColumn: []))) - // A band with any other id is not the Files band — including the other synthetic bands. - XCTAssertFalse(FilesBandBuilder.isFilesBand(ContextBand(name: "x", color: FilesBandBuilder.color))) - XCTAssertFalse(FilesBandBuilder.isFilesBand(ClipboardBandBuilder.build(from: []))) - XCTAssertFalse(FilesBandBuilder.isFilesBand(AIBand.band(from: []))) - } - - func testSentinelIsDistinctFromTheOtherSyntheticBands() { - XCTAssertNotEqual(FilesBandBuilder.bandID, ClipboardBandBuilder.bandID) - XCTAssertNotEqual(FilesBandBuilder.bandID, AIBand.bandID) - } - - // MARK: - Entry → item mapping - - func testEachEntryBecomesAFileEntryItemPreservingOrderAndTitle() { - let entries = [ - fileEntry("/Home/Docs", isDirectory: true, kind: .folder), - fileEntry("/Home/photo.png", kind: .image), - fileEntry("/Home/a.txt", kind: .text), - ] - let band = FilesBandBuilder.build(currentColumn: entries) - XCTAssertEqual(band.items.count, 3) - XCTAssertEqual(band.items.map(\.title), ["Docs", "photo.png", "a.txt"]) - for (item, entry) in zip(band.items, entries) { - guard case let .fileEntry(carried) = item.kind else { - return XCTFail("a Files-band item must carry .fileEntry, got \(item.kind)") - } - XCTAssertEqual(carried, entry, "the item must carry the exact source entry") - } - } - - func testItemIconIsTheKindGlyph() { - let band = FilesBandBuilder.build(currentColumn: [fileEntry("/Home/clip.mp4", kind: .video)]) - XCTAssertEqual(band.items.first?.icon, FilesBandBuilder.glyph(for: .video)) - XCTAssertEqual(band.items.first?.icon, .sfSymbol("film")) - } - - // MARK: - Path-stable identity (the anti-strobe guarantee, design D2) - - func testReListingTheSamePathYieldsTheSameItemID() { - // Re-entry / a changed file → a fresh FileEntry with the same path but new metadata. The derived - // LaunchItem.id MUST be identical so the SwiftUI selection keeps its target and never strobes. - let path = "/Home/Docs/a.txt" - let first = FilesBandBuilder.item(for: fileEntry(path, kind: .text, - mod: Date(timeIntervalSince1970: 0))) - let relisted = FilesBandBuilder.item(for: fileEntry(path, kind: .text, - mod: Date(timeIntervalSince1970: 9_999))) - XCTAssertEqual(first.id, relisted.id) - // …and the rebuilt band agrees, item-for-item, across two independent projections of the column. - let a = FilesBandBuilder.build(currentColumn: [fileEntry(path, kind: .text)]) - let b = FilesBandBuilder.build(currentColumn: [fileEntry(path, kind: .text)]) - XCTAssertEqual(a.items.map(\.id), b.items.map(\.id)) - } - - func testDistinctPathsYieldDistinctItemIDs() { - let ids = FilesBandBuilder.build(currentColumn: [ - fileEntry("/Home/a.txt"), fileEntry("/Home/b.txt"), fileEntry("/Home/Docs/a.txt"), - ]).items.map(\.id) - XCTAssertEqual(Set(ids).count, 3, "different paths must not collide onto one UUID") - } - - func testUUIDForPathIsDeterministicAndPathSensitive() { - XCTAssertEqual(FilesBandBuilder.uuid(forPath: "/Home/a.txt"), - FilesBandBuilder.uuid(forPath: "/Home/a.txt")) - XCTAssertNotEqual(FilesBandBuilder.uuid(forPath: "/Home/a.txt"), - FilesBandBuilder.uuid(forPath: "/Home/A.txt")) - } - - // MARK: - Glyphs - - func testGlyphCoversEveryFileKind() { - // Every kind maps to a concrete, non-empty SF Symbol — folders never fall through to the generic - // document glyph, so a Files row's icon always reads as its kind. - for kind in FileKind.allCases { - guard case let .sfSymbol(name) = FilesBandBuilder.glyph(for: kind) else { - return XCTFail("\(kind) must map to an .sfSymbol glyph") - } - XCTAssertFalse(name.isEmpty, "\(kind) glyph name must be non-empty") - } - XCTAssertEqual(FilesBandBuilder.glyph(for: .folder), .sfSymbol("folder.fill")) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/FilesColumnRoutingTests.swift b/Tests/ThreeFingerSwitcherTests/FilesColumnRoutingTests.swift deleted file mode 100644 index c4720cd..0000000 --- a/Tests/ThreeFingerSwitcherTests/FilesColumnRoutingTests.swift +++ /dev/null @@ -1,413 +0,0 @@ -import XCTest -import Foundation -@testable import ThreeFingerSwitcherCore - -/// Tests the Files-band *integration* into `LauncherModel` (change: files-band, tasks 6.1–6.3): the -/// `currentBandIsFiles` gating, horizontal drilling (descend / ascend) reprojecting the band's items, -/// vertical highlight motion (clamping at the top — no search), remembered-location persistence and the -/// restore-on-re-entry toggle, and the descending sort's folders-first preservation — plus the -/// `FilesColumnController`'s sync-model ⇄ cache bridge in isolation. -/// -/// Determinism: every test drives a `FilesColumnController` over a **fully-seeded fixture cache** (the -/// whole tree handed in at init), so the controller never touches `FileManager` and never has to await an -/// async listing — the synchronous navigation model reads the warm cache immediately. This mirrors how -/// `FilesNavigationModelTests` injects a fixture lister; here the fixture is the cache the bridge owns. -@MainActor -final class FilesColumnRoutingTests: XCTestCase { - - // MARK: - Fixture tree - - private func entry(_ path: String, dir: Bool, kind: FileKind, mod: Date? = nil) -> FileEntry { - FileEntry(url: URL(fileURLWithPath: path), name: URL(fileURLWithPath: path).lastPathComponent, - isDirectory: dir, modificationDate: mod, kind: kind) - } - - /// The standard two-root tree (same shape as `FilesNavigationModelTests`): - /// /Home → Docs/, photo.png - /// /Home/Docs → Sub/, a.txt - /// /Home/Docs/Sub → deep.txt - /// /Work → notes.md - private func standardTree() -> [String: [FileEntry]] { - [ - "/Home": [ - entry("/Home/Docs", dir: true, kind: .folder), - entry("/Home/photo.png", dir: false, kind: .image), - ], - "/Home/Docs": [ - entry("/Home/Docs/Sub", dir: true, kind: .folder), - entry("/Home/Docs/a.txt", dir: false, kind: .text), - ], - "/Home/Docs/Sub": [ - entry("/Home/Docs/Sub/deep.txt", dir: false, kind: .text), - ], - "/Work": [ - entry("/Work/notes.md", dir: false, kind: .text), - ], - ] - } - - private var homeRoot: URL { URL(fileURLWithPath: "/Home") } - private var workRoot: URL { URL(fileURLWithPath: "/Work") } - - /// A controller seeded with the whole tree (no live FS, no async round-trip), landing on the roots list. - /// `record` collects the `(path, root)` pairs the controller asks to persist (the model's remembered sink). - private func makeController(remembered: [URL: URL] = [:], - direction: FilesSortDirection = .ascending) -> FilesColumnController { - FilesColumnController(roots: [homeRoot, workRoot], - remembered: remembered, - sortOrder: .name, - sortDirection: direction, - seededCache: standardTree()) - } - - /// A `LauncherModel` carrying a normal band 0 plus a Files band at index 1, with the controller wired in. - /// Mirrors how the wiring layer will call `setBands` (the Files band built from the controller's column). - private func makeModelWithFilesBand( - controller: FilesColumnController, - onRemember: ((_ path: String, _ root: String) -> Void)? = nil - ) -> LauncherModel { - let model = LauncherModel() - model.onFilesRememberLocation = onRemember - let appBand = [LaunchItem(title: "A0", icon: .appDefault, - kind: .app(bundleURL: URL(fileURLWithPath: "/Applications/A0.app"), strategy: nil))] - let filesItems = FilesBandBuilder.build(currentColumn: controller.visibleEntries).items - model.setBands([appBand, filesItems], - names: ["Dev", FilesBandBuilder.name], - colors: [ItemColor(red: 0, green: 0, blue: 1), FilesBandBuilder.color], - startBand: 1, column: 0, - filesBandIndex: 1, filesColumn: controller) - return model - } - - /// Drive the model onto the Files band's grid (it starts on the band list since there are two bands): - /// RIGHT crosses into the grid, where the Files band's directory drill takes over. - private func enterFilesGrid(_ model: LauncherModel) { - XCTAssertEqual(model.currentBand, 1) - XCTAssertEqual(model.focus, .bands, "two bands → lands on the band list") - model.stepHorizontal(1) // band list → grid (enter the Files column) - XCTAssertEqual(model.focus, .grid) - } - - // MARK: - Gating - - func testCurrentBandIsFilesGating() { - let controller = makeController() - let model = makeModelWithFilesBand(controller: controller) - XCTAssertTrue(model.currentBandIsFiles, "the active band (index 1) is the Files band") - XCTAssertNotNil(model.filesColumn) - // Switch to band 0 (up the list, previous band) and the gate flips off. - model.stepVertical(1) - XCTAssertEqual(model.currentBand, 0) - XCTAssertFalse(model.currentBandIsFiles) - } - - func testNoFilesBandWhenNotConfigured() { - let model = LauncherModel() - model.setBands([[LaunchItem(title: "A0", icon: .appDefault, - kind: .app(bundleURL: URL(fileURLWithPath: "/Applications/A0.app"), strategy: nil))]], - names: ["Dev"], colors: [ItemColor(red: 0, green: 0, blue: 1)], - startBand: 0, column: 0) - XCTAssertNil(model.filesBandIndex) - XCTAssertNil(model.filesColumn) - XCTAssertFalse(model.currentBandIsFiles) - } - - // MARK: - Focus-aware crossing & the drill-engaged predicate (refinements 1 + 2) - - /// On the band RAIL (`focus == .bands`) the Files band is current but the drill is NOT engaged — a lift - /// there must dismiss like any other band, so `filesDrillEngaged` (which gates `filesDrillActive`) is - /// false until focus crosses INTO the column. - func testFilesDrillEngagedIsFalseOnTheBandRail() { - let controller = makeController() - let model = makeModelWithFilesBand(controller: controller) - XCTAssertEqual(model.focus, .bands, "two bands → lands on the Files band icon (the rail)") - XCTAssertTrue(model.currentBandIsFiles) - XCTAssertFalse(model.filesDrillEngaged, - "the drill is NOT engaged while resting on the band icon — a lift dismisses") - } - - /// A horizontal step toward the grid CROSSES focus `.bands` → `.grid` and engages the drill, WITHOUT - /// descending — it lands on the column the navigator already displays (the roots list here), not a level - /// deeper. Descend/ascend only apply once `focus == .grid`. - func testHorizontalAtBandsCrossesToGridWithoutDescending() { - let controller = makeController() - let model = makeModelWithFilesBand(controller: controller) - XCTAssertEqual(controller.current, .roots) - model.stepHorizontal(1) // band rail → grid (a pure focus cross, no descend) - XCTAssertEqual(model.focus, .grid, "the horizontal step crossed into the file column") - XCTAssertTrue(model.filesDrillEngaged, "the drill engages once focus is in the column") - XCTAssertEqual(controller.current, .roots, "crossing in did NOT descend — still the displayed column") - XCTAssertEqual(model.items.map(\.title), ["Home", "Work"], "the displayed roots column is unchanged") - XCTAssertEqual(model.selectedIndex, 0, "lands at the top of the column (the navigator's highlight)") - } - - /// The descend happens only on a SECOND horizontal step — the first crossed focus in (no descend), the - /// second (now at `.grid`) descends. This is the "descend only at `.grid`" invariant. - func testDescendOnlyHappensOnceFocusIsGrid() { - let controller = makeController() - let model = makeModelWithFilesBand(controller: controller) - model.stepHorizontal(1) // cross to .grid (no descend) - XCTAssertEqual(controller.current, .roots) - model.stepHorizontal(1) // now at .grid → descend into the highlighted root (Home) - XCTAssertEqual(controller.current, .folder(homeRoot), "the second step (at .grid) descends") - XCTAssertTrue(model.filesDrillEngaged) - XCTAssertEqual(model.items.map(\.title), ["Docs", "photo.png"]) - } - - /// A vertical step on the band rail switches bands and never descends — the drill stays disengaged the - /// whole time, and switching off the Files band disengages it (so `filesDrillActive` releases). - func testVerticalAtBandsSwitchesBandsAndNeverDescends() { - let controller = makeController() - let model = makeModelWithFilesBand(controller: controller) - model.stepVertical(1) // up the rail → previous band (band 0) - XCTAssertEqual(model.currentBand, 0, "vertical on the rail switched bands") - XCTAssertEqual(controller.current, .roots, "switching bands never descended the Files column") - XCTAssertFalse(model.filesDrillEngaged, "off the Files band → the drill is disengaged") - } - - /// With restore-at-open ON, the navigator OPENS displaying the remembered deep folder; crossing in from - /// the icon lands exactly there (the displayed state and the landing MATCH — no jump). A single right - /// step crosses focus; it does NOT then descend a further level past the restored folder. - func testCrossingInLandsOnTheRestoredFolderWithNoJump() { - // A controller that restores last-location to /Home/Docs on open (refinement 2). - let controller = FilesColumnController(roots: [homeRoot, workRoot], - remembered: [homeRoot: URL(fileURLWithPath: "/Home/Docs")], - sortOrder: .name, - sortDirection: .ascending, - restoreLastLocation: true, - seededCache: standardTree()) - XCTAssertEqual(controller.current, .folder(URL(fileURLWithPath: "/Home/Docs")), - "restore-at-open lands the navigator on the remembered folder before any crossing") - let model = makeModelWithFilesBand(controller: controller) - // The band already DISPLAYS the restored folder's contents while still on the icon. - XCTAssertEqual(model.items.map(\.title), ["Sub", "a.txt"], - "the column shows the restored folder while on the band icon") - model.stepHorizontal(1) // cross in - XCTAssertEqual(model.focus, .grid) - XCTAssertEqual(controller.current, .folder(URL(fileURLWithPath: "/Home/Docs")), - "crossing in landed on the SAME displayed folder — no jump deeper") - XCTAssertEqual(model.items.map(\.title), ["Sub", "a.txt"], "the displayed state and the landing match") - XCTAssertEqual(model.selectedIndex, 0, "at the top of the restored column") - } - - // MARK: - Top-of-column up clamps (no search to overflow into) - - /// An up-step at the top of the column is a pure clamp now: the highlight stays put and nothing else - /// happens (no search field to focus, no depth change). - func testUpAtTopOfColumnClamps() { - let controller = makeController() - let model = makeModelWithFilesBand(controller: controller) - enterFilesGrid(model) - model.stepHorizontal(1) // → Home, highlight at the top (index 0) - model.stepVertical(1) // up while already at the top → clamp, no-op - XCTAssertEqual(model.selectedIndex, 0, "the highlight stays put on a top-of-column up-step") - XCTAssertEqual(controller.current, .folder(homeRoot), "no depth change, no side effect") - } - - // MARK: - Horizontal drill (descend / ascend) reprojects the band - - func testHorizontalRightDescendsAndReprojects() { - let controller = makeController() - let model = makeModelWithFilesBand(controller: controller) - enterFilesGrid(model) - // Roots column: [Home, Work]; highlight is Home. - XCTAssertEqual(model.items.map(\.title), ["Home", "Work"]) - model.stepHorizontal(1) // descend into Home - XCTAssertEqual(controller.current, .folder(homeRoot)) - XCTAssertEqual(model.items.map(\.title), ["Docs", "photo.png"], - "the band's items are reprojected from the new column") - XCTAssertEqual(model.selectedIndex, 0, "descend resets the highlight to the top") - // Descend again into Docs. - model.stepHorizontal(1) - XCTAssertEqual(controller.current, .folder(URL(fileURLWithPath: "/Home/Docs"))) - XCTAssertEqual(model.items.map(\.title), ["Sub", "a.txt"]) - } - - func testHorizontalLeftAscendsAndReprojects() { - let controller = makeController() - let model = makeModelWithFilesBand(controller: controller) - enterFilesGrid(model) - model.stepHorizontal(1) // → Home - model.stepHorizontal(1) // → Home/Docs - XCTAssertEqual(model.items.map(\.title), ["Sub", "a.txt"]) - model.stepHorizontal(-1) // ascend → Home - XCTAssertEqual(controller.current, .folder(homeRoot)) - XCTAssertEqual(model.items.map(\.title), ["Docs", "photo.png"]) - XCTAssertEqual(model.selectedItem?.title, "Docs", - "ascend re-highlights the folder we came up from") - } - - func testHorizontalLeftAtRootsCrossesBackToBandList() { - let controller = makeController() - let model = makeModelWithFilesBand(controller: controller) - enterFilesGrid(model) - // On the roots list, ascend can't go further → a LEFT crosses back to the band list (band-0 escape). - XCTAssertFalse(controller.canAscend) - model.stepHorizontal(-1) - XCTAssertEqual(model.focus, .bands, "left at the roots list returns to the band list") - XCTAssertEqual(model.currentBand, 1, "the Files band stays active") - XCTAssertEqual(controller.current, .roots, "no spurious ascend happened") - } - - func testHorizontalRightOnAFileDoesNotDescend() { - let controller = makeController() - let model = makeModelWithFilesBand(controller: controller) - enterFilesGrid(model) - model.stepHorizontal(1) // → Home ([Docs, photo.png]) - model.stepVertical(-1) // highlight down → photo.png (a file) - XCTAssertEqual(model.selectedItem?.title, "photo.png") - model.stepHorizontal(1) // descend on a file is a no-op in the navigator - XCTAssertEqual(controller.current, .folder(homeRoot), "a file doesn't descend") - XCTAssertEqual(model.items.map(\.title), ["Docs", "photo.png"]) - } - - // MARK: - Vertical highlight (with inversion already applied upstream) - - func testVerticalMovesHighlightDownThenUp() { - let controller = makeController() - let model = makeModelWithFilesBand(controller: controller) - enterFilesGrid(model) - model.stepHorizontal(1) // → Home ([Docs, photo.png]), highlight Docs (0) - XCTAssertEqual(model.selectedIndex, 0) - model.stepVertical(-1) // down → index 1 (photo.png) - XCTAssertEqual(model.selectedIndex, 1) - XCTAssertEqual(controller.highlightedEntry?.name, "photo.png") - model.stepVertical(1) // up → back to index 0 (Docs) - XCTAssertEqual(model.selectedIndex, 0) - XCTAssertEqual(controller.highlightedEntry?.name, "Docs") - } - - func testVerticalDownClampsAtTheBottom() { - let controller = makeController() - let model = makeModelWithFilesBand(controller: controller) - enterFilesGrid(model) - model.stepHorizontal(1) // → Home (2 entries) - model.stepVertical(-1) // → index 1 - model.stepVertical(-1) // already last → clamp - XCTAssertEqual(model.selectedIndex, 1, "down clamps at the last row") - } - - // MARK: - Remembered-location persistence on a depth change - - func testDepthChangePersistsRememberedLocation() { - let controller = makeController() - var recorded: [(path: String, root: String)] = [] - let model = makeModelWithFilesBand(controller: controller) { path, root in - recorded.append((path, root)) - } - enterFilesGrid(model) - model.stepHorizontal(1) // descend into Home → a depth change persists - XCTAssertTrue(recorded.contains { $0.path == "/Home" && $0.root == "/Home" }, - "descending into a root records it as that root's remembered deepest location") - model.stepHorizontal(1) // descend into Docs - XCTAssertTrue(recorded.contains { $0.path == "/Home/Docs" && $0.root == "/Home" }, - "descending deeper updates the remembered location under the owning root") - } - - func testHorizontalThatDoesNotChangeDepthDoesNotPersist() { - let controller = makeController() - var recorded: [(path: String, root: String)] = [] - let model = makeModelWithFilesBand(controller: controller) { path, root in - recorded.append((path, root)) - } - enterFilesGrid(model) - // A LEFT at the roots list crosses to the band list (no depth change) → nothing persisted. - model.stepHorizontal(-1) - XCTAssertTrue(recorded.isEmpty, "crossing back to the band list is not a depth change") - } - - func testReentryRestoresRememberedLocationWhenRestoreOn() { - // Restore ON: the controller opens deep (/Home/Docs); ascend back to roots, then re-descend into Home - // → `enterRoot` restores the remembered /Home/Docs again. - let controller = FilesColumnController(roots: [homeRoot, workRoot], - remembered: [homeRoot: URL(fileURLWithPath: "/Home/Docs")], - sortOrder: .name, - sortDirection: .ascending, - restoreLastLocation: true, - seededCache: standardTree()) - XCTAssertEqual(controller.current, .folder(URL(fileURLWithPath: "/Home/Docs")), "opens restored") - controller.ascend(); controller.ascend() // /Home/Docs → /Home → roots - XCTAssertEqual(controller.current, .roots) - controller.descend() // re-enter Home → restores /Home/Docs - XCTAssertEqual(controller.current, .folder(URL(fileURLWithPath: "/Home/Docs"))) - XCTAssertEqual(controller.visibleEntries.map(\.name), ["Sub", "a.txt"]) - } - - func testReentryDoesNotRestoreWhenRestoreOff() { - // The bug fix at the integration layer: restore OFF opens on the roots list, and descending into a - // root from the band lands on the root's TOP level — never the remembered deep folder. - let controller = makeController(remembered: [homeRoot: URL(fileURLWithPath: "/Home/Docs")]) // restore off - let model = makeModelWithFilesBand(controller: controller) - XCTAssertEqual(controller.current, .roots, "restore off → opens on roots") - enterFilesGrid(model) - model.stepHorizontal(1) // descend into Home → lands on the TOP, not the remembered /Home/Docs - XCTAssertEqual(controller.current, .folder(homeRoot), - "restore off → descending into a root never jumps to the last-visited folder") - XCTAssertEqual(model.items.map(\.title), ["Docs", "photo.png"]) - } - - // MARK: - Descending sort preserves folders-first - - func testDescendingSortPreservesFoldersFirst() { - // A column with two folders and two files, listed ascending folders-first, then flipped descending. - let ascending: [FileEntry] = [ - entry("/X/Alpha", dir: true, kind: .folder), - entry("/X/Beta", dir: true, kind: .folder), - entry("/X/apple.txt", dir: false, kind: .text), - entry("/X/banana.txt", dir: false, kind: .text), - ] - let descending = FilesColumnController.applyingDirection(.descending, to: ascending) - // Folders must still lead (not sink to the tail), but reversed within their own group; files too. - XCTAssertEqual(descending.map(\.name), ["Beta", "Alpha", "banana.txt", "apple.txt"]) - XCTAssertTrue(descending.prefix(2).allSatisfy(\.isDirectory), - "descending keeps folders-first — they do not move below the files") - // Ascending is the identity (the lister already produced folders-first ascending). - XCTAssertEqual(FilesColumnController.applyingDirection(.ascending, to: ascending).map(\.name), - ["Alpha", "Beta", "apple.txt", "banana.txt"]) - } - - func testSortFieldMapping() { - XCTAssertEqual(FilesColumnController.sortOrder(field: .name), .name) - XCTAssertEqual(FilesColumnController.sortOrder(field: .date), .dateModified) - XCTAssertEqual(FilesColumnController.sortOrder(field: .kind), .kind) - } - - // MARK: - The controller's cache bridge in isolation (async miss → warm → re-feed) - - func testAsyncListingWarmsTheCacheAndRefeeds() async { - // Build a controller with an EMPTY seed and an async fixture lister, so a descend misses the cache, - // kicks off the async listing, and the re-feed fills the column once it settles. - let tree = standardTree() - let lister: (URL, FilesSortOrder) async -> [FileEntry] = { url, _ in - tree[url.standardizedFileURL.path] ?? [] - } - let controller = FilesColumnController(roots: [homeRoot, workRoot], - remembered: [:], - sortOrder: .name, - sortDirection: .ascending, - seededCache: [:], - lister: lister) - // The roots column is synthesized (no listing), so it's populated even before anything settles. - XCTAssertEqual(controller.visibleEntries.map(\.name), ["Home", "Work"], "roots need no listing") - // Descend into Home WITHOUT awaiting first: the init's preview-peek task for /Home hasn't run yet - // (no suspension point since init), so the cache is a genuine MISS and the column is empty for a beat. - controller.descend() - XCTAssertTrue(controller.visibleEntries.isEmpty, - "the synchronous model shows empty until the async listing lands") - await controller.settle() // drive the async bridge (and its peek cascade) to completion - XCTAssertEqual(controller.visibleEntries.map(\.name), ["Docs", "photo.png"], - "the landed listing was stored and re-fed into the current column") - XCTAssertNotNil(controller.cache["/Home"], "the completed listing is cached") - } - - func testPreviewFolderPeekFlowsThroughTheCache() { - // With the whole tree seeded, the highlighted folder's preview peek reads the cache (no live list). - let controller = makeController() - controller.descend() // → Home, highlight Docs (a folder) - guard case let .folder(folderEntry, contents)? = controller.previewTarget else { - return XCTFail("highlighting a folder yields a folder preview target") - } - XCTAssertEqual(folderEntry.name, "Docs") - XCTAssertEqual(contents.map(\.name), ["Sub", "a.txt"], "the peek's contents come from the cache") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/FilesDeliveryTests.swift b/Tests/ThreeFingerSwitcherTests/FilesDeliveryTests.swift deleted file mode 100644 index f8c28ff..0000000 --- a/Tests/ThreeFingerSwitcherTests/FilesDeliveryTests.swift +++ /dev/null @@ -1,34 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Unit tests for the pure delivery-payload builder (`Files/FilesDelivery.swift`) — the dual-representation -/// payload that `files-contextual-delivery` writes so a text target receives the path and Finder receives -/// the file. -final class FilesDeliveryTests: XCTestCase { - - func testPayloadCarriesBothRepresentations() { - let entry = FileEntry(url: URL(fileURLWithPath: "/Users/me/notes.txt"), name: "notes.txt", - isDirectory: false, modificationDate: nil, kind: .text) - let payload = FilesDelivery.payload(for: entry) - XCTAssertEqual(payload.url, URL(fileURLWithPath: "/Users/me/notes.txt").standardizedFileURL, - "the file reference is present for Finder") - XCTAssertEqual(payload.path, "/Users/me/notes.txt", "the path string is present for text targets") - } - - func testPayloadPathIsStandardized() { - // A non-standard path (a trailing component that standardizes away) is canonicalized. - let raw = URL(fileURLWithPath: "/Users/me/sub/../notes.txt") - let entry = FileEntry(url: raw, name: "notes.txt", isDirectory: false, modificationDate: nil, kind: .text) - let payload = FilesDelivery.payload(for: entry) - XCTAssertEqual(payload.path, "/Users/me/notes.txt", "the delivered path is the standardized absolute path") - XCTAssertEqual(payload.path, entry.id, "and equals the entry's stable id") - } - - func testFolderPayload() { - let entry = FileEntry(url: URL(fileURLWithPath: "/tmp/dir"), name: "dir", - isDirectory: true, modificationDate: nil, kind: .folder) - let payload = FilesDelivery.payload(for: entry) - XCTAssertEqual(payload.path, "/tmp/dir") - XCTAssertTrue(payload.url.isFileURL) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/FilesDrillRecognizerTests.swift b/Tests/ThreeFingerSwitcherTests/FilesDrillRecognizerTests.swift deleted file mode 100644 index e81dbd4..0000000 --- a/Tests/ThreeFingerSwitcherTests/FilesDrillRecognizerTests.swift +++ /dev/null @@ -1,402 +0,0 @@ -import XCTest -import CoreGraphics -@testable import ThreeFingerSwitcherCore - -/// Records the Files-drill intents (plus the switcher/launcher activation intents) so tests can assert -/// the recognizer's sustained modal sub-state emits depth/highlight/open/open-with/discard correctly and -/// — crucially — that while `filesDrillActive` a fresh contact never routes to the switcher or launcher. -@MainActor -private final class FilesDrillMockDelegate: GestureRecognizerDelegate { - enum Event: Equatable { - case depth(Int), highlight(Int), open, openWith, discard - case edge(Int, Int) // launcherEdgeChanged(dx,dy): the held-at-edge sign that drives auto-repeat - // Tripwires: these MUST stay empty while drilling (the latch is bypassed). - case sActivate, lActivate - } - private(set) var events: [Event] = [] - - // Files drill - func filesDepth(_ d: Int) { events.append(.depth(d)) } - func filesHighlight(_ d: Int) { events.append(.highlight(d)) } - func filesOpen() { events.append(.open) } - func filesOpenWith() { events.append(.openWith) } - func filesDiscard() { events.append(.discard) } - // The held-at-edge sign (depth/highlight border auto-repeat): the recognizer publishes it through the - // launcher edge delegate, and the controller turns it into edge-triggered auto-repeat. - func launcherEdgeChanged(dx: Int, dy: Int) { events.append(.edge(dx, dy)) } - - // Tripwires (must never fire during a drill) - func gestureDidActivate() { events.append(.sActivate) } - func launcherDidActivate() { events.append(.lActivate) } - // The remaining base switcher intents have no protocol default (only the launcher / files-drill - // intents do), so they must be stubbed. They are not asserted here — while drilling, the bypass - // means a fresh contact never re-latches the switcher, so these stay silent — so plain no-ops. - func gestureDidStep(_ direction: Int) {} - func gestureDidStepRow(_ direction: Int) {} - func gestureDidTriggerMissionControl(up: Bool) {} - func gestureDidCommit() {} - func gestureDidCancel() {} - // The launcher and files-drill intents not overridden above are left to the protocol's default no-ops. - - var depths: [Int] { events.compactMap { if case let .depth(d) = $0 { return d } else { return nil } } } - var highlights: [Int] { events.compactMap { if case let .highlight(d) = $0 { return d } else { return nil } } } - var lastEdgeDX: Int? { events.reversed().compactMap { if case let .edge(dx, _) = $0 { return dx } else { return nil } }.first } - var openCount: Int { events.filter { $0 == .open }.count } - var openWithCount: Int { events.filter { $0 == .openWith }.count } - var discardCount: Int { events.filter { $0 == .discard }.count } - var didSwitcherActivate: Bool { events.contains(.sActivate) } - var didLauncherActivate: Bool { events.contains(.lActivate) } -} - -@MainActor -final class FilesDrillRecognizerTests: XCTestCase { - - // MARK: - Fixture - - /// The drill navigation is the **odometer** (restored v0.11.0 model) on BOTH axes: `launcherStepDistance` - /// is the per-step travel distance, so a centroid Δ of one step emits one depth/highlight step (with - /// carry), and holding a contact at a trackpad edge sets the held sign so the controller auto-repeats — - /// uniform on both axes (depth auto-drills the folder tree). `launcherActivationThreshold` is the - /// four-finger discard-swipe threshold; `axisLockRatio` gates the discard to a horizontal-dominant - /// sweep. With step 0.1, a centroid Δ of 0.15 ≈ one step and 0.35 ≈ three steps. - private func makeSettings( - launcherActivationThreshold: Double = 0.045, - step: Double = 0.1, - axisLockRatio: Double = 1.4, - reverseDirection: Bool = false, - reverseVerticalDirection: Bool = false, - filesDrill: GestureBindings.FilesDrillBinding = .default - ) -> AppSettings { - let defaults = UserDefaults(suiteName: "ThreeFingerSwitcherTests.\(UUID().uuidString)")! - let settings = AppSettings(defaults: defaults) - settings.launcherActivationThreshold = launcherActivationThreshold - settings.launcherStepDistance = step // odometer step (travel per depth/highlight step) - settings.axisLockRatio = axisLockRatio - settings.reverseDirection = reverseDirection - settings.reverseVerticalDirection = reverseVerticalDirection - settings.gestureBindings.filesDrill = filesDrill // the configured drill resolution binding - return settings - } - - /// Builds a recognizer already in the drill sub-state (the controller flips the flag when the Files - /// navigator opens). `launcherEnabled` is left ON so the bypass tests prove the drill pre-empts the - /// four-finger launcher latch too. - private func makeDrillRecognizer(_ settings: AppSettings) -> (GestureRecognizer, FilesDrillMockDelegate) { - let delegate = FilesDrillMockDelegate() - let rec = GestureRecognizer(settings: settings) - rec.delegate = delegate - rec.launcherEnabled = true - rec.filesDrillActive = true - return (rec, delegate) - } - - private func feed(_ rec: GestureRecognizer, x: Double, y: Double, fingers: Int) { - rec.feed(TouchFrame(testFingerCount: fingers, centroid: CGPoint(x: x, y: y))) - } - - // MARK: - Depth (horizontal) / Highlight (vertical) - - func test_horizontalTravel_odometerDepth() { - // Depth is the odometer: a +0.35 sweep (≈3 steps) descends three levels with carry; moving back - // steps it back (ascends). The first ≥2-finger frame seeds the origin. - let (rec, d) = makeDrillRecognizer(makeSettings()) - feed(rec, x: 0.20, y: 0.50, fingers: 2) // seed (0.20, 0.50) - feed(rec, x: 0.55, y: 0.50, fingers: 2) // Δx +0.35 → +3 depth steps - XCTAssertEqual(d.depths, [1, 1, 1], "depth accumulates one step per step-distance of travel") - XCTAssertTrue(d.highlights.isEmpty, "pure horizontal emits no highlight steps") - feed(rec, x: 0.20, y: 0.50, fingers: 2) // Δx −0.35 → 3 depth steps back (ascend) - XCTAssertEqual(d.depths, [1, 1, 1, -1, -1, -1]) - } - - func test_horizontalHeldAtBorder_setsDepthHeldSign_forAutoRepeat() { - // Uniform auto-repeat: holding depth at the trackpad border sets the HORIZONTAL held sign, so the - // controller auto-drills (the border acceleration the user asked for). - let (rec, d) = makeDrillRecognizer(makeSettings()) - feed(rec, x: 0.50, y: 0.50, fingers: 2) // seed mid-trackpad - feed(rec, x: 0.96, y: 0.50, fingers: 2) // push into the right edge zone → depth held - XCTAssertEqual(d.lastEdgeDX, 1, "holding depth at the border sets the horizontal held sign") - XCTAssertFalse(d.depths.isEmpty, "and the depth stepped on the way out") - } - - func test_verticalTravel_odometerHighlight() { - // Highlight is the odometer too: a +0.35 vertical sweep moves the highlight three rows; moving - // back steps it back. - let (rec, d) = makeDrillRecognizer(makeSettings()) - feed(rec, x: 0.50, y: 0.20, fingers: 2) // seed (0.50, 0.20) - feed(rec, x: 0.50, y: 0.55, fingers: 2) // Δy +0.35 → +3 highlight steps - XCTAssertEqual(d.highlights, [1, 1, 1], "the highlight accumulates one step per step-distance") - XCTAssertTrue(d.depths.isEmpty, "pure vertical emits no depth steps") - feed(rec, x: 0.50, y: 0.20, fingers: 2) // Δy −0.35 → 3 highlight steps back - XCTAssertEqual(d.highlights, [1, 1, 1, -1, -1, -1]) - } - - func test_directionInversion_isHonored_onBothAxes() { - // The drill applies the SAME reverseDirection / reverseVerticalDirection as the launcher: a forward - // (+x / +y) push flips to -1 when inverted. - let (rec, d) = makeDrillRecognizer(makeSettings(reverseDirection: true, reverseVerticalDirection: true)) - feed(rec, x: 0.20, y: 0.20, fingers: 2) // seed - feed(rec, x: 0.35, y: 0.20, fingers: 2) // +x push (1 step), inverted → depth -1 - XCTAssertEqual(d.depths, [-1]) - feed(rec, x: 0.20, y: 0.20, fingers: 2) // back (steps the depth the other way) - feed(rec, x: 0.20, y: 0.35, fingers: 2) // +y push (1 step), inverted → highlight -1 - XCTAssertEqual(d.highlights, [-1]) - } - - func test_negativeTravel_emitsNegativeDepthSteps() { - // Leftward depth ascends: a −0.35 sweep → −3 steps. - let (rec, d) = makeDrillRecognizer(makeSettings()) - feed(rec, x: 0.80, y: 0.50, fingers: 3) // seed at three fingers (still the relaxed posture) - feed(rec, x: 0.45, y: 0.50, fingers: 3) // Δx −0.35 → −3 depth steps - XCTAssertEqual(d.depths, [-1, -1, -1]) - } - - // MARK: - Re-baseline on contact-count change (no phantom step) - - func test_contactCountChange_reBaselines_emitsNoPhantomStep() { - // A finger leaving shifts the centroid; without re-baselining that jump would read as a huge travel - // and fire spurious steps. Assert zero steps from the count change, then one step from a push. - let (rec, d) = makeDrillRecognizer(makeSettings()) - feed(rec, x: 0.20, y: 0.20, fingers: 3) // seed at three - feed(rec, x: 0.60, y: 0.80, fingers: 2) // drop to two AND a large jump → re-baseline, no step - XCTAssertTrue(d.depths.isEmpty, "the count-change centroid jump emits no depth step") - XCTAssertTrue(d.highlights.isEmpty, "the count-change centroid jump emits no highlight step") - feed(rec, x: 0.75, y: 0.80, fingers: 2) // Δx +0.15 from the NEW baseline → one depth step - XCTAssertEqual(d.depths, [1]) - XCTAssertTrue(d.highlights.isEmpty) - } - - func test_landingFinger_alsoReBaselines() { - // Symmetric to a leaving finger: a landing finger re-baselines too, so the centroid jump emits no step. - let (rec, d) = makeDrillRecognizer(makeSettings()) - feed(rec, x: 0.20, y: 0.50, fingers: 2) // seed at two - feed(rec, x: 0.55, y: 0.50, fingers: 3) // a finger lands; big jump → re-baseline, no step - XCTAssertTrue(d.depths.isEmpty) - feed(rec, x: 0.70, y: 0.50, fingers: 3) // Δx +0.15 from the new baseline → one depth step - XCTAssertEqual(d.depths, [1]) - } - - // MARK: - Relative +1 finger → Open-With on the lift - - func test_relativePlusOne_fromTwoToThree_liftEmitsOpenWith() { - let (rec, d) = makeDrillRecognizer(makeSettings()) - feed(rec, x: 0.50, y: 0.50, fingers: 2) // seed; relaxed baseline = 2 - feed(rec, x: 0.50, y: 0.50, fingers: 3) // +1 above baseline → latch Open-With (no step) - feed(rec, x: 0.50, y: 0.50, fingers: 0) // resolving lift - XCTAssertEqual(d.openWithCount, 1, "a relative +1 finger resolves Open-With") - XCTAssertEqual(d.openCount, 0) - } - - func test_relativePlusOne_baselineThreeToFour_isOpenWithNotDiscard() { - // The trigger is "a finger was added above the baseline", not "exactly three": a baseline of three - // rising to four is the Open-With morph; a plain lift from four (no swipe) resolves Open-With. - let (rec, d) = makeDrillRecognizer(makeSettings()) - feed(rec, x: 0.50, y: 0.50, fingers: 3) // seed; relaxed baseline = 3 - feed(rec, x: 0.50, y: 0.50, fingers: 4) // 3 → 4: relative +1, latch Open-With - feed(rec, x: 0.50, y: 0.50, fingers: 4) // hold four, no swipe → no discard - feed(rec, x: 0.50, y: 0.50, fingers: 0) // lift → Open-With - XCTAssertEqual(d.openWithCount, 1) - XCTAssertEqual(d.discardCount, 0, "four fingers WITHOUT a swipe-away is Open-With, not discard") - XCTAssertEqual(d.openCount, 0) - } - - func test_noAddedFinger_liftEmitsPlainOpen() { - let (rec, d) = makeDrillRecognizer(makeSettings()) - feed(rec, x: 0.20, y: 0.50, fingers: 2) // seed - feed(rec, x: 0.50, y: 0.50, fingers: 2) // navigate (depth steps), no finger added - feed(rec, x: 0.50, y: 0.50, fingers: 0) // resolving lift → plain Open - XCTAssertEqual(d.openCount, 1) - XCTAssertEqual(d.openWithCount, 0) - } - - // MARK: - Four-finger horizontal swipe → discard - - func test_fourFingerHorizontalSwipe_emitsDiscard() { - // A fresh deliberate four-finger horizontal swipe-away resolves discard — and does NOT spray depth - // steps on the way (the four-finger arming posture suppresses navigation). - let (rec, d) = makeDrillRecognizer(makeSettings()) - feed(rec, x: 0.50, y: 0.50, fingers: 4) // seed at four (the arming posture) - feed(rec, x: 0.20, y: 0.50, fingers: 4) // dx -0.30 ≥ 0.045, horizontal-dominant → discard - XCTAssertEqual(d.discardCount, 1) - XCTAssertTrue(d.depths.isEmpty, "the four-finger swipe-away emits no depth steps") - } - - func test_fourFingerDiscard_winsOverPendingOpenWith() { - // Relaxed baseline two → four is a relative +1 (latches Open-With), but a deliberate four-finger - // horizontal swipe resolves discard and the later lift emits nothing (one-shot). - let (rec, d) = makeDrillRecognizer(makeSettings()) - feed(rec, x: 0.50, y: 0.50, fingers: 2) // seed; baseline = 2 - feed(rec, x: 0.50, y: 0.50, fingers: 4) // 2 → 4: latches Open-With AND re-baselines origin - feed(rec, x: 0.18, y: 0.50, fingers: 4) // big horizontal sweep → discard - feed(rec, x: 0.18, y: 0.50, fingers: 0) // lift after a resolution → no-op - XCTAssertEqual(d.discardCount, 1) - XCTAssertEqual(d.openWithCount, 0, "discard pre-empts the latched Open-With") - XCTAssertEqual(d.openCount, 0) - } - - func test_fourFingerSmallNudge_doesNotDiscard() { - // A small horizontal nudge at four fingers is below the discard threshold: no discard, and (arming - // posture) no depth steps either; a lift then resolves Open-With from the +1. - let (rec, d) = makeDrillRecognizer(makeSettings(launcherActivationThreshold: 0.045)) - feed(rec, x: 0.50, y: 0.50, fingers: 3) // seed; baseline = 3 - feed(rec, x: 0.50, y: 0.50, fingers: 4) // 3 → 4: latch Open-With, re-baseline - feed(rec, x: 0.52, y: 0.50, fingers: 4) // dx +0.02 < 0.045 → no discard - XCTAssertEqual(d.discardCount, 0) - XCTAssertTrue(d.depths.isEmpty) - feed(rec, x: 0.52, y: 0.50, fingers: 0) // lift → Open-With - XCTAssertEqual(d.openWithCount, 1) - } - - func test_fourFingerVerticalSweep_doesNotDiscard() { - // Discard is a HORIZONTAL swipe-away; a vertical sweep at four fingers is not horizontal-dominant, - // so it does not discard (the axis-lock ratio gates it). - let (rec, d) = makeDrillRecognizer(makeSettings(axisLockRatio: 1.4)) - feed(rec, x: 0.50, y: 0.50, fingers: 4) // seed at four - feed(rec, x: 0.50, y: 0.18, fingers: 4) // dy -0.32, dx 0 → not horizontal → no discard - XCTAssertEqual(d.discardCount, 0) - } - - // MARK: - One-shot resolution (a stray re-lift is a no-op) - - func test_resolution_isOneShot_strayReLiftEmitsNothing() { - let (rec, d) = makeDrillRecognizer(makeSettings()) - feed(rec, x: 0.20, y: 0.50, fingers: 2) // seed - feed(rec, x: 0.35, y: 0.50, fingers: 2) // navigate: Δx +0.15 → one depth step - feed(rec, x: 0.35, y: 0.50, fingers: 0) // lift → Open (resolved) - XCTAssertEqual(d.openCount, 1) - // A stray re-lift: fingers return, then lift again — must emit nothing further. - feed(rec, x: 0.40, y: 0.50, fingers: 2) // fingers return (re-seed only) - feed(rec, x: 0.60, y: 0.50, fingers: 2) // travel while resolved → no steps - feed(rec, x: 0.60, y: 0.50, fingers: 0) // re-lift → no second resolution - XCTAssertEqual(d.openCount, 1, "the resolution is one-shot for the whole session") - XCTAssertEqual(d.openWithCount, 0) - XCTAssertEqual(d.depths, [1], "no NEW navigation steps after the session resolved") - } - - func test_reEntry_clearsResolvedState_forANewSession() { - // The controller flips the flag off on hide and on again when the navigator re-opens; the didSet - // re-seeds a fresh session, so a new Open can resolve. - let (rec, d) = makeDrillRecognizer(makeSettings()) - feed(rec, x: 0.50, y: 0.50, fingers: 2) // seed - feed(rec, x: 0.50, y: 0.50, fingers: 0) // lift → Open (resolved) - XCTAssertEqual(d.openCount, 1) - rec.filesDrillActive = false // navigator hides - rec.filesDrillActive = true // navigator re-opens → fresh session - feed(rec, x: 0.50, y: 0.50, fingers: 2) // seed again - feed(rec, x: 0.50, y: 0.50, fingers: 0) // lift → a SECOND Open - XCTAssertEqual(d.openCount, 2, "re-entering the sub-state clears the one-shot resolution") - } - - // MARK: - Lift debounce (edge flicker is not a resolution) - - func test_singleFingerFlicker_doesNotResolve_butSustainedDropDoes() { - // A momentary dip to one finger (an edge flicker) is debounced — only a true lift (0) or a - // sustained drop below two resolves. Mirrors the launcher's belowTargetFrames >= 2 rule. - let (rec, d) = makeDrillRecognizer(makeSettings()) - feed(rec, x: 0.50, y: 0.50, fingers: 2) // seed - feed(rec, x: 0.50, y: 0.50, fingers: 1) // one frame at 1 → debounced, no resolution - XCTAssertEqual(d.openCount, 0, "a single below-two frame is debounced") - feed(rec, x: 0.50, y: 0.50, fingers: 1) // a second below-two frame → sustained drop → resolve - XCTAssertEqual(d.openCount, 1) - } - - // MARK: - Bypass: a fresh contact never routes to the switcher / launcher - - func test_drillActive_freshThreeFingerContact_doesNotOpenSwitcher() { - // While drilling, a fresh three-finger contact + horizontal scrub routes to the drill (depth steps), - // never to the switcher — the early short-circuit bypasses the idle re-latch. - let (rec, d) = makeDrillRecognizer(makeSettings()) - feed(rec, x: 0.20, y: 0.50, fingers: 3) // fresh three-finger contact (would normally arm switcher) - feed(rec, x: 0.35, y: 0.50, fingers: 3) // horizontal push, Δx +0.15 → one depth step - XCTAssertFalse(d.didSwitcherActivate, "a fresh contact during drill never opens the switcher") - XCTAssertEqual(d.depths, [1], "it routes to the drill instead") - } - - func test_drillActive_freshFourFingerContact_doesNotOpenLauncher() { - // A fresh four-finger contact during drill must not open a second launcher on top of the navigator; - // it routes to the drill (here: the four-finger arming posture, resolved by a horizontal discard). - let (rec, d) = makeDrillRecognizer(makeSettings()) - feed(rec, x: 0.50, y: 0.50, fingers: 4) // fresh four-finger contact (would normally arm launcher) - feed(rec, x: 0.20, y: 0.50, fingers: 4) // horizontal sweep → discard (drill), not launcher - XCTAssertFalse(d.didLauncherActivate, "a fresh four-finger contact during drill never opens the launcher") - XCTAssertEqual(d.discardCount, 1) - } - - // MARK: - Configured drill binding maps the physical excursion to the bound intent (§9.4) - - // The recognizer detects only the PHYSICAL excursion (a plain lift, a relative +1-finger lift, a fresh - // four-finger horizontal swipe-away); which intent it fires is the user's configured `filesDrill` - // binding. The default binding reproduces today's grammar; a remap fires the swapped intents. - - func test_defaultBinding_plainLift_emitsOpen() { - // Default: lift → open. - let (rec, d) = makeDrillRecognizer(makeSettings(filesDrill: .default)) - feed(rec, x: 0.50, y: 0.50, fingers: 2) // seed - feed(rec, x: 0.50, y: 0.50, fingers: 0) // plain lift - XCTAssertEqual(d.openCount, 1) - XCTAssertEqual(d.openWithCount, 0) - XCTAssertEqual(d.discardCount, 0) - } - - func test_defaultBinding_plusOneLift_emitsOpenWith() { - // Default: +1-finger lift → Open-With. - let (rec, d) = makeDrillRecognizer(makeSettings(filesDrill: .default)) - feed(rec, x: 0.50, y: 0.50, fingers: 2) // seed; relaxed baseline = 2 - feed(rec, x: 0.50, y: 0.50, fingers: 3) // relative +1 → latch the +1-finger-lift excursion - feed(rec, x: 0.50, y: 0.50, fingers: 0) // lift - XCTAssertEqual(d.openWithCount, 1) - XCTAssertEqual(d.openCount, 0) - } - - func test_defaultBinding_fourFingerHorizontal_emitsDiscard() { - // Default: four-finger horizontal swipe-away → discard. - let (rec, d) = makeDrillRecognizer(makeSettings(filesDrill: .default)) - feed(rec, x: 0.50, y: 0.50, fingers: 4) // seed at four (the arming posture) - feed(rec, x: 0.20, y: 0.50, fingers: 4) // horizontal sweep-away - XCTAssertEqual(d.discardCount, 1) - XCTAssertEqual(d.openCount, 0) - XCTAssertEqual(d.openWithCount, 0) - } - - func test_remappedBinding_liftDiscards_andFourFingerOpens() { - // Swap lift and four-finger horizontal: `assigning(.lift, to: .discard)` gives discard the lift - // excursion and open inherits the four-finger horizontal (the one-to-one swap). So now a PLAIN - // LIFT discards and a FOUR-FINGER HORIZONTAL swipe-away opens — the physical move is unchanged, the - // intent is swapped purely by the binding. - let remap = GestureBindings.FilesDrillBinding.default.assigning(.lift, to: .discard) - XCTAssertEqual(remap.discard, .lift, "precondition: discard now bound to the lift excursion") - XCTAssertEqual(remap.open, .fourFingerHorizontal, "precondition: open now bound to four-finger horizontal") - - // A plain lift now fires discard (NOT open). - let (rec1, d1) = makeDrillRecognizer(makeSettings(filesDrill: remap)) - feed(rec1, x: 0.50, y: 0.50, fingers: 2) // seed - feed(rec1, x: 0.50, y: 0.50, fingers: 0) // plain lift → discard - XCTAssertEqual(d1.discardCount, 1, "the lift now discards") - XCTAssertEqual(d1.openCount, 0, "the lift no longer opens") - XCTAssertEqual(d1.openWithCount, 0) - - // A four-finger horizontal swipe-away now fires open (NOT discard). - let (rec2, d2) = makeDrillRecognizer(makeSettings(filesDrill: remap)) - feed(rec2, x: 0.50, y: 0.50, fingers: 4) // seed at four - feed(rec2, x: 0.20, y: 0.50, fingers: 4) // horizontal sweep-away → open - XCTAssertEqual(d2.openCount, 1, "the four-finger swipe now opens") - XCTAssertEqual(d2.discardCount, 0, "the four-finger swipe no longer discards") - XCTAssertEqual(d2.openWithCount, 0) - } - - func test_remappedBinding_plusOneLiftFiresSwappedIntent() { - // Swap Open-With and open: `assigning(.lift, to: .openWith)` gives Open-With the lift excursion and - // open inherits the +1-finger lift. So a relative +1-finger lift now OPENS, and a plain lift now - // fires OPEN-WITH. - let remap = GestureBindings.FilesDrillBinding.default.assigning(.lift, to: .openWith) - XCTAssertEqual(remap.openWith, .lift) - XCTAssertEqual(remap.open, .plusOneFingerLift) - - let (rec, d) = makeDrillRecognizer(makeSettings(filesDrill: remap)) - feed(rec, x: 0.50, y: 0.50, fingers: 2) // seed; baseline = 2 - feed(rec, x: 0.50, y: 0.50, fingers: 3) // relative +1 → +1-finger-lift excursion - feed(rec, x: 0.50, y: 0.50, fingers: 0) // lift → now OPEN (the +1-finger lift is bound to open) - XCTAssertEqual(d.openCount, 1, "the +1-finger lift now opens") - XCTAssertEqual(d.openWithCount, 0, "and no longer fires Open-With") - XCTAssertEqual(d.discardCount, 0) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/FilesDwellArmingTests.swift b/Tests/ThreeFingerSwitcherTests/FilesDwellArmingTests.swift deleted file mode 100644 index 100d8ed..0000000 --- a/Tests/ThreeFingerSwitcherTests/FilesDwellArmingTests.swift +++ /dev/null @@ -1,63 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// The pure identity-keyed restart decision behind the Files-band dwell-to-arm (change: -/// add-files-band-dwell-arm). The controller owns the timer + haptic; this verifies the *decision* that gates -/// them — the non-obvious bit being that an unchanged identity yields `.keep`, which is exactly why the -/// `+1`-finger morph (it moves no highlight) preserves the arm the user charged on the row. -final class FilesDwellArmingTests: XCTestCase { - - func testLandingOnARealRowRestarts() { - var arming = FilesDwellArming() - XCTAssertEqual(arming.update(identity: "/Home/Docs"), .restart) - XCTAssertEqual(arming.lastIdentity, "/Home/Docs") - } - - func testSameIdentityKeeps_thePlusOneFingerPreservesTheArm() { - var arming = FilesDwellArming() - XCTAssertEqual(arming.update(identity: "/Home/Docs"), .restart) - // A `+1`-finger morph (then the lift) re-evaluates with the SAME highlighted row → no restart, so the - // charge that already armed this row survives into the menu-open gate. - XCTAssertEqual(arming.update(identity: "/Home/Docs"), .keep) - XCTAssertEqual(arming.update(identity: "/Home/Docs"), .keep) - } - - func testMovingToAnotherRowRestarts() { - var arming = FilesDwellArming() - _ = arming.update(identity: "/Home/Docs") - XCTAssertEqual(arming.update(identity: "/Home/photo.png"), .restart) - XCTAssertEqual(arming.update(identity: "/Home/photo.png"), .keep) - } - - func testMovingOntoAnEmptyColumnDisarms() { - var arming = FilesDwellArming() - _ = arming.update(identity: "/Home/Docs") - XCTAssertEqual(arming.update(identity: nil), .disarm) - // Still nothing highlighted → nothing to (re)arm; keep (already disarmed). - XCTAssertEqual(arming.update(identity: nil), .keep) - } - - func testStartingOnNothingKeeps() { - var arming = FilesDwellArming() - XCTAssertEqual(arming.update(identity: nil), .keep, "nothing was armed, nothing to disarm") - } - - func testSubColumnTransitionsEachRestart() { - var arming = FilesDwellArming() - XCTAssertEqual(arming.update(identity: "/Home/Docs"), .restart) // folder row - XCTAssertEqual(arming.update(identity: "menu:0"), .restart) // entered the action menu - XCTAssertEqual(arming.update(identity: "menu:1"), .restart) // scrubbed a menu row - XCTAssertEqual(arming.update(identity: "picker:2"), .restart) // descended into the app grid - XCTAssertEqual(arming.update(identity: "/Home/Docs"), .restart) // backed out to the folder row - } - - func testResetForcesAFreshChargeOnTheSameRow() { - var arming = FilesDwellArming() - _ = arming.update(identity: "/Home/Docs") - XCTAssertEqual(arming.update(identity: "/Home/Docs"), .keep) - // A delivery-failure re-arm in place: reset, then the SAME row must charge afresh (the user re-dwells). - arming.reset() - XCTAssertNil(arming.lastIdentity) - XCTAssertEqual(arming.update(identity: "/Home/Docs"), .restart) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/FilesNavigationModelTests.swift b/Tests/ThreeFingerSwitcherTests/FilesNavigationModelTests.swift deleted file mode 100644 index 4eaf4fc..0000000 --- a/Tests/ThreeFingerSwitcherTests/FilesNavigationModelTests.swift +++ /dev/null @@ -1,382 +0,0 @@ -import XCTest -import Foundation -@testable import ThreeFingerSwitcherCore - -/// Tests the PURE `FilesNavigationModel` column state machine (change: files-band, tasks 2.3–2.5): the -/// ancestors/current/highlight stack, descend pushes / ascend pops / back-out-to-roots, per-root -/// remembered-location restore (gated by the restore-last-location toggle, at init AND on re-entry into a -/// root), stable path-ids across a re-list, and preview-target for file vs folder. (Type-to-filter search -/// was removed — the navigator is pure-trackpad — so there is no search filter or focus-search signal.) -/// -/// Determinism: the model never touches `FileManager` — every folder's contents come from an injected -/// fixture lister, so the whole machine is exercised without a filesystem or a running app (mirroring -/// `FilesSeamsTests`, which is likewise AppKit-free and not `@MainActor`). -final class FilesNavigationModelTests: XCTestCase { - - // MARK: - Fixture filesystem - - /// A tiny in-memory tree the fixture lister serves. Paths are absolute; a folder maps to its child - /// entries, a file is absent from the map (the lister only ever lists folders). - private func fileEntry(_ path: String, isDirectory: Bool, kind: FileKind, mod: Date? = nil) -> FileEntry { - FileEntry(url: URL(fileURLWithPath: path), name: URL(fileURLWithPath: path).lastPathComponent, - isDirectory: isDirectory, modificationDate: mod, kind: kind) - } - - /// Build a lister over a `[folderPath: [FileEntry]]` map. Unlisted/absent folders return empty. - private func lister(_ tree: [String: [FileEntry]]) -> (URL) -> [FileEntry] { - { url in tree[url.standardizedFileURL.path] ?? [] } - } - - /// A standard two-root tree: - /// /Home → Docs/, photo.png - /// /Home/Docs → Sub/, a.txt - /// /Home/Docs/Sub → deep.txt - /// /Work → notes.md - private func standardModel(remembered: [URL: URL] = [:], - restoreLastLocation: Bool = false) -> FilesNavigationModel { - let tree: [String: [FileEntry]] = [ - "/Home": [ - fileEntry("/Home/Docs", isDirectory: true, kind: .folder), - fileEntry("/Home/photo.png", isDirectory: false, kind: .image), - ], - "/Home/Docs": [ - fileEntry("/Home/Docs/Sub", isDirectory: true, kind: .folder), - fileEntry("/Home/Docs/a.txt", isDirectory: false, kind: .text), - ], - "/Home/Docs/Sub": [ - fileEntry("/Home/Docs/Sub/deep.txt", isDirectory: false, kind: .text), - ], - "/Work": [ - fileEntry("/Work/notes.md", isDirectory: false, kind: .text), - ], - ] - return FilesNavigationModel(roots: [URL(fileURLWithPath: "/Home"), URL(fileURLWithPath: "/Work")], - remembered: remembered, - restoreLastLocation: restoreLastLocation, - lister: lister(tree)) - } - - // MARK: - Entry: the roots column - - func testLandsOnTheRootsList() { - let model = standardModel() - XCTAssertEqual(model.current, .roots) - XCTAssertEqual(model.ancestors, []) - XCTAssertEqual(model.visibleEntries.map { $0.name }, ["Home", "Work"]) - XCTAssertEqual(model.highlightedIndex, 0) - XCTAssertEqual(model.highlightedEntry?.name, "Home") - XCTAssertFalse(model.canAscend, "ascend is a no-op on the roots list") - } - - // MARK: - Descend pushes an ancestor; ascend pops - - func testDescendIntoRootMakesItCurrentWithEmptyAncestors() { - var model = standardModel() - model.descend() // into /Home (highlighted root) - XCTAssertEqual(model.current, .folder(URL(fileURLWithPath: "/Home"))) - XCTAssertEqual(model.ancestors, [], "a root is the base of its column, not an ancestor") - XCTAssertEqual(model.visibleEntries.map { $0.name }, ["Docs", "photo.png"]) - XCTAssertEqual(model.highlightedIndex, 0, "descend resets the highlight to the top") - } - - func testDescendIntoFolderPushesPriorCurrentOntoAncestors() { - var model = standardModel() - model.descend() // → /Home (ancestors []) - model.descend() // highlight is Docs → /Home/Docs - XCTAssertEqual(model.current, .folder(URL(fileURLWithPath: "/Home/Docs"))) - XCTAssertEqual(model.ancestors, [URL(fileURLWithPath: "/Home")], - "the prior current folder becomes the deepest ancestor") - XCTAssertEqual(model.visibleEntries.map { $0.name }, ["Sub", "a.txt"]) - } - - func testAscendPopsTheDeepestAncestorBackToCurrent() { - var model = standardModel() - model.descend() // → /Home - model.descend() // → /Home/Docs (ancestors [/Home]) - model.ascend() // pop → /Home - XCTAssertEqual(model.current, .folder(URL(fileURLWithPath: "/Home"))) - XCTAssertEqual(model.ancestors, [], "the popped ancestor is dropped") - XCTAssertEqual(model.highlightedEntry?.name, "Docs", - "ascend re-highlights the folder we came up from") - } - - func testAscendFromARootTopLevelReturnsToTheRootsList() { - var model = standardModel() - model.descend() // → /Home (ancestors []) - model.ascend() // back-out-to-roots - XCTAssertEqual(model.current, .roots) - XCTAssertEqual(model.ancestors, []) - XCTAssertEqual(model.visibleEntries.map { $0.name }, ["Home", "Work"]) - XCTAssertEqual(model.highlightedEntry?.name, "Home", - "the root we came up from is re-highlighted") - } - - func testAscendOnTheRootsListIsANoOp() { - var model = standardModel() - model.ascend() - XCTAssertEqual(model.current, .roots) - XCTAssertEqual(model.visibleEntries.map { $0.name }, ["Home", "Work"]) - } - - func testDescendOnAFileIsANoOp() { - var model = standardModel() - model.descend() // → /Home - model.highlightDown() // highlight photo.png (a file) - let before = model.current - model.descend() // files open, they don't descend - XCTAssertEqual(model.current, before, "descending onto a file does nothing") - XCTAssertEqual(model.highlightedEntry?.name, "photo.png") - } - - // MARK: - Remembered-location restore (gated by the toggle, on RE-ENTRY into a root) - - func testReEnteringARootRestoresTheDeepestLocationWhenRestoreOn() { - // Pre-seed Home's remembered deepest as /Home/Docs/Sub. With restore ON the band opens there (init); - // ascend all the way back to the roots list, then RE-ENTER /Home — `enterRoot` restores it again. - let remembered = [URL(fileURLWithPath: "/Home"): URL(fileURLWithPath: "/Home/Docs/Sub")] - var model = standardModel(remembered: remembered, restoreLastLocation: true) - model.ascend(); model.ascend(); model.ascend() // /Home/Docs/Sub → … → roots - XCTAssertEqual(model.current, .roots) - model.descend() // re-enter /Home → restore straight to /Home/Docs/Sub - XCTAssertEqual(model.current, .folder(URL(fileURLWithPath: "/Home/Docs/Sub"))) - XCTAssertEqual(model.visibleEntries.map { $0.name }, ["deep.txt"]) - XCTAssertEqual(model.ancestors, - [URL(fileURLWithPath: "/Home"), URL(fileURLWithPath: "/Home/Docs")], - "the ancestor chain is rebuilt so ascend walks back up correctly") - } - - func testDescendIntoARootIgnoresRememberedWhenRestoreOff() { - // The bug fix: with restore OFF the band opens on the roots list AND descending into a root lands on - // the root's TOP level — it must NOT jump to the remembered deep folder (the toggle was off). - let remembered = [URL(fileURLWithPath: "/Home"): URL(fileURLWithPath: "/Home/Docs/Sub")] - var model = standardModel(remembered: remembered, restoreLastLocation: false) - XCTAssertEqual(model.current, .roots, "restore off → opens on the roots list") - model.descend() // enter /Home → land on the TOP level, not /Home/Docs/Sub - XCTAssertEqual(model.current, .folder(URL(fileURLWithPath: "/Home")), - "restore off → descending into a root never jumps to the remembered folder") - XCTAssertEqual(model.ancestors, []) - XCTAssertEqual(model.visibleEntries.map { $0.name }, ["Docs", "photo.png"]) - } - - func testNavigatingRecordsTheDeepestLocationForPersistence() { - var model = standardModel() - model.descend() // → /Home - model.descend() // → /Home/Docs - // The model surfaces what the caller should persist: Home now remembers /Home/Docs. - XCTAssertEqual(model.rememberedLocations[URL(fileURLWithPath: "/Home")], - URL(fileURLWithPath: "/Home/Docs")) - // Work is untouched. - XCTAssertNil(model.rememberedLocations[URL(fileURLWithPath: "/Work")]) - } - - func testEachRootRemembersIndependently() { - var model = standardModel() - model.descend() // → /Home - model.descend() // → /Home/Docs - model.ascend(); model.ascend() // back to roots - model.highlightDown() // highlight /Work - model.descend() // → /Work - XCTAssertEqual(model.rememberedLocations[URL(fileURLWithPath: "/Home")], - URL(fileURLWithPath: "/Home/Docs"), "Home kept its own deepest") - XCTAssertEqual(model.rememberedLocations[URL(fileURLWithPath: "/Work")], - URL(fileURLWithPath: "/Work"), "Work remembers its own top level") - } - - func testStaleRememberedLocationOutsideTheRootIsIgnored() { - // A remembered path that is NOT under the root must not be restored (e.g. the root moved) even with - // restore ON: enterRoot skips it and lands on the top level. - let remembered = [URL(fileURLWithPath: "/Home"): URL(fileURLWithPath: "/Elsewhere/x")] - var model = standardModel(remembered: remembered, restoreLastLocation: true) - model.descend() // enter /Home → land on the top level, not the stale path - XCTAssertEqual(model.current, .folder(URL(fileURLWithPath: "/Home"))) - XCTAssertEqual(model.ancestors, []) - } - - // MARK: - Stable identity across a re-list - - func testEntryIDsAreStableAcrossAReList() { - var model = standardModel() - model.descend() // → /Home - let idsBefore = model.visibleEntries.map { $0.id } - model.descend() // → /Home/Docs - model.ascend() // re-list /Home - let idsAfter = model.visibleEntries.map { $0.id } - XCTAssertEqual(idsBefore, idsAfter, "re-listing the same folder yields the same path-derived ids") - XCTAssertEqual(idsAfter, ["/Home/Docs", "/Home/photo.png"]) - } - - // MARK: - Preview target (file → self, folder → its contents) - - func testPreviewTargetForAFileIsTheFileItself() { - var model = standardModel() - model.descend() // → /Home - model.highlightDown() // highlight photo.png - guard case let .file(entry) = model.previewTarget else { - return XCTFail("a highlighted file previews itself") - } - XCTAssertEqual(entry.name, "photo.png") - } - - func testPreviewTargetForAFolderIsItsContents() { - var model = standardModel() - model.descend() // → /Home, highlight Docs - guard case let .folder(entry, contents) = model.previewTarget else { - return XCTFail("a highlighted folder previews its contents") - } - XCTAssertEqual(entry.name, "Docs") - XCTAssertEqual(contents.map { $0.name }, ["Sub", "a.txt"], - "the peek lists exactly what descending would make current") - } - - func testPreviewTargetIsNilOnAnEmptyColumn() { - // Descend into an empty folder so the current column has no entries → no preview target. - let tree: [String: [FileEntry]] = [ - "/R": [fileEntry("/R/Empty", isDirectory: true, kind: .folder)], - "/R/Empty": [], - ] - var model = FilesNavigationModel(roots: [URL(fileURLWithPath: "/R")], lister: lister(tree)) - model.descend() // → /R (highlight Empty) - model.descend() // → /R/Empty (empty column) - XCTAssertNil(model.previewTarget) - } - - // MARK: - Highlight stepping (clamps both ends; no search overflow) - - func testHighlightDownAndUpStepWithinBounds() { - var model = standardModel() - model.descend() // → /Home (Docs, photo.png) - XCTAssertEqual(model.highlightedIndex, 0) - model.highlightDown() - XCTAssertEqual(model.highlightedIndex, 1) - model.highlightDown() // clamp at the last row - XCTAssertEqual(model.highlightedIndex, 1) - model.highlightUp() - XCTAssertEqual(model.highlightedIndex, 0) - } - - func testHighlightUpAtTheTopClamps() { - var model = standardModel() - model.descend() // → /Home, highlight at index 0 - model.highlightUp() // up at the top → clamp, no overflow, no side effect - XCTAssertEqual(model.highlightedIndex, 0, "the highlight stays clamped at the top") - model.highlightUp() // and again — still a no-op - XCTAssertEqual(model.highlightedIndex, 0) - } - - // MARK: - Restore-at-init (refinement 2: the band OPENS displaying the last folder) - - func testRestoreAtInitOpensDisplayingTheRememberedFolderWithRebuiltAncestors() { - // A prior session left off at /Home/Docs/Sub; with the restore toggle ON the band must OPEN there. - let remembered = [URL(fileURLWithPath: "/Home"): URL(fileURLWithPath: "/Home/Docs/Sub")] - let model = standardModel(remembered: remembered, restoreLastLocation: true) - XCTAssertEqual(model.current, .folder(URL(fileURLWithPath: "/Home/Docs/Sub")), - "restore happens AT INIT, not on the first descend") - XCTAssertEqual(model.ancestors, - [URL(fileURLWithPath: "/Home"), URL(fileURLWithPath: "/Home/Docs")], - "the ancestor chain is reconstructed so ascending walks back up") - XCTAssertEqual(model.visibleEntries.map { $0.name }, ["deep.txt"], - "the restored folder's contents are listed on open") - XCTAssertEqual(model.highlightedIndex, 0) - } - - func testRestoreAtInitWithRootTopLevelRememberedLandsOnThatRoot() { - // Remembered is the root itself (the user only ever reached its top level): open there, no ancestors. - let remembered = [URL(fileURLWithPath: "/Home"): URL(fileURLWithPath: "/Home")] - let model = standardModel(remembered: remembered, restoreLastLocation: true) - XCTAssertEqual(model.current, .folder(URL(fileURLWithPath: "/Home"))) - XCTAssertEqual(model.ancestors, [], "a root top-level restore has no ancestors") - XCTAssertEqual(model.visibleEntries.map { $0.name }, ["Docs", "photo.png"]) - } - - func testRestoreAtInitOffLandsOnTheRootsListEvenWithARememberedLocation() { - // The toggle OFF must ignore the remembered map and land fresh on the roots list. - let remembered = [URL(fileURLWithPath: "/Home"): URL(fileURLWithPath: "/Home/Docs/Sub")] - let model = standardModel(remembered: remembered, restoreLastLocation: false) - XCTAssertEqual(model.current, .roots) - XCTAssertEqual(model.ancestors, []) - XCTAssertEqual(model.visibleEntries.map { $0.name }, ["Home", "Work"]) - } - - func testRestoreAtInitWithNothingRememberedFallsBackToTheRootsList() { - // Toggle ON but no remembered location → still the roots list (nothing to restore). - let model = standardModel(restoreLastLocation: true) - XCTAssertEqual(model.current, .roots) - XCTAssertEqual(model.visibleEntries.map { $0.name }, ["Home", "Work"]) - } - - func testRestoreAtInitSkipsAStaleRememberedLocationOutsideTheRoot() { - // A remembered path no longer under the root (the root moved) must not be restored; fall back to roots. - let remembered = [URL(fileURLWithPath: "/Home"): URL(fileURLWithPath: "/Elsewhere/x")] - let model = standardModel(remembered: remembered, restoreLastLocation: true) - XCTAssertEqual(model.current, .roots, "a stale remembered path is ignored at restore") - } - - func testRestoreAtInitPicksTheDeepestRememberedLocationAcrossRoots() { - // Both roots are remembered; the deeper one (/Home/Docs/Sub) wins over /Work's top level. - let remembered = [ - URL(fileURLWithPath: "/Home"): URL(fileURLWithPath: "/Home/Docs/Sub"), - URL(fileURLWithPath: "/Work"): URL(fileURLWithPath: "/Work"), - ] - let model = standardModel(remembered: remembered, restoreLastLocation: true) - XCTAssertEqual(model.current, .folder(URL(fileURLWithPath: "/Home/Docs/Sub")), - "the deepest remembered path is the most specific 'where you left off'") - XCTAssertEqual(model.ancestors, - [URL(fileURLWithPath: "/Home"), URL(fileURLWithPath: "/Home/Docs")]) - } - - func testRestoredDepthAtInitAscendsBackUpToTheRootsList() { - // The reconstructed chain must let ascend walk all the way back to the roots list. - let remembered = [URL(fileURLWithPath: "/Home"): URL(fileURLWithPath: "/Home/Docs/Sub")] - var model = standardModel(remembered: remembered, restoreLastLocation: true) - model.ascend() // → /Home/Docs - XCTAssertEqual(model.current, .folder(URL(fileURLWithPath: "/Home/Docs"))) - model.ascend() // → /Home - XCTAssertEqual(model.current, .folder(URL(fileURLWithPath: "/Home"))) - model.ascend() // → roots - XCTAssertEqual(model.current, .roots) - } - - // MARK: - Breadcrumb (refinement 4: root → … → highlighted item) - - func testBreadcrumbAtTheRootsListIsTheHighlightedRoot() { - let model = standardModel() - XCTAssertEqual(model.breadcrumb.map { $0.name }, ["Home"], - "at the entry column the highlighted root is the whole path") - XCTAssertEqual(model.breadcrumb.map { $0.url }, [URL(fileURLWithPath: "/Home")]) - } - - func testBreadcrumbIsRootThroughCurrentToHighlightedEntry() { - var model = standardModel() - model.descend() // → /Home, highlight Docs - model.descend() // → /Home/Docs, highlight Sub - // ancestors [/Home] + current /Home/Docs + highlighted Sub. - XCTAssertEqual(model.breadcrumb.map { $0.name }, ["Home", "Docs", "Sub"]) - XCTAssertEqual(model.breadcrumb.map { $0.url }, - [URL(fileURLWithPath: "/Home"), - URL(fileURLWithPath: "/Home/Docs"), - URL(fileURLWithPath: "/Home/Docs/Sub")]) - } - - func testBreadcrumbUpdatesLiveAsTheHighlightMoves() { - var model = standardModel() - model.descend() // → /Home, highlight Docs (a folder) - XCTAssertEqual(model.breadcrumb.map { $0.name }, ["Home", "Docs"]) - model.highlightDown() // highlight photo.png (a file) - XCTAssertEqual(model.breadcrumb.map { $0.name }, ["Home", "photo.png"], - "the leaf follows the highlight live (folder → file)") - } - - func testBreadcrumbStopsAtTheCurrentFolderWhenTheColumnIsEmpty() { - // An empty folder → nothing highlighted, so the path stops at the current folder. - let tree: [String: [FileEntry]] = [ - "/R": [fileEntry("/R/Empty", isDirectory: true, kind: .folder)], - "/R/Empty": [], - ] - var model = FilesNavigationModel(roots: [URL(fileURLWithPath: "/R")], lister: lister(tree)) - model.descend() // → /R - model.descend() // → /R/Empty (empty column) - XCTAssertNil(model.highlightedEntry) - XCTAssertEqual(model.breadcrumb.map { $0.name }, ["R", "Empty"], - "with nothing highlighted the path stops at the current folder") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/FilesSeamsTests.swift b/Tests/ThreeFingerSwitcherTests/FilesSeamsTests.swift deleted file mode 100644 index 7630e78..0000000 --- a/Tests/ThreeFingerSwitcherTests/FilesSeamsTests.swift +++ /dev/null @@ -1,176 +0,0 @@ -import XCTest -import Foundation -@testable import ThreeFingerSwitcherCore - -/// Tests for the two Files-band foundation types (change: files-band, tasks 2.1 / 3.1): -/// `FileEntry` (a path-stable, ephemeral value type), the `FileWorkspace` seam (proven conformable by a -/// pure-Foundation stub, no AppKit), and the `FileActionError` taxonomy (clean per-case headlines, raw -/// error text only on the side, never on the headline). -final class FilesSeamsTests: XCTestCase { - - // MARK: - FileEntry: stable, path-derived identity - - func testIDIsTheStandardizedAbsolutePath() { - let url = URL(fileURLWithPath: "/Users/x/Documents/report.pdf") - let entry = FileEntry(url: url, name: "report.pdf", isDirectory: false, - modificationDate: nil, kind: .pdf) - XCTAssertEqual(entry.id, "/Users/x/Documents/report.pdf") - XCTAssertEqual(entry.id, entry.path) - } - - func testRelistingTheSamePathYieldsTheSameID() { - // Re-listing the same folder (on re-entry, or because a file changed) must keep the SAME id so the - // selection highlight has a stable target and never strobes — the core reason the id is path-derived. - let url = URL(fileURLWithPath: "/tmp/files-band/a.txt") - let first = FileEntry(url: url, name: "a.txt", isDirectory: false, - modificationDate: Date(timeIntervalSince1970: 0), kind: .text) - let relistedSamePathNewModDate = FileEntry(url: url, name: "a.txt", isDirectory: false, - modificationDate: Date(timeIntervalSince1970: 999), - kind: .text) - XCTAssertEqual(first.id, relistedSamePathNewModDate.id) - } - - func testEqualPathsButDifferentMetadataAreNotEqualButShareIdentity() { - let url = URL(fileURLWithPath: "/tmp/files-band/a.txt") - let a = FileEntry(url: url, name: "a.txt", isDirectory: false, - modificationDate: Date(timeIntervalSince1970: 0), kind: .text) - let b = FileEntry(url: url, name: "a.txt", isDirectory: false, - modificationDate: Date(timeIntervalSince1970: 1), kind: .text) - // Same identity (path) but not value-equal (the mod date differs) — exactly what keeps the SwiftUI - // row stable while still letting the view notice a metadata change. - XCTAssertEqual(a.id, b.id) - XCTAssertNotEqual(a, b) - } - - func testStandardizationCollapsesRelativeComponents() { - let messy = URL(fileURLWithPath: "/Users/x/Documents/../Documents/./report.pdf") - let entry = FileEntry(url: messy, name: "report.pdf", isDirectory: false, - modificationDate: nil, kind: .pdf) - XCTAssertEqual(entry.id, "/Users/x/Documents/report.pdf") - XCTAssertEqual(entry.url.path, "/Users/x/Documents/report.pdf") - } - - func testDirectoryFlagAndKindArePreserved() { - let dir = FileEntry(url: URL(fileURLWithPath: "/Users/x/Projects"), name: "Projects", - isDirectory: true, modificationDate: nil, kind: .folder) - XCTAssertTrue(dir.isDirectory) - XCTAssertEqual(dir.kind, .folder) - XCTAssertEqual(dir.name, "Projects") - } - - // MARK: - FileWorkspace: the seam is conformable without AppKit - - func testStubConformsAndRecordsDefaultOpen() async throws { - let workspace = RecordingFileWorkspace() - let url = URL(fileURLWithPath: "/tmp/a.txt") - try await workspace.open(url) - XCTAssertEqual(workspace.openedDefault, [url]) - } - - func testStubRecordsOpenWith() async throws { - let workspace = RecordingFileWorkspace() - let url = URL(fileURLWithPath: "/tmp/a.txt") - let app = URL(fileURLWithPath: "/Applications/TextEdit.app") - try await workspace.open(url, withApplicationAt: app) - XCTAssertEqual(workspace.openedWith.count, 1) - XCTAssertEqual(workspace.openedWith.first?.file, url) - XCTAssertEqual(workspace.openedWith.first?.app, app) - } - - func testStubSurfacesAssociationQueries() { - let app = URL(fileURLWithPath: "/Applications/TextEdit.app") - let workspace = RecordingFileWorkspace(apps: [app], defaultApp: app) - let url = URL(fileURLWithPath: "/tmp/a.txt") - XCTAssertEqual(workspace.urlsForApplications(toOpen: url), [app]) - XCTAssertEqual(workspace.urlForApplication(toOpen: url), app) - } - - func testStubOpenCanThrowAFileActionError() async { - let workspace = RecordingFileWorkspace(openError: .openFailed(name: "a.txt", details: nil)) - do { - try await workspace.open(URL(fileURLWithPath: "/tmp/a.txt")) - XCTFail("expected the stub to throw") - } catch let error as FileActionError { - XCTAssertEqual(error, .openFailed(name: "a.txt", details: nil)) - } catch { - XCTFail("expected a FileActionError, got \(error)") - } - } - - // MARK: - FileActionError: clean, per-case headlines - - /// A headline must read as a human sentence — never a reflected enum dump or raw OS text. - private func assertHeadlineIsClean(_ error: FileActionError, - file: StaticString = #filePath, line: UInt = #line) { - let headline = error.errorDescription - XCTAssertNotNil(headline, "every case is self-describing", file: file, line: line) - let h = headline ?? "" - XCTAssertFalse(h.isEmpty, "headline is non-empty", file: file, line: line) - for needle in ["Domain=", "Code=", "Error Domain", "UserInfo", "FileActionError"] { - XCTAssertFalse(h.contains(needle), - "headline must not contain raw error text (\(needle)): \(h)", - file: file, line: line) - } - } - - func testEveryCaseHasACleanHeadline() { - assertHeadlineIsClean(.folderUnreadable(name: "Secret", details: nil)) - assertHeadlineIsClean(.openFailed(name: "report.pdf", details: nil)) - assertHeadlineIsClean(.noApplicationForFile(name: "thing.weird")) - } - - func testHeadlineNamesTheFileButNotTheRawDetails() { - let rawDetails = "Error Domain=CocoaErrorDomain Code=257 \"You don't have permission.\"" - let error = FileActionError.openFailed(name: "report.pdf", details: rawDetails) - let headline = error.errorDescription ?? "" - XCTAssertTrue(headline.contains("report.pdf")) - // The raw OS text rides on the side payload, NOT the headline. - XCTAssertFalse(headline.contains("CocoaErrorDomain")) - XCTAssertFalse(headline.contains("Code=257")) - // ...but it IS available for an opt-in disclosure / log. - XCTAssertEqual(error.copyableDetails, rawDetails) - } - - func testNoApplicationCaseCarriesNoDetails() { - XCTAssertNil(FileActionError.noApplicationForFile(name: "x").copyableDetails) - } - - func testFolderUnreadableExposesDetails() { - let error = FileActionError.folderUnreadable(name: "Secret", - details: "Error Domain=NSCocoaErrorDomain Code=257") - XCTAssertEqual(error.copyableDetails, "Error Domain=NSCocoaErrorDomain Code=257") - } -} - -// MARK: - Test stub (Foundation-only — proves `FileWorkspace` conforms without AppKit) - -/// A pure-Foundation `FileWorkspace` for tests: records opens and returns canned associations. Its mere -/// existence (no `import AppKit`) demonstrates the seam is dependency-light enough to stub. `fileprivate` -/// so it never collides with the open-service agent's own stub. -private final class RecordingFileWorkspace: FileWorkspace { - private(set) var openedDefault: [URL] = [] - private(set) var openedWith: [(file: URL, app: URL)] = [] - private let apps: [URL] - private let defaultApp: URL? - private let openError: FileActionError? - - init(apps: [URL] = [], defaultApp: URL? = nil, openError: FileActionError? = nil) { - self.apps = apps - self.defaultApp = defaultApp - self.openError = openError - } - - func open(_ url: URL) async throws { - if let openError { throw openError } - openedDefault.append(url) - } - - func open(_ url: URL, withApplicationAt applicationURL: URL) async throws { - if let openError { throw openError } - openedWith.append((file: url, app: applicationURL)) - } - - func urlsForApplications(toOpen url: URL) -> [URL] { apps } - - func urlForApplication(toOpen url: URL) -> URL? { defaultApp } -} diff --git a/Tests/ThreeFingerSwitcherTests/FirstTouchWizardModelTests.swift b/Tests/ThreeFingerSwitcherTests/FirstTouchWizardModelTests.swift index 1ee5b80..d5bf2bc 100644 --- a/Tests/ThreeFingerSwitcherTests/FirstTouchWizardModelTests.swift +++ b/Tests/ThreeFingerSwitcherTests/FirstTouchWizardModelTests.swift @@ -20,9 +20,8 @@ final class FirstTouchWizardModelTests: XCTestCase { store = FirstRunStore(defaults: defaults) let settings = AppSettings(defaults: defaults) settings.keepClipboardHistory = false - settings.aiCommandsEnabled = false context = WizardContext(settings: settings, permissions: PermissionsService()) - context.launcherBands = { clipboardOn, _ in + context.launcherBands = { clipboardOn in var bands = [ContextBand(name: "Work", color: ItemColor(red: 0, green: 0, blue: 1), items: [LaunchItem(title: "Thing", icon: .sfSymbol("star"), kind: .url(URL(string: "https://example.com")!))])] @@ -59,7 +58,7 @@ final class FirstTouchWizardModelTests: XCTestCase { } func testEmptyBandsLeaveTheDemoEmptyWithoutError() { - context.launcherBands = { _, _ in [] } + context.launcherBands = { _ in [] } store.stage = .playground let model = FirstTouchWizardModel(context: context, store: store) model.resume() diff --git a/Tests/ThreeFingerSwitcherTests/FleetErrorTests.swift b/Tests/ThreeFingerSwitcherTests/FleetErrorTests.swift deleted file mode 100644 index 381b6d6..0000000 --- a/Tests/ThreeFingerSwitcherTests/FleetErrorTests.swift +++ /dev/null @@ -1,33 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// `FleetError` taxonomy + the single translator routing (tasks 4.1, 4.2, design D7). -final class FleetErrorTests: XCTestCase { - - func testEachCaseHasCleanNonEmptyDescription() { - let cannot = FleetError.cannotAdmit(modelName: "Video model", evictedDetails: "chat") - let cloud = FleetError.cloudDisabled(modelName: "GLM-5.2") - XCTAssertFalse((cannot.errorDescription ?? "").isEmpty) - XCTAssertFalse((cloud.errorDescription ?? "").isEmpty) - } - - /// 4.2 — routes through `AIError.message(for:)`; eviction list rides in details, NOT the headline. - func testCannotAdmitRoutesThroughTranslatorEvictionInDetails() { - let err = FleetError.cannotAdmit(modelName: "Video model", - evictedDetails: "Tried to evict: chat") - let presented = AIError.message(for: err) - XCTAssertEqual(presented.headline, err.errorDescription) - // The eviction list must NOT leak into the headline (raw-interpolation ban). - XCTAssertFalse(presented.headline.contains("Tried to evict")) - XCTAssertEqual(presented.details, "Tried to evict: chat") - } - - func testCloudDisabledRoutesThroughTranslator() { - let err = FleetError.cloudDisabled(modelName: "GLM-5.2") - let presented = AIError.message(for: err) - XCTAssertEqual(presented.headline, err.errorDescription) - XCTAssertNil(presented.details) - // The headline names the model (a clean known string), not a raw error dump. - XCTAssertTrue(presented.headline.contains("GLM-5.2")) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/FleetRosterTests.swift b/Tests/ThreeFingerSwitcherTests/FleetRosterTests.swift deleted file mode 100644 index 225b525..0000000 --- a/Tests/ThreeFingerSwitcherTests/FleetRosterTests.swift +++ /dev/null @@ -1,120 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Fleet enums, the extended `ModelDescriptor`, the `ModelRegistry` protocol, `FleetRoster`, and the -/// `StubModelRegistry` (tasks 1.1, 1.2, 2.1, 2.2, 2.3). Pure Core — no weights, no network. -final class FleetRosterTests: XCTestCase { - - // MARK: - 1.1 Enums (exhaustive cases) - - func testModelRoleCases() { - XCTAssertEqual(Set(ModelRole.allCases), - [.chat, .ternaryChat, .image, .video, .cloudEscalation]) - } - - func testModelProviderCases() { - XCTAssertEqual(Set(ModelProvider.allCases), [.onDevice, .cloud]) - } - - // MARK: - 1.2 Extended descriptor defaults - - /// A bare construction (pre-fleet shape) defaults to the single-GPU-chat fleet shape, and the new - /// fields read back. `residencyBytes` derives from `sizeBytes` when unset (additive D1). - func testDescriptorDefaultsMatchFleetOfOne() { - let d = ModelDescriptor( - id: "chat", displayName: "Chat", - sizeBytes: 17 * 1024 * 1024 * 1024, - integritySHA: "x", - downloadURL: URL(string: "https://example.com")!, - capabilities: [.text, .vision], - quantization: .qat4bit) - XCTAssertEqual(d.role, .chat) - XCTAssertEqual(d.lane, .gpu) - XCTAssertEqual(d.provider, .onDevice) - XCTAssertEqual(d.maxContextTokens, 131_072) - // Derived from sizeBytes when the caller passes no explicit residencyBytes. - XCTAssertEqual(d.residencyBytes, UInt64(17 * 1024 * 1024 * 1024)) - } - - func testDescriptorExplicitFleetFields() { - let cloud = ModelDescriptor( - id: "glm", displayName: "GLM-5.2", - sizeBytes: 0, integritySHA: "cloud", - downloadURL: URL(string: "https://open.bigmodel.cn")!, - capabilities: [.text, .vision], quantization: .bf16, - maxContextTokens: 1_000_000, - role: .cloudEscalation, lane: nil, provider: .cloud, residencyBytes: 0) - XCTAssertEqual(cloud.role, .cloudEscalation) - XCTAssertNil(cloud.lane) - XCTAssertEqual(cloud.provider, .cloud) - XCTAssertEqual(cloud.residencyBytes, 0) - } - - // MARK: - 2.2 FleetRoster - - func testStandardRosterDescriptorsIncludeBothCloudMembers() { - let ids = FleetRoster.standard.descriptors().map(\.id) - XCTAssertTrue(ids.contains("gemma-4-31b")) // chat - XCTAssertTrue(ids.contains("ternary-cpu-chat")) // ternary - XCTAssertTrue(ids.contains("image-q4")) - XCTAssertTrue(ids.contains("image-fp16")) - XCTAssertTrue(ids.contains("video-ltxv")) - XCTAssertTrue(ids.contains("claude-cloud")) - XCTAssertTrue(ids.contains("glm-5.2-cloud")) - } - - func testGLM52FactsInNameAndCapabilities() { - let glm = FleetRoster.standard.descriptor(id: "glm-5.2-cloud") - XCTAssertNotNil(glm) - XCTAssertTrue(glm!.displayName.contains("753B")) - XCTAssertTrue(glm!.displayName.contains("MIT")) - XCTAssertEqual(glm!.maxContextTokens, 1_000_000) - XCTAssertEqual(glm!.provider, .cloud) - XCTAssertEqual(glm!.role, .cloudEscalation) - XCTAssertEqual(glm!.residencyBytes, 0) - } - - /// `resident()` never includes any cloud member even after ensureResident on it (cloud is a no-op). - func testResidentExcludesEveryCloudMember() async throws { - let roster = FleetRoster.standard - try await roster.ensureResident("claude-cloud") - try await roster.ensureResident("glm-5.2-cloud") - let residentProviders = roster.resident().map(\.provider) - XCTAssertFalse(residentProviders.contains(.cloud)) - XCTAssertTrue(roster.resident().isEmpty, "cloud admissions never make anything resident") - } - - func testCapabilitySelectionFindsChatModel() throws { - let chat = try FleetRoster.standard.selectModel(requiring: [.text]) - XCTAssertEqual(chat.role, .chat) - XCTAssertEqual(chat.id, "gemma-4-31b") - } - - // MARK: - 2.3 StubModelRegistry - - func testFleetOfOneConforms() async throws { - let reg: ModelRegistry = StubModelRegistry.fleetOfOne() - XCTAssertEqual(reg.descriptors().count, 1) - try await reg.ensureResident("gemma-4-31b") - XCTAssertEqual(reg.resident().map(\.id), ["gemma-4-31b"]) - } - - func testScriptedMultiMemberRosterQueryable() { - let stub = StubModelRegistry(members: [ - descriptor(id: "a", role: .chat, lane: .gpu, bytes: 1), - descriptor(id: "b", role: .ternaryChat, lane: .cpuTernary, bytes: 1) - ]) - XCTAssertEqual(Set(stub.descriptors().map(\.id)), ["a", "b"]) - XCTAssertEqual(stub.descriptor(id: "b")?.role, .ternaryChat) - } - - // MARK: - Helper - - private func descriptor(id: String, role: ModelRole, lane: ComputeLane?, bytes: UInt64, - provider: ModelProvider = .onDevice) -> ModelDescriptor { - ModelDescriptor(id: id, displayName: id, sizeBytes: Int64(bytes), integritySHA: "x", - downloadURL: URL(string: "https://example.com/\(id)")!, - capabilities: [.text], quantization: .qat4bit, - role: role, lane: lane, provider: provider, residencyBytes: bytes) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/FullPotentialGateTests.swift b/Tests/ThreeFingerSwitcherTests/FullPotentialGateTests.swift deleted file mode 100644 index c30e53c..0000000 --- a/Tests/ThreeFingerSwitcherTests/FullPotentialGateTests.swift +++ /dev/null @@ -1,155 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Unit tests for the pure Full Potential gate (`AI/FullPotential/FullPotentialGate.swift`). -/// -/// The gate is total and pure (no throw/async/IO), so the truth table is fully exhaustible here: -/// (a) AI-commands off ⇒ all locked; (b) master off ⇒ all locked; (c) with both masters on, one -/// sub-flag unlocks exactly one capability and no other; (d) `allCases` exhaustiveness. -final class FullPotentialGateTests: XCTestCase { - - // MARK: - Capability enum - - /// Exactly five capabilities, one per sub-flag (addendum §D1), with stable raw values. - func testCapabilityHasFiveStableCases() { - XCTAssertEqual(FullPotentialCapability.allCases.count, 5) - XCTAssertEqual(FullPotentialCapability.cpuLane.rawValue, "cpuLane") - XCTAssertEqual(FullPotentialCapability.batchedRuntime.rawValue, "batchedRuntime") - XCTAssertEqual(FullPotentialCapability.mediaGen.rawValue, "mediaGen") - XCTAssertEqual(FullPotentialCapability.backgroundAutonomy.rawValue, "backgroundAutonomy") - XCTAssertEqual(FullPotentialCapability.fleetCloud.rawValue, "fleetCloud") - } - - /// The raw values round-trip through `Codable` (so the enum can ride persistence/telemetry). - func testCapabilityRawValueRoundTrips() { - for cap in FullPotentialCapability.allCases { - XCTAssertEqual(FullPotentialCapability(rawValue: cap.rawValue), cap) - } - } - - // MARK: - Flags value-type equality - - func testFlagsValueEquality() { - let a = FullPotentialFlags(aiCommandsEnabled: true, fullPotentialEnabled: true, - cpuLane: true, batchedRuntime: false, mediaGen: true, - backgroundAutonomy: false, fleetCloud: true) - var b = a - XCTAssertEqual(a, b) - b.mediaGen = false - XCTAssertNotEqual(a, b, "flipping one field breaks equality") - } - - // MARK: - Helpers - - /// All-on flags (AI-commands + master + every sub-flag) — the maximally-unlocked baseline. - private func allOn() -> FullPotentialFlags { - FullPotentialFlags(aiCommandsEnabled: true, fullPotentialEnabled: true, - cpuLane: true, batchedRuntime: true, mediaGen: true, - backgroundAutonomy: true, fleetCloud: true) - } - - /// Flags with both masters on but every sub-flag OFF — the baseline for "one sub-flag unlocks one". - private func mastersOnSubFlagsOff() -> FullPotentialFlags { - FullPotentialFlags(aiCommandsEnabled: true, fullPotentialEnabled: true, - cpuLane: false, batchedRuntime: false, mediaGen: false, - backgroundAutonomy: false, fleetCloud: false) - } - - /// Set exactly one capability's sub-flag to `value` on a copy of `base`. - private func with(_ base: FullPotentialFlags, _ cap: FullPotentialCapability, _ value: Bool) -> FullPotentialFlags { - var f = base - switch cap { - case .cpuLane: f.cpuLane = value - case .batchedRuntime: f.batchedRuntime = value - case .mediaGen: f.mediaGen = value - case .backgroundAutonomy: f.backgroundAutonomy = value - case .fleetCloud: f.fleetCloud = value - } - return f - } - - // MARK: - (a) AI-commands off ⇒ all locked - - func testAICommandsOffLocksEverythingRegardless() { - // Master + every sub-flag on, but AI-commands OFF: every capability is locked. - var flags = allOn() - flags.aiCommandsEnabled = false - let gate = FullPotentialGate(flags: flags) - for cap in FullPotentialCapability.allCases { - XCTAssertFalse(gate.isUnlocked(cap), "\(cap) must be locked when AI-commands is off") - } - } - - // MARK: - (b) Master off ⇒ all locked - - func testMasterOffLocksEverything() { - var flags = allOn() - flags.fullPotentialEnabled = false - let gate = FullPotentialGate(flags: flags) - for cap in FullPotentialCapability.allCases { - XCTAssertFalse(gate.isUnlocked(cap), "\(cap) must be locked when the master is off") - } - } - - // MARK: - All masters + sub-flags on ⇒ all unlocked - - func testAllOnUnlocksEverything() { - let gate = FullPotentialGate(flags: allOn()) - for cap in FullPotentialCapability.allCases { - XCTAssertTrue(gate.isUnlocked(cap), "\(cap) must be unlocked when all flags are on") - } - } - - // MARK: - (c) One sub-flag unlocks exactly its own capability - - func testEachSubFlagGatesExactlyItsOwnCapability() { - for target in FullPotentialCapability.allCases { - let flags = with(mastersOnSubFlagsOff(), target, true) - let gate = FullPotentialGate(flags: flags) - for cap in FullPotentialCapability.allCases { - if cap == target { - XCTAssertTrue(gate.isUnlocked(cap), "\(target) on must unlock \(cap)") - } else { - XCTAssertFalse(gate.isUnlocked(cap), "\(target) on must NOT unlock \(cap)") - } - } - } - } - - // MARK: - (d) Exhaustive gating across allCases - - /// With both masters on, the gate's answer for each capability equals exactly that capability's - /// own stored sub-flag — proving no cross-talk and exhaustive coverage. - func testGateMirrorsOwnSubFlagWhenMastersOn() { - let flags = FullPotentialFlags(aiCommandsEnabled: true, fullPotentialEnabled: true, - cpuLane: true, batchedRuntime: false, mediaGen: true, - backgroundAutonomy: false, fleetCloud: true) - let gate = FullPotentialGate(flags: flags) - XCTAssertTrue(gate.isUnlocked(.cpuLane)) - XCTAssertFalse(gate.isUnlocked(.batchedRuntime)) - XCTAssertTrue(gate.isUnlocked(.mediaGen)) - XCTAssertFalse(gate.isUnlocked(.backgroundAutonomy)) - XCTAssertTrue(gate.isUnlocked(.fleetCloud)) - } - - // MARK: - Panic-off retains values (read-only gate) - - /// The gate never mutates its flags: flipping the master off and back on restores every capability - /// (the stored sub-flags were retained — the gate relocks by computation, not by zeroing). - func testPanicOffRetainsSubFlagsByComputation() { - var flags = allOn() - // Panic-off: master off ⇒ all locked, but the sub-flags are untouched. - flags.fullPotentialEnabled = false - let lockedGate = FullPotentialGate(flags: flags) - for cap in FullPotentialCapability.allCases { - XCTAssertFalse(lockedGate.isUnlocked(cap)) - } - XCTAssertTrue(flags.cpuLane, "the gate did not zero the stored sub-flag") - // Re-arm: master back on ⇒ the retained sub-flags unlock again. - flags.fullPotentialEnabled = true - let rearmedGate = FullPotentialGate(flags: flags) - for cap in FullPotentialCapability.allCases { - XCTAssertTrue(rearmedGate.isUnlocked(cap), "\(cap) restored on re-arm") - } - } -} diff --git a/Tests/ThreeFingerSwitcherTests/FullPotentialSettingsTests.swift b/Tests/ThreeFingerSwitcherTests/FullPotentialSettingsTests.swift deleted file mode 100644 index 922363a..0000000 --- a/Tests/ThreeFingerSwitcherTests/FullPotentialSettingsTests.swift +++ /dev/null @@ -1,153 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Unit tests for the Full Potential persistence + settings→gate adapter on `AppSettings` -/// (`ai-full-potential-toggle`, addendum §D1). Isolated `UserDefaults` suite per test. -@MainActor -final class FullPotentialSettingsTests: XCTestCase { - private var suiteName: String! - private var defaults: UserDefaults! - - override func setUp() { - super.setUp() - suiteName = "ThreeFingerSwitcherTests.FullPotential.\(UUID().uuidString)" - defaults = UserDefaults(suiteName: suiteName) - precondition(defaults != nil, "Failed to create isolated UserDefaults suite") - } - - override func tearDown() { - defaults?.removePersistentDomain(forName: suiteName) - defaults = nil - suiteName = nil - super.tearDown() - } - - private func makeSettings() -> AppSettings { AppSettings(defaults: defaults) } - - /// The six keys, paired with a read accessor, so a table-driven test stays exhaustive. - private func allSix(_ s: AppSettings) -> [(String, Bool)] { - [("fullPotentialEnabled", s.fullPotentialEnabled), - ("cpuLaneEnabled", s.cpuLaneEnabled), - ("batchedRuntimeEnabled", s.batchedRuntimeEnabled), - ("mediaGenEnabled", s.mediaGenEnabled), - ("backgroundAutonomyEnabled", s.backgroundAutonomyEnabled), - ("fleetCloudEscalationEnabled", s.fleetCloudEscalationEnabled)] - } - - // MARK: - 2.1 Defaults (all six false on a fresh store) - - func testAllSixDefaultFalse() { - let s = makeSettings() - for (key, value) in allSix(s) { - XCTAssertFalse(value, "\(key) must default OFF (calm-by-default)") - } - XCTAssertFalse(AppSettings.Defaults.fullPotentialEnabled) - XCTAssertFalse(AppSettings.Defaults.cpuLaneEnabled) - XCTAssertFalse(AppSettings.Defaults.batchedRuntimeEnabled) - XCTAssertFalse(AppSettings.Defaults.mediaGenEnabled) - XCTAssertFalse(AppSettings.Defaults.backgroundAutonomyEnabled) - XCTAssertFalse(AppSettings.Defaults.fleetCloudEscalationEnabled) - } - - // MARK: - Persistence (writes documented keys; round-trips) - - func testAllSixPersistAcrossInstancesAndWriteDocumentedKeys() { - let writer = makeSettings() - writer.fullPotentialEnabled = true - writer.cpuLaneEnabled = true - writer.batchedRuntimeEnabled = true - writer.mediaGenEnabled = true - writer.backgroundAutonomyEnabled = true - writer.fleetCloudEscalationEnabled = true - - // Raw documented key names (a rename breaks this). - XCTAssertEqual(defaults.object(forKey: "fullPotentialEnabled") as? Bool, true) - XCTAssertEqual(defaults.object(forKey: "cpuLaneEnabled") as? Bool, true) - XCTAssertEqual(defaults.object(forKey: "batchedRuntimeEnabled") as? Bool, true) - XCTAssertEqual(defaults.object(forKey: "mediaGenEnabled") as? Bool, true) - XCTAssertEqual(defaults.object(forKey: "backgroundAutonomyEnabled") as? Bool, true) - XCTAssertEqual(defaults.object(forKey: "fleetCloudEscalationEnabled") as? Bool, true) - - let reader = AppSettings(defaults: defaults) - for (key, value) in allSix(reader) { - XCTAssertTrue(value, "\(key) persists across instances") - } - } - - // MARK: - 2.2 Legacy-load (keys absent ⇒ false; existing settings untouched) - - func testLegacySettingsDecodeWithAllSixOffAndUntouched() { - // A pre-feature store: unrelated keys present, but NONE of the six Full Potential keys. - defaults.set(0.0777, forKey: "stepDistance") - defaults.set(true, forKey: "aiCommandsEnabled") - for key in ["fullPotentialEnabled", "cpuLaneEnabled", "batchedRuntimeEnabled", - "mediaGenEnabled", "backgroundAutonomyEnabled", "fleetCloudEscalationEnabled"] { - XCTAssertNil(defaults.object(forKey: key), "precondition: \(key) not on disk") - } - - let s = AppSettings(defaults: defaults) - for (key, value) in allSix(s) { - XCTAssertFalse(value, "\(key) absent ⇒ false (legacy-load)") - } - // Pre-existing settings are untouched (purely additive). - XCTAssertEqual(s.stepDistance, 0.0777, accuracy: 1e-9) - XCTAssertTrue(s.aiCommandsEnabled) - } - - // MARK: - 2.3 Reset preserve-set (the six are retained, a normal tunable still resets) - - func testResetToDefaultsPreservesAllSixButResetsNormalTunable() { - let s = makeSettings() - s.fullPotentialEnabled = true - s.cpuLaneEnabled = true - s.batchedRuntimeEnabled = true - s.mediaGenEnabled = true - s.backgroundAutonomyEnabled = true - s.fleetCloudEscalationEnabled = true - // A normal tunable that SHOULD reset. - s.stepDistance = 0.5 - - s.resetToDefaults() - - for (key, value) in allSix(s) { - XCTAssertTrue(value, "\(key) is an AI opt-in — preserved by reset, not zeroed") - } - XCTAssertEqual(s.stepDistance, AppSettings.Defaults.stepDistance, accuracy: 1e-9, - "a normal tunable still resets") - } - - // MARK: - 3.1 Settings → gate adapter - - func testAdapterMapsSettingsToFlagsVerbatim() { - let s = makeSettings() - s.aiCommandsEnabled = true // consumed verbatim into aiCommandsEnabled - s.fullPotentialEnabled = true - s.mediaGenEnabled = true - // others stay off - - let flags = s.fullPotentialFlags - XCTAssertTrue(flags.aiCommandsEnabled, "enableAICommands flows into aiCommandsEnabled") - XCTAssertTrue(flags.fullPotentialEnabled) - XCTAssertTrue(flags.mediaGen) - XCTAssertFalse(flags.cpuLane) - XCTAssertFalse(flags.batchedRuntime) - XCTAssertFalse(flags.backgroundAutonomy) - XCTAssertFalse(flags.fleetCloud) - - // The convenience gate resolves from the same flags. - let gate = s.fullPotentialGate - XCTAssertTrue(gate.isUnlocked(.mediaGen), "media unlocked per the fixture") - XCTAssertFalse(gate.isUnlocked(.cpuLane), "cpu lane stays locked") - } - - /// The adapter honors the AI-commands gate: master + sub-flag on, but AI-commands off ⇒ locked. - func testAdapterGateLocksWhenAICommandsOff() { - let s = makeSettings() - s.aiCommandsEnabled = false - s.fullPotentialEnabled = true - s.mediaGenEnabled = true - - XCTAssertFalse(s.fullPotentialGate.isUnlocked(.mediaGen), - "AI-commands off ⇒ every capability locked (fleet is a subset of the AI feature)") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/GestureBindingsTests.swift b/Tests/ThreeFingerSwitcherTests/GestureBindingsTests.swift index 95c14cc..9de5d90 100644 --- a/Tests/ThreeFingerSwitcherTests/GestureBindingsTests.swift +++ b/Tests/ThreeFingerSwitcherTests/GestureBindingsTests.swift @@ -4,31 +4,13 @@ import XCTest /// Unit tests for the pure `GestureBindings` model (Gesture/GestureBindings.swift) and its persistence /// through `AppSettings`. /// -/// Covers: defaults reproduce today's mapping for all three surfaces; `assigning(...)` swaps on conflict -/// and never double-maps an excursion; reserved excursions are absent from the vocabularies; and the -/// `reverseDirection` / `reverseVerticalDirection` accessors round-trip through the switcher binding. +/// Covers: defaults reproduce today's behavior; the `reverseDirection` / `reverseVerticalDirection` +/// accessors round-trip through the switcher binding; persistence + reset; Codable round-trip (including +/// tolerance of retired surfaces' extra keys in older stored blobs). final class GestureBindingsTests: XCTestCase { // MARK: - Defaults reproduce today's behavior - /// The AI canvas default is exactly today's hardcoded mapping: down = commit, horizontal = dismiss - /// (bound to left), up = ignore. - func testCanvasDefaultsMatchTodaysMapping() { - let c = GestureBindings.CanvasBinding.default - XCTAssertEqual(c.commit, .swipeDown, "default down = commit") - XCTAssertEqual(c.dismiss, .swipeLeft, "default horizontal (left) = dismiss") - XCTAssertEqual(c.ignore, .swipeUp, "default up = ignore") - } - - /// The Files-drill default is exactly today's mapping: lift = open, +1-finger = Open-With, - /// four-finger horizontal = discard. - func testFilesDrillDefaultsMatchTodaysMapping() { - let f = GestureBindings.FilesDrillBinding.default - XCTAssertEqual(f.open, .lift, "default lift = open") - XCTAssertEqual(f.openWith, .plusOneFingerLift, "default +1-finger = Open-With") - XCTAssertEqual(f.discard, .fourFingerHorizontal, "default four-finger horizontal = discard") - } - /// The switcher default is both axes normal (no reversal) — exactly today's behavior. func testSwitcherDefaultsAreBothAxesNormal() { let s = GestureBindings.SwitcherBinding.default @@ -38,105 +20,12 @@ final class GestureBindingsTests: XCTestCase { XCTAssertFalse(s.spacesAxis.isReversed) } - /// The aggregate default wires all three surfaces to their per-surface defaults. + /// The aggregate default wires the switcher surface to its per-surface default. func testAggregateDefaultComposesPerSurfaceDefaults() { let g = GestureBindings.default - XCTAssertEqual(g.canvas, .default) - XCTAssertEqual(g.filesDrill, .default) XCTAssertEqual(g.switcher, .default) } - // MARK: - Reserved / invalid excursions are absent from the vocabularies - - /// Single-finger motion and the canvas's sub-threshold two-finger scroll are never bindable — the - /// canvas vocabulary is exactly the four two-finger swipe excursions, nothing else. - func testCanvasVocabularyExcludesReservedExcursions() { - let all = Set(GestureBindings.CanvasExcursion.allCases.map(\.rawValue)) - XCTAssertEqual(all, ["swipeUp", "swipeDown", "swipeLeft", "swipeRight"]) - // No single-finger / scroll/read member exists. - XCTAssertFalse(all.contains { $0.lowercased().contains("single") }) - XCTAssertFalse(all.contains { $0.lowercased().contains("scroll") }) - XCTAssertFalse(all.contains { $0.lowercased().contains("read") }) - XCTAssertFalse(all.contains { $0.lowercased().contains("pan") }) - } - - /// The Files-drill vocabulary is exactly its three excursions; single-finger is not a member. - func testFilesVocabularyExcludesReservedExcursions() { - let all = Set(GestureBindings.FilesExcursion.allCases.map(\.rawValue)) - XCTAssertEqual(all, ["lift", "plusOneFingerLift", "fourFingerHorizontal"]) - XCTAssertFalse(all.contains { $0.lowercased().contains("single") }) - } - - // MARK: - Canvas: assign swaps on conflict, never double-maps - - /// Assigning a taken excursion swaps the two actions so the mapping stays one-to-one. Spec scenario: - /// swipe-down is bound to commit; assigning swipe-down to dismiss makes commit inherit dismiss's old - /// excursion (no excursion maps to two actions). - func testCanvasAssignSwapsOnConflict() { - let c = GestureBindings.CanvasBinding.default // commit=down, dismiss=left, ignore=up - let r = c.assigning(.swipeDown, to: .dismiss) - - XCTAssertEqual(r.dismiss, .swipeDown, "dismiss now holds the requested excursion") - XCTAssertEqual(r.commit, .swipeLeft, "commit inherited dismiss's former excursion (the swap)") - XCTAssertEqual(r.ignore, .swipeUp, "the uninvolved action is untouched") - assertCanvasOneToOne(r) - } - - /// Assigning the excursion an action already holds is a no-op (returns an equal binding). - func testCanvasAssignSameExcursionIsNoOp() { - let c = GestureBindings.CanvasBinding.default - XCTAssertEqual(c.assigning(.swipeDown, to: .commit), c) - } - - /// Assigning the spare (4th) excursion — not held by any action — moves it to the target without a - /// swap, leaving the previously-held excursion now spare; the mapping stays one-to-one. - func testCanvasAssignSpareExcursionLeavesNoConflict() { - let c = GestureBindings.CanvasBinding.default // right is the spare - let r = c.assigning(.swipeRight, to: .commit) - - XCTAssertEqual(r.commit, .swipeRight) - XCTAssertEqual(r.dismiss, .swipeLeft, "untouched") - XCTAssertEqual(r.ignore, .swipeUp, "untouched") - assertCanvasOneToOne(r) - } - - /// After ANY canvas assignment the mapping is one-to-one (no excursion maps to two actions). - func testCanvasAssignmentsStayOneToOne() { - var c = GestureBindings.CanvasBinding.default - for action in GestureBindings.CanvasAction.allCases { - for excursion in GestureBindings.CanvasExcursion.allCases { - c = c.assigning(excursion, to: action) - XCTAssertEqual(c.excursion(for: action), excursion, "the target action holds the requested excursion") - assertCanvasOneToOne(c) - } - } - } - - // MARK: - Files: assign swaps on conflict, never double-maps - - /// Files: assigning a taken excursion swaps the two actions so the mapping stays one-to-one. - func testFilesAssignSwapsOnConflict() { - let f = GestureBindings.FilesDrillBinding.default // open=lift, openWith=+1, discard=4f - let r = f.assigning(.lift, to: .discard) - - XCTAssertEqual(r.discard, .lift, "discard now holds the requested excursion") - XCTAssertEqual(r.open, .fourFingerHorizontal, "open inherited discard's former excursion (the swap)") - XCTAssertEqual(r.openWith, .plusOneFingerLift, "uninvolved action untouched") - assertFilesOneToOne(r) - } - - /// Files: every assignment over the full vocabulary stays one-to-one (3 excursions, 3 actions). - func testFilesAssignmentsStayOneToOne() { - var f = GestureBindings.FilesDrillBinding.default - for action in GestureBindings.FilesAction.allCases { - for excursion in GestureBindings.FilesExcursion.allCases { - f = f.assigning(excursion, to: action) - XCTAssertEqual(f.excursion(for: action), excursion) - assertFilesOneToOne(f) - } - } - } - // MARK: - reverseDirection accessors round-trip through the switcher binding /// `AxisDirection(reversed:)` / `.isReversed` are exact inverses, so the boolean accessors round-trip. @@ -184,11 +73,9 @@ final class GestureBindingsTests: XCTestCase { let writer = AppSettings(defaults: defaults) writer.reverseDirection = true - writer.gestureBindings.canvas = writer.gestureBindings.canvas.assigning(.swipeRight, to: .commit) let reader = AppSettings(defaults: defaults) XCTAssertTrue(reader.reverseDirection, "the reverse choice survived the reload") - XCTAssertEqual(reader.gestureBindings.canvas.commit, .swipeRight, "the canvas remap survived the reload") } /// `resetToDefaults()` restores every gesture binding (including the folded reverse axes) to default. @@ -201,8 +88,6 @@ final class GestureBindingsTests: XCTestCase { let settings = AppSettings(defaults: defaults) settings.reverseDirection = true settings.reverseVerticalDirection = true - settings.gestureBindings.canvas = settings.gestureBindings.canvas.assigning(.swipeRight, to: .commit) - settings.gestureBindings.filesDrill = settings.gestureBindings.filesDrill.assigning(.lift, to: .discard) settings.resetToDefaults() @@ -216,8 +101,6 @@ final class GestureBindingsTests: XCTestCase { /// The whole model JSON-round-trips intact (the persistence shape used by `AppSettings`). func testCodableRoundTrip() throws { var g = GestureBindings.default - g.canvas = g.canvas.assigning(.swipeRight, to: .commit) - g.filesDrill = g.filesDrill.assigning(.lift, to: .discard) g.switcher.spacesAxis = .reversed let data = try JSONEncoder().encode(g) @@ -225,17 +108,16 @@ final class GestureBindingsTests: XCTestCase { XCTAssertEqual(decoded, g) } - // MARK: - Helpers - - /// Assert the canvas mapping is one-to-one: the three bound actions hold three distinct excursions. - private func assertCanvasOneToOne(_ c: GestureBindings.CanvasBinding) { - let bound = [c.commit, c.dismiss, c.ignore] - XCTAssertEqual(Set(bound).count, bound.count, "no excursion is bound to two actions") - } - - /// Assert the Files mapping is one-to-one: the three actions hold three distinct excursions. - private func assertFilesOneToOne(_ f: GestureBindings.FilesDrillBinding) { - let bound = [f.open, f.openWith, f.discard] - XCTAssertEqual(Set(bound).count, bound.count, "no excursion is bound to two actions") + /// An OLDER stored blob still carries the retired canvas / files-drill surfaces as extra JSON keys; + /// decoding must ignore them and keep the switcher binding (JSONDecoder skips unknown keys). + func testOlderBlobWithRetiredSurfacesStillDecodes() throws { + let legacy = """ + {"canvas":{"commit":"swipeDown","dismiss":"swipeLeft","ignore":"swipeUp"}, + "filesDrill":{"open":"lift","openWith":"plusOneFingerLift","discard":"fourFingerHorizontal"}, + "switcher":{"windowsAxis":"reversed","spacesAxis":"normal"}} + """.data(using: .utf8)! + let decoded = try JSONDecoder().decode(GestureBindings.self, from: legacy) + XCTAssertEqual(decoded.switcher.windowsAxis, .reversed, "the switcher binding survives") + XCTAssertEqual(decoded.switcher.spacesAxis, .normal) } } diff --git a/Tests/ThreeFingerSwitcherTests/GesturePoseTests.swift b/Tests/ThreeFingerSwitcherTests/GesturePoseTests.swift index c72404a..b4a77b8 100644 --- a/Tests/ThreeFingerSwitcherTests/GesturePoseTests.swift +++ b/Tests/ThreeFingerSwitcherTests/GesturePoseTests.swift @@ -239,18 +239,6 @@ final class GesturePoseTests: XCTestCase { "a lift journey is open + traverse only (rest-and-open, no excursion stroke)") } - func testCanvasResolveDownIsTwoFingerDirected() { - // A canvas commit: a single two-finger downward stroke from top-middle toward center. - let g = GesturePose.canvasResolve(.swipeDown) - XCTAssertEqual(g.strokes.count, 1) - let s = g.strokes[0] - XCTAssertEqual(s.fingers, 2, "the canvas resolve is a two-finger excursion") - // Pose coordinates are y-UP (the trackpad's bottom-left origin, matching the pad renderer's - // `(1 - y)` flip): a DOWNWARD stroke therefore starts at the HIGHER y and descends. - XCTAssertGreaterThan(s.from.y, s.to.y, "swipeDown strokes from the pad's top downward (top → center)") - XCTAssertEqual(s.from.x, s.to.x, accuracy: 1e-9, "a vertical resolve keeps X fixed") - } - func testLiftGapsEmitLifted() { // Across a multi-stroke gesture there must be frames where the hand is lifted (dots empty). let g = GesturePose.launcherOpen() @@ -275,10 +263,7 @@ final class GesturePoseTests: XCTestCase { GesturePose.launcherOpen(), GesturePose.bandJourney(bandFraction: 0.0, inSurface: .lift), GesturePose.bandJourney(bandFraction: 1.0, inSurface: .swipeDown), - GesturePose.bandJourney(bandFraction: 0.5, inSurface: .swipeHorizontal), - GesturePose.canvasResolve(.swipeUp), - GesturePose.canvasResolve(.swipeLeft), - GesturePose.canvasResolve(.swipeRight) + GesturePose.bandJourney(bandFraction: 0.5, inSurface: .swipeHorizontal) ] for g in builders { sweepGesture(g) { frame in diff --git a/Tests/ThreeFingerSwitcherTests/GestureRecognizerLauncherTests.swift b/Tests/ThreeFingerSwitcherTests/GestureRecognizerLauncherTests.swift index 57d72e3..7322b91 100644 --- a/Tests/ThreeFingerSwitcherTests/GestureRecognizerLauncherTests.swift +++ b/Tests/ThreeFingerSwitcherTests/GestureRecognizerLauncherTests.swift @@ -10,7 +10,6 @@ private final class LauncherMockDelegate: GestureRecognizerDelegate { case activate, step(Int), stepRow(Int), missionControl(Bool), commit, cancel case lActivate, lItem(Int), lContext(Int), lEnd, lCancel case lEdge(Int, Int) - case lCanvasResolve(Int, Int) } private(set) var events: [Event] = [] /// Simulates the launcher cursor sitting on the band-title list (left), so **vertical** travel is @@ -34,7 +33,6 @@ private final class LauncherMockDelegate: GestureRecognizerDelegate { func launcherDidCancel() { events.append(.lCancel) } func launcherEdgeChanged(dx: Int, dy: Int) { events.append(.lEdge(dx, dy)) } func launcherFocusIsOnBandList() -> Bool { onBandList } - func launcherCanvasResolve(dx: Int, dy: Int) { events.append(.lCanvasResolve(dx, dy)) } /// The sequence of edge states emitted (dx, dy) per change. var edges: [(Int, Int)] { events.compactMap { if case let .lEdge(x, y) = $0 { return (x, y) } else { return nil } } } @@ -45,10 +43,6 @@ private final class LauncherMockDelegate: GestureRecognizerDelegate { var lCancelCount: Int { events.filter { $0 == .lCancel }.count } var lItems: [Int] { events.compactMap { if case let .lItem(d) = $0 { return d } else { return nil } } } var lContexts: [Int] { events.compactMap { if case let .lContext(d) = $0 { return d } else { return nil } } } - /// The (dx, dy) of each canvas-resolution emitted (horizontal swipe = discard; dy −1 down = apply). - var canvasResolves: [(Int, Int)] { - events.compactMap { if case let .lCanvasResolve(x, y) = $0 { return (x, y) } else { return nil } } - } var commitCount: Int { events.filter { $0 == .commit }.count } var cancelCount: Int { events.filter { $0 == .cancel }.count } var didSwitcherActivate: Bool { events.contains(.activate) } @@ -311,148 +305,4 @@ final class GestureRecognizerLauncherTests: XCTestCase { XCTAssertEqual(d.lEndCount, 1) XCTAssertEqual(d.lCancelCount, 0) } - - // MARK: - Canvas-resolution mode (a fresh TWO-finger FLICK-LIFT resolves the open AI canvas) - // - // The canvas resolves on two fingers (4 = open/dismiss the platform, 2 = act within it). D4 - // (scroll-vs-flick): the resolve fires only on a genuine FLICK-LIFT — the dominant axis must (a) cross - // the travel floor (0.12), (b) reach a peak smoothed velocity ≥ `flickVelocityThreshold` (default 0.8), - // and (c) lift within `flickLiftWindow` (default 0.12s) of the last high-velocity frame. A slow - // deliberate scrub (reading the canvas) — sub-threshold peak velocity, or fingers held without a prompt - // lift — is SCROLL and never resolves. The defaults below match `AppSettings.Defaults`. - // - // Velocities here are synthetic (normalized units/sec). The lift frame is empty (.zero velocity), so the - // flick speed is carried from the last in-contact frame — the tests feed a fast in-contact frame, then a - // `fingers: 0` lift. - - func test_canvasFlick_horizontal_emitsDiscard() { - // A fast two-finger HORIZONTAL flick + lift resolves as a discard: dx != 0, dy == 0. - let (rec, d) = makeRecognizer(makeSettings(), launcher: true) - rec.launcherCanvasResolutionActive = true - feed(rec, x: 0.50, y: 0.50, fingers: 2, t: 0.00) // begin - feed(rec, x: 0.64, y: 0.50, fingers: 2, vx: 2.0, t: 0.02) // dx +0.14, fast → flick candidate - feed(rec, x: 0.64, y: 0.50, fingers: 0, t: 0.03) // prompt lift → discard - XCTAssertEqual(d.canvasResolves.map { [$0.0, $0.1] }, [[1, 0]], "horizontal flick → discard (dx=+1)") - XCTAssertEqual(d.lActivateCount, 0, "resolution mode never opens the launcher") - XCTAssertFalse(d.didSwitcherActivate) - } - - func test_canvasFlick_down_emitsApply() { - // A fast two-finger DOWN flick (y decreases in OMS coords) + lift resolves as apply: dy = −1. - let (rec, d) = makeRecognizer(makeSettings(), launcher: true) - rec.launcherCanvasResolutionActive = true - feed(rec, x: 0.50, y: 0.50, fingers: 2, t: 0.00) - feed(rec, x: 0.50, y: 0.36, fingers: 2, vy: -2.0, t: 0.02) // dy −0.14, fast → flick candidate - feed(rec, x: 0.50, y: 0.36, fingers: 0, t: 0.03) // prompt lift → apply - XCTAssertEqual(d.canvasResolves.map { [$0.0, $0.1] }, [[0, -1]], "down flick → apply (dy=−1)") - } - - func test_canvasFlick_up_emitsUp() { - // A fast UP flick + lift emits dy = +1 (the coordinator parks/ignores it; the recognizer just reports - // the axis-locked sign). - let (rec, d) = makeRecognizer(makeSettings(), launcher: true) - rec.launcherCanvasResolutionActive = true - feed(rec, x: 0.50, y: 0.50, fingers: 2, t: 0.00) - feed(rec, x: 0.50, y: 0.64, fingers: 2, vy: 2.0, t: 0.02) // dy +0.14, fast upward - feed(rec, x: 0.50, y: 0.64, fingers: 0, t: 0.03) // prompt lift → dy=+1 - XCTAssertEqual(d.canvasResolves.map { [$0.0, $0.1] }, [[0, 1]], "up flick → dy=+1") - } - - func test_canvasScroll_slowPastThreshold_doesNotResolve() { - // D4: a SLOW deliberate scrub past the 0.12 travel floor with LOW velocity is reading-scroll — even - // with a lift it must NOT resolve (peak velocity never crossed the flick threshold). - let (rec, d) = makeRecognizer(makeSettings(), launcher: true) - rec.launcherCanvasResolutionActive = true - feed(rec, x: 0.50, y: 0.50, fingers: 2, t: 0.00) - feed(rec, x: 0.57, y: 0.50, fingers: 2, vx: 0.1, t: 0.20) // dx +0.07, slow - feed(rec, x: 0.64, y: 0.50, fingers: 2, vx: 0.1, t: 0.40) // dx +0.14 past floor, still slow - feed(rec, x: 0.64, y: 0.50, fingers: 0, t: 0.41) // lift after a slow scrub - XCTAssertTrue(d.canvasResolves.isEmpty, "a slow scrub past the floor is SCROLL, never a resolve") - XCTAssertEqual(d.lActivateCount, 0) - XCTAssertFalse(d.didSwitcherActivate) - } - - func test_canvasScroll_fastButHeldWithoutLift_doesNotResolve() { - // A fast excursion past the floor that is HELD (fingers still down, no lift) never resolves — the - // resolve fires only on the lift frame. (A continuous scrub that keeps the fingers down is scroll.) - let (rec, d) = makeRecognizer(makeSettings(), launcher: true) - rec.launcherCanvasResolutionActive = true - feed(rec, x: 0.50, y: 0.50, fingers: 2, t: 0.00) - feed(rec, x: 0.64, y: 0.50, fingers: 2, vx: 2.0, t: 0.02) // fast + past floor, but still down… - feed(rec, x: 0.70, y: 0.50, fingers: 2, vx: 0.05, t: 0.30) // …and decelerated, still no lift - XCTAssertTrue(d.canvasResolves.isEmpty, "no lift → no resolve while fingers are held") - } - - func test_canvasScroll_subThresholdPeakVelocity_doesNotResolve() { - // Crosses the 0.12 distance floor but the peak velocity stays BELOW `flickVelocityThreshold`, so even - // a prompt lift is classified SCROLL, not a flick. - let (rec, d) = makeRecognizer(makeSettings(), launcher: true) - rec.launcherCanvasResolutionActive = true - feed(rec, x: 0.50, y: 0.50, fingers: 2, t: 0.00) - feed(rec, x: 0.64, y: 0.50, fingers: 2, vx: 0.5, t: 0.02) // dx +0.14 but vx 0.5 < 0.8 - feed(rec, x: 0.64, y: 0.50, fingers: 0, t: 0.03) // prompt lift, but sub-threshold peak - XCTAssertTrue(d.canvasResolves.isEmpty, "sub-threshold peak velocity → scroll, no resolve") - } - - func test_canvasFlick_belowTravelFloor_doesNotResolve() { - // Fast but BELOW the 0.12 travel floor → no resolve (the floor is still a minimum-travel gate). - let (rec, d) = makeRecognizer(makeSettings(), launcher: true) - rec.launcherCanvasResolutionActive = true - feed(rec, x: 0.50, y: 0.50, fingers: 2, t: 0.00) - feed(rec, x: 0.56, y: 0.50, fingers: 2, vx: 2.0, t: 0.02) // dx +0.06 < 0.12, even though fast - feed(rec, x: 0.56, y: 0.50, fingers: 0, t: 0.03) // lift - XCTAssertTrue(d.canvasResolves.isEmpty, "below the travel floor never resolves") - } - - func test_canvasFlick_pauseBeforeLift_doesNotResolve() { - // A fast frame followed by a PAUSE (the lift arrives well after the last fast frame, beyond - // `flickLiftWindow`) is a decelerated hold, not a flick-lift → no resolve. - let (rec, d) = makeRecognizer(makeSettings(), launcher: true) - rec.launcherCanvasResolutionActive = true - feed(rec, x: 0.50, y: 0.50, fingers: 2, t: 0.00) - feed(rec, x: 0.64, y: 0.50, fingers: 2, vx: 2.0, t: 0.02) // fast, last fast frame at t=0.02 - feed(rec, x: 0.65, y: 0.50, fingers: 2, vx: 0.05, t: 0.50) // long pause; last contact at t=0.50 - feed(rec, x: 0.65, y: 0.50, fingers: 0, t: 0.51) // lift, 0.48s after the last fast frame - XCTAssertTrue(d.canvasResolves.isEmpty, "a pause past the flick window before lift → no resolve") - } - - func test_canvasFlick_emitsOncePerGesture() { - // A flick-lift resolves exactly once; the lift frame's one-shot guard plus the reset mean no - // double-emit even if extra frames arrive. - let (rec, d) = makeRecognizer(makeSettings(), launcher: true) - rec.launcherCanvasResolutionActive = true - feed(rec, x: 0.50, y: 0.50, fingers: 2, t: 0.00) - feed(rec, x: 0.64, y: 0.50, fingers: 2, vx: 2.0, t: 0.02) // fast - feed(rec, x: 0.64, y: 0.50, fingers: 0, t: 0.03) // lift → resolve once - feed(rec, x: 0.64, y: 0.50, fingers: 0, t: 0.04) // stray re-lift → no-op - XCTAssertEqual(d.canvasResolves.count, 1, "exactly one resolution per flick-lift") - } - - func test_canvasFlick_liftReArmsAndResetsVelocityFields() { - // After a flick-lift, a second fresh two-finger flick resolves again — and the velocity/lift fields - // reset, so a stale peak from the first gesture doesn't promote a slow second scrub into a flick. - let (rec, d) = makeRecognizer(makeSettings(), launcher: true) - rec.launcherCanvasResolutionActive = true - // Flick #1 (horizontal discard). - feed(rec, x: 0.50, y: 0.50, fingers: 2, t: 0.00) - feed(rec, x: 0.64, y: 0.50, fingers: 2, vx: 2.0, t: 0.02) - feed(rec, x: 0.64, y: 0.50, fingers: 0, t: 0.03) - // A SLOW second scrub — if the peak velocity hadn't reset it would falsely resolve. - feed(rec, x: 0.50, y: 0.50, fingers: 2, t: 0.10) - feed(rec, x: 0.50, y: 0.36, fingers: 2, vy: 0.1, t: 0.40) // dy −0.14 past floor but SLOW - feed(rec, x: 0.50, y: 0.36, fingers: 0, t: 0.41) - XCTAssertEqual(d.canvasResolves.map { [$0.0, $0.1] }, [[1, 0]], - "only the first (fast) flick resolves; the slow re-armed scrub does not") - } - - func test_canvasFlick_relaxToTwoFingers_stillResolves() { - // Matching the launcher latch feel: after a four-finger start the user may relax to two and a - // flick-lift still resolves. - let (rec, d) = makeRecognizer(makeSettings(), launcher: true) - rec.launcherCanvasResolutionActive = true - feed(rec, x: 0.50, y: 0.50, fingers: 4, t: 0.00) // begin (relaxed-to-two latch) - feed(rec, x: 0.50, y: 0.38, fingers: 2, vy: -2.0, t: 0.02) // relaxed + fast downward - feed(rec, x: 0.50, y: 0.38, fingers: 0, t: 0.03) // flick-lift → apply - XCTAssertEqual(d.canvasResolves.map { [$0.0, $0.1] }, [[0, -1]]) - } } diff --git a/Tests/ThreeFingerSwitcherTests/HubLauncherDemoTests.swift b/Tests/ThreeFingerSwitcherTests/HubLauncherDemoTests.swift index 523e874..8bbe386 100644 --- a/Tests/ThreeFingerSwitcherTests/HubLauncherDemoTests.swift +++ b/Tests/ThreeFingerSwitcherTests/HubLauncherDemoTests.swift @@ -42,16 +42,9 @@ final class HubLauncherDemoTests: XCTestCase { // MARK: - Gesture grammar - func testBandJourneyGestureWithResolveAppendsFreshTail() { + func testBandJourneyGestureIsOpenPlusTraverse() { let plain = HubLauncherDemo.bandJourneyGesture(openLength: 0.3) XCTAssertEqual(plain.strokes.map(\.fingers), [4, 2], "open + traverse only") - let resolving = HubLauncherDemo.bandJourneyGesture(openLength: 0.3, resolve: .swipeDown) - XCTAssertEqual(resolving.strokes.map(\.fingers), [4, 2, 2], - "a resolve journey appends the fresh two-finger canvas swipe") - XCTAssertNil(resolving.strokes[1].gapAfter, - "the traverse LIFTS before the resolve (the firing lift — resolution is a fresh excursion)") - let tail = resolving.strokes[2] - XCTAssertGreaterThan(tail.from.y, tail.to.y, "the commit tail strokes downward (y-up coordinates)") } func testVerticalStrokesDescendInYUpCoordinates() { @@ -115,7 +108,7 @@ final class HubLauncherDemoTests: XCTestCase { XCTAssertEqual(demo.model.selectedIndex, sel, "replayed frames don't re-step the highlight") } - // MARK: - Band-journey drive (Clipboard / Files / AI pages) + // MARK: - Band-journey drive (the Clipboard page) func testDriveBandJourneyWalksToLastBandAndCloses() { let demo = seededDemo(landOnLastBand: true) @@ -137,25 +130,6 @@ final class HubLauncherDemoTests: XCTestCase { XCTAssertFalse(demo.overlayShown, "the lift closes the loop") } - func testDriveResolveJourneyKeepsPanelUpThroughFiringLift() { - let demo = seededDemo(landOnLastBand: true) - demo.drive(pose(2, 1.0, lifted: true), script: .bandJourneyResolve) // prior loop's close - demo.drive(pose(0, 0.8), script: .bandJourneyResolve) - demo.drive(pose(1, 1.0), script: .bandJourneyResolve) - XCTAssertEqual(demo.model.currentBand, 2) - - // The FIRING lift (after the traverse, before the resolve tail) keeps the panel up — the real - // launcher stays visible to show the canvas. - demo.drive(pose(1, 1.0, lifted: true), script: .bandJourneyResolve) - XCTAssertTrue(demo.overlayShown, "the firing lift never closes the panel") - - // The resolve tail drives nothing; its closing lift ends the loop. - demo.drive(pose(2, 0.5), script: .bandJourneyResolve) - XCTAssertEqual(demo.model.currentBand, 2, "the canvas tail leaves the model untouched") - demo.drive(pose(2, 1.0, lifted: true), script: .bandJourneyResolve) - XCTAssertFalse(demo.overlayShown, "the resolve's lift closes the loop") - } - func testHoverPresentsThePageBandStatically() { let demo = seededDemo(landOnLastBand: true) // Mid-loop, hidden, partway down the rail… diff --git a/Tests/ThreeFingerSwitcherTests/LLMRuntimeStubTests.swift b/Tests/ThreeFingerSwitcherTests/LLMRuntimeStubTests.swift deleted file mode 100644 index 3dd498f..0000000 --- a/Tests/ThreeFingerSwitcherTests/LLMRuntimeStubTests.swift +++ /dev/null @@ -1,296 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for the swappable model seam exercised through `StubLLMRuntime`: streaming order + -/// finality, prompt-cancellation stops generation, capability reporting, and the structured -/// validate → repair/retry → decode → decline pipeline (design D1/D2). -final class LLMRuntimeStubTests: XCTestCase { - - // A simple Decodable target for the structured tests. - private struct CalendarEvent: Decodable, Equatable, Sendable { - let title: String - let start: String - } - - private let calendarSchema = StructuredSchema( - name: "calendar_event", - json: #"{"type":"object","required":["title","start"]}"# - ) - - // MARK: Streaming - - func testStreamingEmitsScriptedTokensInOrderWithFinalFlag() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["Hello", ", ", "world"], interTokenDelayNanos: 0) - var texts: [String] = [] - var finals: [Bool] = [] - for try await token in stub.generate(LLMRequest(prompt: "ignored")) { - texts.append(token.text) - finals.append(token.isFinal) - } - XCTAssertEqual(texts, ["Hello", ", ", "world"], "tokens stream in scripted order") - XCTAssertEqual(finals, [false, false, true], "only the last token is marked final") - } - - func testGenerateTextConcatenatesStream() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["a", "b", "c"], interTokenDelayNanos: 0) - let full = try await stub.generateText(LLMRequest(prompt: "x")) - XCTAssertEqual(full, "abc") - } - - func testEmptyScriptEchoesPrompt() async throws { - let stub = StubLLMRuntime(scriptedTokens: [], interTokenDelayNanos: 0) - let full = try await stub.generateText(LLMRequest(prompt: "echo me")) - XCTAssertEqual(full, "echo me", "an unscripted stub echoes the prompt") - } - - // MARK: Cancellation - - func testCancellationStopsGeneration() async throws { - // A slow stream so we can cancel after the first token. 20 chunks at 20ms each. - let stub = StubLLMRuntime(scriptedTokens: Array(repeating: "x", count: 20), - interTokenDelayNanos: 20_000_000) - let counter = TokenCounter() - - // Consume the stream explicitly so we can cancel the *stream-backing* Task (via the iterator - // being dropped on cancellation) and observe the runtime's terminal error. - let stream = stub.generate(LLMRequest(prompt: "p")) - let task = Task { - do { - for try await _ in stream { - await counter.bump() - // Stop consuming after the first token; the stream's onTermination cancels the - // producer, which then finishes the stream with .cancelled on its next checkpoint. - if await counter.value >= 1 { break } - } - return nil - } catch let e as RuntimeError { - return e - } catch is CancellationError { - return .cancelled - } catch { - return nil - } - } - - // Let a couple of tokens flow, then cancel the consuming task. - try await Task.sleep(nanoseconds: 50_000_000) - task.cancel() - _ = await task.value - - // The decisive property: generation did NOT run to completion — cancellation stopped it. - let count = await counter.value - XCTAssertLessThan(count, 20, "cancellation stops generation before all tokens are emitted") - } - - /// Verifies the runtime DETECTS cancellation and stops the generation work (mapping it to - /// `RuntimeError.cancelled` internally). We observe the stub's deterministic `observedCancellation` - /// flag rather than racing the consumer-side iterator termination: when the consuming task is - /// cancelled, the stream's onTermination cancels the producer, whose checkpoint throws - /// `CancellationError` and is recorded. - func testCancellationIsDetectedByTheRuntime() async throws { - let stub = StubLLMRuntime(scriptedTokens: Array(repeating: "x", count: 50), - interTokenDelayNanos: 10_000_000) - let task = Task { - for try await _ in stub.generate(LLMRequest(prompt: "p")) { - // keep consuming until cancelled - } - } - try await Task.sleep(nanoseconds: 30_000_000) - task.cancel() - _ = try? await task.value - - // Poll the deterministic flag with a bounded wait (producer cancel → catch is async). - var observed = stub.observedCancellation - for _ in 0..<50 where !observed { - try await Task.sleep(nanoseconds: 5_000_000) - observed = stub.observedCancellation - } - XCTAssertTrue(observed, "the runtime detected cancellation and stopped generation") - } - - // MARK: Capabilities - - func testCapabilityReporting() { - XCTAssertEqual(StubLLMRuntime().capabilities, [.text, .vision]) - XCTAssertEqual(StubLLMRuntime(capabilities: [.text]).capabilities, [.text]) - } - - func testVisionRequestAgainstTextOnlyRuntimeErrors() async { - let stub = StubLLMRuntime(capabilities: [.text], interTokenDelayNanos: 0) - let request = LLMRequest(prompt: "what is this?", image: Data([0x1, 0x2])) - do { - _ = try await stub.generateText(request) - XCTFail("a vision request on a text-only runtime must error, not degrade") - } catch let e as RuntimeError { - XCTAssertEqual(e, .unsupportedModality(.vision)) - } catch { - XCTFail("unexpected error: \(error)") - } - } - - /// The positive vision path: a `.vision`-capable runtime (the default [.text, .vision]) accepts an - /// image-bearing request and streams the scripted result — no `unsupportedModality` error. - func testVisionRequestAgainstVisionCapableRuntimeStreams() async throws { - let stub = StubLLMRuntime(scriptedTokens: ["a ", "cat"], interTokenDelayNanos: 0) - XCTAssertTrue(stub.capabilities.contains(.vision), "default stub is vision-capable") - let request = LLMRequest(prompt: "what is this?", image: Data([0xFF, 0xD8, 0xFF])) - XCTAssertTrue(request.requiresVision, "an image-bearing request requires vision") - let full = try await stub.generateText(request) - XCTAssertEqual(full, "a cat", "a vision-capable runtime streams the scripted result for an image request") - } - - // MARK: Structured — valid - - func testStructuredDecodesValidValue() async throws { - let stub = StubLLMRuntime( - structuredScript: .valid(json: #"{"title":"Sync","start":"2026-06-08T10:00"}"#) - ) - let outcome = try await stub.structured(LLMRequest(prompt: "p"), schema: calendarSchema, as: CalendarEvent.self) - XCTAssertEqual(outcome.value, CalendarEvent(title: "Sync", start: "2026-06-08T10:00")) - XCTAssertFalse(outcome.isDeclined) - XCTAssertEqual(stub.lastAttemptCount, 1, "a valid first attempt needs no repair") - } - - // MARK: Structured — repair - - func testStructuredRepairsNonConformingThenSucceeds() async throws { - // First emission misses the required `start` key → validation fails → repair attempt succeeds. - let stub = StubLLMRuntime( - structuredScript: .invalidThenRepaired( - bad: #"{"title":"Sync"}"#, - good: #"{"title":"Sync","start":"2026-06-08T10:00"}"# - ) - ) - let outcome = try await stub.structured(LLMRequest(prompt: "p"), schema: calendarSchema, as: CalendarEvent.self) - XCTAssertEqual(outcome.value?.title, "Sync") - XCTAssertEqual(stub.lastAttemptCount, 2, "the bounded loop took one repair attempt") - } - - func testStructuredExhaustsBoundedLoopOnPersistentlyInvalid() async { - let stub = StubLLMRuntime( - structuredScript: .alwaysInvalid(json: #"{"title":"Sync"}"#), // never has `start` - maxRepairAttempts: 3 - ) - do { - _ = try await stub.structured(LLMRequest(prompt: "p"), schema: calendarSchema, as: CalendarEvent.self) - XCTFail("persistently non-conforming output must not be returned as a value") - } catch let e as RuntimeError { - XCTAssertEqual(e, .couldNotProduceValid(attempts: 3), "the loop is bounded and reports failure") - } catch { - XCTFail("unexpected error: \(error)") - } - } - - // MARK: Structured — decline - - func testStructuredCanDeclineRatherThanFabricate() async throws { - let stub = StubLLMRuntime( - structuredScript: .decline(reason: "This text is not a meeting") - ) - let outcome = try await stub.structured(LLMRequest(prompt: "p"), schema: calendarSchema, as: CalendarEvent.self) - XCTAssertTrue(outcome.isDeclined) - XCTAssertEqual(outcome.declineReason, "This text is not a meeting") - XCTAssertNil(outcome.value, "a decline carries no fabricated value") - } - - // MARK: Structured — schema validation in isolation - - /// Directly exercises `StubLLMRuntime.validate(jsonData:against:)` to prove the schema gate is a - /// real check, not a pass-through: an object missing a required key is rejected, while an object - /// carrying every required key (regardless of extra keys) passes. Non-object JSON also fails. - func testValidateRejectsOutputMissingRequiredKey() { - let titleAndStart = StructuredSchema( - name: "calendar_event", - json: #"{"type":"object","required":["title","start"]}"# - ) - func data(_ s: String) -> Data { Data(s.utf8) } - - // Missing the required `start` key → rejected. - XCTAssertFalse( - StubLLMRuntime.validate(jsonData: data(#"{"title":"Sync"}"#), against: titleAndStart), - "an object missing a required key must fail validation" - ) - // All required keys present (plus an extra) → accepted. - XCTAssertTrue( - StubLLMRuntime.validate(jsonData: data(#"{"title":"Sync","start":"t","extra":1}"#), against: titleAndStart), - "an object carrying every required key passes validation" - ) - // Non-object JSON → rejected. - XCTAssertFalse( - StubLLMRuntime.validate(jsonData: data("[1,2,3]"), against: titleAndStart), - "non-object JSON fails validation" - ) - // A schema with no `required` array accepts any object. - let noRequired = StructuredSchema(name: "freeform", json: #"{"type":"object"}"#) - XCTAssertTrue( - StubLLMRuntime.validate(jsonData: data(#"{"anything":true}"#), against: noRequired), - "a schema with no required keys accepts any object" - ) - } - - /// Proves `validate()` actually runs inside the pipeline: an always-invalid emission that is missing - /// a required key is never returned as a value; the bounded loop exhausts and reports failure with - /// the attempt count. (Distinct from `testStructuredExhaustsBoundedLoopOnPersistentlyInvalid`, this - /// pins the rejection to schema validation by using a smaller, explicit budget.) - func testStructuredRejectsMissingRequiredKeyAndExhausts() async { - let stub = StubLLMRuntime( - structuredScript: .alwaysInvalid(json: #"{"title":"Sync"}"#), // missing required `start` - maxRepairAttempts: 2 - ) - do { - _ = try await stub.structured(LLMRequest(prompt: "p"), schema: calendarSchema, as: CalendarEvent.self) - XCTFail("output missing a required key must be rejected by validation, never returned") - } catch let e as RuntimeError { - XCTAssertEqual(e, .couldNotProduceValid(attempts: 2), - "validation rejects every attempt, so the bounded loop exhausts") - XCTAssertEqual(stub.lastAttemptCount, 2, "each attempt was validated and rejected") - } catch { - XCTFail("unexpected error: \(error)") - } - } - - // MARK: Structured — cancellation - - /// Cancelling the Task that drives a `structured(...)` call makes it throw rather than burning the - /// full repair budget. We pre-cancel the task before its body runs (deterministic: no real sleep, - /// no wall-clock race), so the call's first `Task.checkCancellation()` checkpoint trips. The script - /// is `alwaysInvalid` with a large budget so, absent cancellation, the call WOULD run many attempts - /// — letting us assert it did NOT. - func testStructuredCancellationThrowsBeforeExhaustingBudget() async { - let stub = StubLLMRuntime( - structuredScript: .alwaysInvalid(json: #"{"title":"Sync"}"#), - maxRepairAttempts: 1000 - ) - // Create the driving task, then cancel it before yielding control so its body observes - // cancellation at the very first checkpoint. - let task = Task { () -> Error? in - do { - _ = try await stub.structured(LLMRequest(prompt: "p"), - schema: calendarSchema, as: CalendarEvent.self) - return nil - } catch { - return error - } - } - task.cancel() - let thrown = await task.value - - switch thrown { - case is CancellationError: - break - case let e as RuntimeError where e == .cancelled: - break - default: - XCTFail("cancellation must surface as CancellationError/RuntimeError.cancelled, got \(String(describing: thrown))") - } - // The decisive property: it bailed out, it did NOT run the full repair budget. - XCTAssertLessThan(stub.lastAttemptCount, 1000, - "cancellation stops the repair loop instead of running the full budget") - } -} - -/// Tiny actor to count emitted tokens across the cancellation boundary without data races. -private actor TokenCounter { - private(set) var value = 0 - func bump() { value += 1 } -} diff --git a/Tests/ThreeFingerSwitcherTests/LaunchItemTests.swift b/Tests/ThreeFingerSwitcherTests/LaunchItemTests.swift index adb6dba..5eadce6 100644 --- a/Tests/ThreeFingerSwitcherTests/LaunchItemTests.swift +++ b/Tests/ThreeFingerSwitcherTests/LaunchItemTests.swift @@ -106,24 +106,25 @@ final class LaunchItemTests: XCTestCase { XCTAssertNil(newWindow, "a legacy .url decodes with no new-window flag") } - /// The synthetic `.aiCommand` kind round-trips through Codable (it shares the launcher plumbing), - /// even though it is never written into the persisted Favorites record (it's built fresh on open). - func testAICommandKindEncodesAndDecodes() throws { - let command = AICommand(name: "Fix Grammar", icon: .sfSymbol("text.badge.checkmark"), - tint: ItemColor(red: 0.25, green: 0.72, blue: 0.40), - input: .selection, - promptTemplate: "Fix: {input}", - output: .runTask(.addToCalendar), - model: .onDevice(modelID: "gemma-4-31b"), - confirmBeforeRun: true) - let item = LaunchItem(id: command.id, title: command.name, icon: command.icon, - tint: command.tint, kind: .aiCommand(command)) - let data = try JSONEncoder().encode(item) - let back = try JSONDecoder().decode(LaunchItem.self, from: data) - XCTAssertEqual(back, item, "the .aiCommand kind and its AICommand payload round-trip") - guard case let .aiCommand(decoded) = back.kind else { return XCTFail("kind is .aiCommand") } - XCTAssertEqual(decoded, command, "the carried AICommand survives the round-trip intact") - XCTAssertFalse(item.isConsequential, "an AI command is not a fire-notification kind") + /// A stored band that still carries a RETIRED item kind (the removed `.aiCommand`, or any future + /// unknown) decodes with that item DROPPED, never failing the whole band (the v3 lossy decode — + /// a full-record decode failure would reseed and wipe the user's bands). + func testBandWithRetiredItemKindDecodesLossily() throws { + let keeper = LaunchItem(title: "Keeper", icon: .sfSymbol("star"), + kind: .url(URL(string: "https://example.com")!)) + let band = ContextBand(name: "Mixed", color: ItemColor(red: 0, green: 0, blue: 1), + items: [keeper]) + var json = try JSONSerialization.jsonObject(with: JSONEncoder().encode(band)) as! [String: Any] + var items = json["items"] as! [[String: Any]] + // A hand-built legacy `.aiCommand` item (the retired kind's on-disk shape). + items.append(["id": UUID().uuidString, "title": "Old AI", + "icon": ["sfSymbol": ["_0": "wand.and.stars"]], + "kind": ["aiCommand": ["_0": ["anything": "at all"]]]]) + json["items"] = items + let data = try JSONSerialization.data(withJSONObject: json) + let back = try JSONDecoder().decode(ContextBand.self, from: data) + XCTAssertEqual(back.items.map(\.title), ["Keeper"], + "the retired-kind item is dropped; the rest of the band survives") } // MARK: - Persistence (versioned key) diff --git a/Tests/ThreeFingerSwitcherTests/LauncherCanvasModeTests.swift b/Tests/ThreeFingerSwitcherTests/LauncherCanvasModeTests.swift deleted file mode 100644 index 3ba4053..0000000 --- a/Tests/ThreeFingerSwitcherTests/LauncherCanvasModeTests.swift +++ /dev/null @@ -1,223 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for the launcher's AI preview-canvas state machine (spec launcher-overlay MODIFIED: -/// "Armed AI command lift opens the preview canvas" + "Swipe-to-resolve (commit / discard)"). These -/// cover the genuinely-headless logic: the model's canvas state, and the controller's lift / commit / -/// discard transitions wired to the executor callbacks — WITHOUT real gestures (the gesture feel is in -/// the manual-test checklist). Every NON-AI lift-fire-and-dismiss path is asserted unchanged. -@MainActor -final class LauncherCanvasModeTests: XCTestCase { - - private func appItem(_ name: String) -> LaunchItem { - LaunchItem(title: name, icon: .appDefault, - kind: .app(bundleURL: URL(fileURLWithPath: "/Applications/\(name).app"), strategy: nil)) - } - - private func aiItem(_ name: String) -> LaunchItem { - let command = AICommand(name: name, icon: .sfSymbol("sparkles"), input: .selection, - promptTemplate: "{input}", output: .previewOnly) - return LaunchItem(id: command.id, title: name, icon: command.icon, kind: .aiCommand(command)) - } - - // MARK: - Model canvas state - - func testModelEnterAndExitCanvas() { - let model = LauncherModel() - let cmd = AICommand(name: "Fix", icon: .emoji("✅"), input: .selection, - promptTemplate: "{input}", output: .previewOnly) - XCTAssertFalse(model.canvasActive) - model.enterCanvas(cmd) - XCTAssertTrue(model.canvasActive) - XCTAssertEqual(model.canvasCommand, cmd) - model.exitCanvas() - XCTAssertFalse(model.canvasActive) - XCTAssertNil(model.canvasCommand) - } - - // MARK: - Controller: armed AI lift opens the canvas (does NOT dismiss) - - func testArmedAILiftOpensCanvasAndDoesNotDismiss() { - let controller = LauncherOverlayController() - var fired: [LaunchItem] = [] - var committed = 0 - var discarded = 0 - controller.onFire = { item, _ in fired.append(item) } - controller.onCommitCanvas = { committed += 1 } - controller.onDiscardCanvas = { discarded += 1 } - - let ai = aiItem("Summarize") - let band = ContextBand(id: AIBand.bandID, name: "AI", - color: AIBand.color, items: [ai]) - controller.show(bands: [band], startBand: 0, startColumn: 0, dwell: 0.01) - controller.model.setArmed() // simulate the dwell having armed the AI item - - let result = controller.end() // first lift - XCTAssertTrue(result, "an armed AI lift reports a fire") - XCTAssertEqual(fired.map(\.title), ["Summarize"], "the AI command is handed off via onFire") - XCTAssertTrue(controller.canvasActive, "the canvas is open — the overlay did NOT dismiss") - XCTAssertTrue(controller.isVisible, "the panel stays visible behind the canvas") - XCTAssertEqual(committed, 0) - XCTAssertEqual(discarded, 0) - controller.cancel() // teardown (also discards the open canvas) - } - - // MARK: - Controller: a second lift is a NO-OP (commit is a fresh DOWN swipe, not a lift) - - func testSecondLiftIsNoOpWhileCanvasOpen() { - // After the firing lift opens the canvas the fingers are already up, so a re-touch-and-lift - // must NOT resolve the canvas — the canvas is now resolved by a fresh four-finger swipe - // (`resolveCanvasCommit` / `discardCanvas`). A stray lift therefore leaves the canvas intact. - let controller = LauncherOverlayController() - var committed = 0 - var discarded = 0 - controller.onFire = { _, _ in } - controller.onCommitCanvas = { committed += 1 } - controller.onDiscardCanvas = { discarded += 1 } - - let ai = aiItem("Explain") - let band = ContextBand(id: AIBand.bandID, name: "AI", - color: AIBand.color, items: [ai]) - controller.show(bands: [band], startBand: 0, startColumn: 0, dwell: 0.01) - controller.model.setArmed() - controller.end() // open the canvas - XCTAssertTrue(controller.canvasActive) - - let liftResult = controller.end() // a second lift is a no-op - XCTAssertTrue(liftResult, "the lift still reports handled (the canvas owns the gesture)") - XCTAssertEqual(committed, 0, "a lift never commits") - XCTAssertEqual(discarded, 0, "a lift never discards — the canvas waits for a resolving swipe") - XCTAssertTrue(controller.canvasActive, "the canvas stays open for a resolving swipe") - XCTAssertTrue(controller.isVisible) - controller.cancel() // teardown (this legitimately discards the open canvas) - } - - // MARK: - Controller: a DOWN-swipe resolution commits and dismisses - - func testResolveCanvasCommitCommitsAndDismisses() { - let controller = LauncherOverlayController() - var committed = 0 - controller.onFire = { _, _ in } - controller.onCommitCanvas = { committed += 1 } - controller.onDiscardCanvas = { XCTFail("commit must not discard") } - - let ai = aiItem("Explain") - let band = ContextBand(id: AIBand.bandID, name: "AI", - color: AIBand.color, items: [ai]) - controller.show(bands: [band], startBand: 0, startColumn: 0, dwell: 0.01) - controller.model.setArmed() - controller.end() // open the canvas - XCTAssertTrue(controller.canvasActive) - - controller.resolveCanvasCommit() // the DOWN-swipe resolution - XCTAssertEqual(committed, 1, "a down-swipe commits exactly once") - XCTAssertFalse(controller.canvasActive, "committing closes the canvas") - XCTAssertFalse(controller.isVisible, "and dismisses the overlay") - } - - // MARK: - Controller: resolveCanvasCommit is a no-op when there is no open canvas - - func testResolveCanvasCommitNoOpWhenCanvasClosed() { - let controller = LauncherOverlayController() - controller.onCommitCanvas = { XCTFail("nothing to commit when the canvas is closed") } - controller.resolveCanvasCommit() // canvas never opened → must be inert - XCTAssertFalse(controller.canvasActive) - } - - // MARK: - Controller: a deliberate horizontal excursion discards and dismisses (fallback path) - - func testHorizontalExcursionDiscardsAndDismisses() { - // Exercises the controller-level fallback discard (`stepHorizontal` → `accumulateCanvasDiscard`). - // In production the recognizer's canvas-resolution swipe drives discard via `discardCanvas`; this - // path remains as a defensive secondary and is asserted directly here. - let controller = LauncherOverlayController() - var discarded = 0 - controller.clipboardPinSteps = 3 // the deliberate-excursion threshold reused for the discard - controller.onFire = { _, _ in } - controller.onCommitCanvas = { XCTFail("a discard excursion must not commit") } - controller.onDiscardCanvas = { discarded += 1 } - - let ai = aiItem("Translate") - let band = ContextBand(id: AIBand.bandID, name: "AI", - color: AIBand.color, items: [ai]) - controller.show(bands: [band], startBand: 0, startColumn: 0, dwell: 0.01) - controller.model.setArmed() - controller.end() // open the canvas - - // A small jitter under the threshold does NOT discard. - controller.stepHorizontal(1) - controller.stepHorizontal(-1) - XCTAssertEqual(discarded, 0, "sub-threshold horizontal jitter never discards") - XCTAssertTrue(controller.canvasActive) - - // A deliberate excursion past the threshold discards once. - controller.stepHorizontal(1) - controller.stepHorizontal(1) - controller.stepHorizontal(1) - XCTAssertEqual(discarded, 1, "one deliberate excursion discards exactly once") - XCTAssertFalse(controller.canvasActive, "discarding closes the canvas") - XCTAssertFalse(controller.isVisible, "and dismisses the overlay") - } - - // MARK: - Controller: hard cancel while canvas open discards generation - - func testCancelWhileCanvasOpenDiscards() { - let controller = LauncherOverlayController() - var discarded = 0 - controller.onFire = { _, _ in } - controller.onDiscardCanvas = { discarded += 1 } - - let ai = aiItem("Fix") - let band = ContextBand(id: AIBand.bandID, name: "AI", - color: AIBand.color, items: [ai]) - controller.show(bands: [band], startBand: 0, startColumn: 0, dwell: 0.01) - controller.model.setArmed() - controller.end() // open the canvas - - controller.cancel() // gesture abandoned mid-canvas - XCTAssertEqual(discarded, 1, "a hard cancel while the canvas is open discards generation") - XCTAssertFalse(controller.canvasActive) - XCTAssertFalse(controller.isVisible) - } - - // MARK: - PRESERVED: a non-AI armed lift still fires AND dismisses (order-out-before-fire) - - func testNonAIArmedLiftFiresAndDismissesAsBefore() { - let controller = LauncherOverlayController() - var fired: [LaunchItem] = [] - controller.onFire = { item, _ in - // The panel MUST already be ordered out before a non-AI fire (the regression guard). - XCTAssertFalse(controller.isVisible, "non-AI items order the panel out BEFORE firing") - fired.append(item) - } - controller.onCommitCanvas = { XCTFail("a non-AI item never enters the canvas") } - - let app = appItem("Safari") - let band = ContextBand(name: "Dev", color: ItemColor(red: 0, green: 0, blue: 1), items: [app]) - controller.show(bands: [band], startBand: 0, startColumn: 0, dwell: 0.01) - controller.model.setArmed() - - let result = controller.end() - XCTAssertTrue(result) - XCTAssertEqual(fired.map(\.title), ["Safari"], "the app fires exactly as before") - XCTAssertFalse(controller.canvasActive, "a non-AI item never opens the canvas") - XCTAssertFalse(controller.isVisible, "and the overlay dismisses on the lift, unchanged") - } - - // MARK: - PRESERVED: an unarmed lift still just dismisses - - func testUnarmedLiftDismissesWithoutFiring() { - let controller = LauncherOverlayController() - var fired = 0 - controller.onFire = { _, _ in fired += 1 } - let app = appItem("Mail") - let band = ContextBand(name: "Dev", color: ItemColor(red: 0, green: 0, blue: 1), items: [app]) - controller.show(bands: [band], startBand: 0, startColumn: 0, dwell: 0.5) - // No setArmed() — the dwell hasn't armed. - let result = controller.end() - XCTAssertFalse(result, "an unarmed lift fires nothing") - XCTAssertEqual(fired, 0) - XCTAssertFalse(controller.isVisible, "and the overlay dismisses") - XCTAssertFalse(controller.canvasActive) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/LauncherModelTests.swift b/Tests/ThreeFingerSwitcherTests/LauncherModelTests.swift index 35c664a..b5d34da 100644 --- a/Tests/ThreeFingerSwitcherTests/LauncherModelTests.swift +++ b/Tests/ThreeFingerSwitcherTests/LauncherModelTests.swift @@ -193,41 +193,4 @@ final class LauncherModelTests: XCTestCase { LauncherGridLayout.windowHeight(itemCount: 3), "default matches the no-list case") } - // MARK: - Files band layout = the Clipboard container's EXACT dimensions (refinement 3) - - func testFilesBandContainerEqualsClipboardDimensionsExactly() { - // The Files navigator's container is a FIXED size equal to the Clipboard band's — so it never - // resizes or moves when crossing in or changing depth (the list scrolls inside instead). - XCTAssertEqual(FilesBandLayout.containerWidth, ClipboardBandLayout.containerWidth, - "the Files container width is the Clipboard width, exactly") - XCTAssertEqual(FilesBandLayout.containerHeight, ClipboardBandLayout.containerHeight, - "the Files container height is the Clipboard height, exactly") - } - - func testFilesBandInteriorPanesSumToTheFixedContainerWidth() { - // rail + current list + preview + the two dividers + the outer padding == the fixed container width, - // so the three panes lay out WITHIN the fixed width with no overflow and no dead space. - let sum = FilesBandLayout.ancestorRailWidth - + FilesBandLayout.currentColumnWidth - + FilesBandLayout.previewWidth - + 2 * FilesBandLayout.dividerWidth - + 2 * FilesBandLayout.padding - XCTAssertEqual(sum, FilesBandLayout.containerWidth, accuracy: 0.01, - "the three-pane split fills the fixed container exactly") - XCTAssertGreaterThan(FilesBandLayout.previewWidth, FilesBandLayout.currentColumnWidth, - "the preview fills the roomy remainder — wider than the current list") - } - - func testFilesBandRowAreaIsContainerMinusChrome() { - // The scrollable row area is the fixed container height minus the breadcrumb bar and the outer - // padding — and a denser row fits MORE rows in that same fixed area (the container never grows for - // density; only how many rows show before it scrolls changes). - XCTAssertEqual(FilesBandLayout.rowAreaHeight, - FilesBandLayout.containerHeight - - FilesBandLayout.breadcrumbBarHeight - 2 * FilesBandLayout.padding, - accuracy: 0.01) - XCTAssertGreaterThan(FilesBandLayout.visibleRowCount(for: .compact), - FilesBandLayout.visibleRowCount(for: .spacious), - "a tighter row packs more rows into the same fixed row area") - } } diff --git a/Tests/ThreeFingerSwitcherTests/LocalImageGenerationTests.swift b/Tests/ThreeFingerSwitcherTests/LocalImageGenerationTests.swift deleted file mode 100644 index f1d57fa..0000000 --- a/Tests/ThreeFingerSwitcherTests/LocalImageGenerationTests.swift +++ /dev/null @@ -1,312 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// `ai-local-image-generation` — the Core (MLX-free) substrate of the local image backend: the image -/// `ModelDescriptor` variant table + `imageModelID` selection (§1), the pure `ImageResidencyClass` -/// classifier (§2), `MediaRequest` validation + the seed-capability gate (§3.1), the `StubImageRuntime` -/// (§3.2–3.3), and the cost-disclosure + busy-painting state values (§5.1–5.3 pure parts). The -/// `MFluxImageRuntime` MLX conformer (§4) is native-linked — `xcodebuild` compile-verify only. -final class LocalImageGenerationTests: XCTestCase { - - // MARK: - 1.1 Descriptor variant table (bytes / role / lane / provider / capabilities) - - func testQ4AndFP16DescriptorsHonestShape() { - let q4 = ImageModelCatalog.q4Descriptor - XCTAssertEqual(q4.role, .image) - XCTAssertEqual(q4.lane, .gpu) - XCTAssertEqual(q4.provider, .onDevice) - XCTAssertEqual(q4.residencyBytes, 7 * 1024 * 1024 * 1024) - // Klein 4B is t2i + conditioning-mode i2i, but NOT inpaint (no mask field on the seam) — see - // ImageModelCatalog's type doc. - XCTAssertEqual(ImageModelCatalog.tags(for: q4.id), [.image, .img2img]) - - let fp16 = ImageModelCatalog.fp16Descriptor - XCTAssertEqual(fp16.role, .image) - XCTAssertEqual(fp16.lane, .gpu) - XCTAssertEqual(fp16.provider, .onDevice) - XCTAssertEqual(fp16.residencyBytes, 24 * 1024 * 1024 * 1024) - XCTAssertEqual(ImageModelCatalog.tags(for: fp16.id), [.image, .img2img]) - } - - func testQ4ResidesBelowFP16() { - XCTAssertLessThan(ImageModelCatalog.q4ResidencyBytes, ImageModelCatalog.fp16ResidencyBytes) - } - - // MARK: - 1.2 imageModelID selection (default Q4; FP16 by id; unknown rejected) - - func testDefaultSelectionIsQ4() { - XCTAssertEqual(ImageModelCatalog.selected(imageModelID: nil)?.id, ImageModelCatalog.q4ID) - XCTAssertEqual(ImageModelCatalog.selected(imageModelID: "")?.id, ImageModelCatalog.q4ID) - XCTAssertEqual(ImageModelCatalog.defaultID, ImageModelCatalog.q4ID) - } - - func testFP16SelectedByID() { - XCTAssertEqual(ImageModelCatalog.selected(imageModelID: ImageModelCatalog.fp16ID)?.id, - ImageModelCatalog.fp16ID) - } - - func testUnknownIDRejectedNotDefaulted() { - // An unknown id resolves to nil (rejected) — NOT silently coerced to the Q4 default. - XCTAssertNil(ImageModelCatalog.selected(imageModelID: "no-such-image-model")) - XCTAssertFalse(ImageModelCatalog.isKnown("no-such-image-model")) - XCTAssertTrue(ImageModelCatalog.isKnown(ImageModelCatalog.q4ID)) - XCTAssertTrue(ImageModelCatalog.isKnown(ImageModelCatalog.fp16ID)) - } - - // MARK: - 1.3 Descriptors registered with the fleet ModelRegistry - - func testDescriptorsRegisteredInFleet() async throws { - let registry = StubModelRegistry(members: ImageModelCatalog.descriptors) - let ids = registry.descriptors().map(\.id) - XCTAssertTrue(ids.contains(ImageModelCatalog.q4ID)) - XCTAssertTrue(ids.contains(ImageModelCatalog.fp16ID)) - // And the SAME ids appear in the production roster (this slice does not fork the roster ids). - let rosterIDs = FleetRoster.standard.descriptors().map(\.id) - XCTAssertTrue(rosterIDs.contains(ImageModelCatalog.q4ID)) - XCTAssertTrue(rosterIDs.contains(ImageModelCatalog.fp16ID)) - } - - // MARK: - 2.1/2.2 Residency classification (pure, injected resident set + ceiling) - - private func chat() -> ModelDescriptor { FleetRoster.standard.descriptor(id: "gemma-4-31b")! } - private func ternary() -> ModelDescriptor { FleetRoster.standard.descriptor(id: "ternary-cpu-chat")! } - - func testQ4CoResidesWithChatAndTernary() { - let classifier = ImageResidencyClassifier() - let result = classifier.classify(image: ImageModelCatalog.q4Descriptor, - resident: [chat(), ternary()], - ceilingBytes: FleetRoster.unifiedBudget48GB) - XCTAssertEqual(result, .coResident) // ~17 + ~0.5 + ~7 + ~4 KV ≈ 28.5 GB < 48 GB - } - - func testFP16EvictsChat() { - let classifier = ImageResidencyClassifier() - let result = classifier.classify(image: ImageModelCatalog.fp16Descriptor, - resident: [chat(), ternary()], - ceilingBytes: FleetRoster.unifiedBudget48GB) - // FP16 (24 GB > 16 GB threshold) is a HEAVY GPU gen → GPU-lane exclusivity evicts chat even though - // ~17 + ~0.5 + ~24 + ~4 KV ≈ 45.5 GB technically fits under 48 (it can't stream chat + paint at once). - XCTAssertEqual(result, .evictsChat) - } - - func testCeilingBoundaryIsDeterministic() { - // Construct a tight ceiling so the boundary is exact: chat-only resident, image fits iff - // chat + image + KV <= ceiling. Use a 0 KV reserve to make the boundary clean. - let classifier = ImageResidencyClassifier(kvReserveBytes: 0) - let c = chat() // 17 GB - let img = ImageModelCatalog.q4Descriptor // 7 GB - let exact = c.residencyBytes + img.residencyBytes // 24 GB - // Exactly at the ceiling → fits → co-resident. - XCTAssertEqual(classifier.classify(image: img, resident: [c], ceilingBytes: exact), .coResident) - // One byte under → does not fit → evicts chat (chat is the resident GPU victim). - XCTAssertEqual(classifier.classify(image: img, resident: [c], ceilingBytes: exact - 1), .evictsChat) - } - - func testNoChatResidentNeverEvictsChat() { - // With no chat resident, even a too-big image can't be classified as evicting chat. - let classifier = ImageResidencyClassifier(kvReserveBytes: 0) - let result = classifier.classify(image: ImageModelCatalog.fp16Descriptor, - resident: [ternary()], - ceilingBytes: 1) // ceiling far below the image - XCTAssertEqual(result, .coResident) // nothing chat-like to evict - } - - // MARK: - 2.3 The same class value drives disclosure AND the busy-painting state - - func testSameClassDrivesDisclosureAndBusyState() { - let evicts = ImageResidencyClass.evictsChat - let disclosure = ImageCostDisclosure.make(descriptor: ImageModelCatalog.fp16Descriptor, - classification: evicts) - let busy = ImageBusyPaintingState(evicts) - XCTAssertTrue(disclosure.evictsChat) - XCTAssertEqual(busy, .busyPainting) - XCTAssertEqual(busy.residencyNote, .busyPainting) - - let coResident = ImageResidencyClass.coResident - let d2 = ImageCostDisclosure.make(descriptor: ImageModelCatalog.q4Descriptor, classification: coResident) - XCTAssertFalse(d2.evictsChat) - XCTAssertEqual(ImageBusyPaintingState(coResident), .coResident) - XCTAssertEqual(ImageBusyPaintingState(coResident).residencyNote, .coResident) - } - - // MARK: - 3.1 Request validation + seed capability gate - - private func pngSeed() -> Data { Data(MediaSeedValidation.pngMagic + [0x00, 0x01]) } - - func testValidTextToImagePasses() { - let req = MediaRequest(prompt: "a cat", kind: .image, - parameters: MediaParameters(size: .square1024, steps: 28)) - XCTAssertNil(ImageRequestValidator.validate(req, descriptor: ImageModelCatalog.q4Descriptor)) - } - - func testValidImg2ImgWithSeedCapableDescriptorPasses() { - let req = MediaRequest(prompt: "repaint", seed: pngSeed(), kind: .image, - parameters: MediaParameters(size: MediaSize(width: 512, height: 512), steps: 20)) - XCTAssertNil(ImageRequestValidator.validate(req, descriptor: ImageModelCatalog.q4Descriptor)) - } - - func testSeedAgainstNonSeedCapableDescriptorIsError() { - // A t2i-only descriptor (no img2img tag) — seed must be rejected, not silently dropped. - let t2iOnly = ModelDescriptor( - id: "image-t2i-only", displayName: "t2i only", sizeBytes: 1, - integritySHA: "x", downloadURL: URL(string: "https://example.com")!, - capabilities: [.text], quantization: .qat4bit, - role: .image, lane: .gpu, provider: .onDevice, residencyBytes: 1) - // It is NOT in the catalog's tags → not seed-capable. - XCTAssertFalse(ImageModelCatalog.isSeedCapable(t2iOnly.id)) - let req = MediaRequest(prompt: "x", seed: pngSeed(), kind: .image) - let err = ImageRequestValidator.validate(req, descriptor: t2iOnly) - XCTAssertNotNil(err) - if case .generationFailed = err {} else { XCTFail("expected a generationFailed mismatch error, got \(String(describing: err))") } - } - - func testOutOfRangeParamsRejected() { - let big = MediaRequest(prompt: "x", kind: .image, - parameters: MediaParameters(size: MediaSize(width: 9000, height: 512), steps: 28)) - XCTAssertNotNil(ImageRequestValidator.validate(big, descriptor: ImageModelCatalog.q4Descriptor)) - - let tooManySteps = MediaRequest(prompt: "x", kind: .image, - parameters: MediaParameters(size: .square1024, steps: 9999)) - XCTAssertNotNil(ImageRequestValidator.validate(tooManySteps, descriptor: ImageModelCatalog.q4Descriptor)) - - var badGuidance = MediaParameters(size: .square1024, steps: 28) - badGuidance.guidance = 999 - XCTAssertNotNil(ImageRequestValidator.validate(MediaRequest(prompt: "x", kind: .image, parameters: badGuidance), - descriptor: ImageModelCatalog.q4Descriptor)) - } - - func testNonImageKindRejected() { - let vid = MediaRequest(prompt: "x", kind: .video) - XCTAssertNotNil(ImageRequestValidator.validate(vid, descriptor: ImageModelCatalog.q4Descriptor)) - } - - func testUnknownImageModelIDValidationRejected() { - let req = MediaRequest(prompt: "x", kind: .image) - XCTAssertNotNil(ImageRequestValidator.validate(req, imageModelID: "no-such")) - XCTAssertNil(ImageRequestValidator.validate(req, imageModelID: ImageModelCatalog.fp16ID)) - XCTAssertNil(ImageRequestValidator.validate(req, imageModelID: nil)) // default Q4 - } - - // MARK: - 3.2 The image stub: ordered steps → finished asset with a readable PNG; cancellation - - func testStubEmitsOrderedStepsThenFinishedReadablePNG() async throws { - let runtime = StubImageRuntime() - let req = MediaRequest(prompt: "a cat", kind: .image, - parameters: MediaParameters(size: MediaSize(width: 640, height: 480), steps: 4)) - var indices: [Int] = [] - var finished: MediaAsset? - for try await progress in runtime.generate(req) { - switch progress { - case let .step(index, total, preview): - indices.append(index) - XCTAssertEqual(total, 4) - XCTAssertNotNil(preview) - case let .finished(asset): - finished = asset - } - } - XCTAssertEqual(indices, [0, 1, 2, 3]) // ordered, ascending - let asset = try XCTUnwrap(finished) - XCTAssertEqual(asset.kind, .image) - XCTAssertEqual(asset.width, 640) // dimensions match the request - XCTAssertEqual(asset.height, 480) - // The URL points at a real, readable PNG. - let data = try Data(contentsOf: asset.url) - XCTAssertTrue(MediaSeedValidation.isDecodablePNG(data)) - try? FileManager.default.removeItem(at: asset.url) - } - - func testStubCancellationEmitsNoFinished() async throws { - // A per-step delay makes the cancellation window deterministic (we cancel before the steps drain). - let runtime = StubImageRuntime(maxSteps: 100, perStepDelayNanos: 2_000_000) // ~2 ms/step - let req = MediaRequest(prompt: "x", kind: .image, - parameters: MediaParameters(size: .square1024, steps: 100)) - let task = Task { () -> Bool in - var sawFinished = false - for try await progress in runtime.generate(req) { - if case .finished = progress { sawFinished = true } - } - return sawFinished - } - // Give it a moment to start, then cancel. - try await Task.sleep(nanoseconds: 5_000_000) - task.cancel() - let sawFinished = try await task.value - XCTAssertFalse(sawFinished) // cancellation → stream ends WITHOUT a .finished - } - - func testStubSeedAgainstNonSeedDescriptorThrows() async { - let t2iOnly = ModelDescriptor( - id: "image-t2i-only", displayName: "t2i only", sizeBytes: 1, - integritySHA: "x", downloadURL: URL(string: "https://example.com")!, - capabilities: [.text], quantization: .qat4bit, - role: .image, lane: .gpu, provider: .onDevice, residencyBytes: 1) - let runtime = StubImageRuntime(descriptor: t2iOnly) - let req = MediaRequest(prompt: "x", seed: pngSeed(), kind: .image) - do { - for try await _ in runtime.generate(req) {} - XCTFail("expected the stub to throw a MediaError for a seed vs non-seed descriptor") - } catch let e as MediaError { - if case .generationFailed = e {} else { XCTFail("unexpected MediaError \(e)") } - } catch { - XCTFail("unexpected error \(error)") - } - } - - // MARK: - 3.3 End-to-end: route → progress → asset over the stub (incl. the seed/img2img branch) - - func testEndToEndImg2ImgBranchProducesImageAsset() async throws { - let runtime = StubImageRuntime() // Q4, seed-capable - let req = MediaRequest(prompt: "repaint", seed: pngSeed(), kind: .image, - parameters: MediaParameters(size: MediaSize(width: 256, height: 256), steps: 3)) - var finished: MediaAsset? - for try await progress in runtime.generate(req) { - if case let .finished(asset) = progress { finished = asset } - } - let asset = try XCTUnwrap(finished) - XCTAssertEqual(asset.kind, .image) - XCTAssertEqual(asset.width, 256) - XCTAssertEqual(asset.height, 256) - XCTAssertEqual(runtime.receivedRequests.count, 1) - XCTAssertNotNil(runtime.receivedRequests.first?.seed) // the seed threaded through - try? FileManager.default.removeItem(at: asset.url) - } - - // MARK: - 5.1 Cost disclosure tracks the chosen quant - - func testDisclosureTracksChosenQuant() { - let q4 = ImageCostDisclosure.make(descriptor: ImageModelCatalog.q4Descriptor, classification: .coResident) - XCTAssertFalse(q4.evictsChat) - XCTAssertTrue(q4.ram.contains("7")) - XCTAssertTrue(q4.ram.lowercased().contains("co-resides")) - XCTAssertFalse(q4.heat.isEmpty) - XCTAssertFalse(q4.latency.isEmpty) - - let fp16 = ImageCostDisclosure.make(descriptor: ImageModelCatalog.fp16Descriptor, classification: .evictsChat) - XCTAssertTrue(fp16.evictsChat) - XCTAssertTrue(fp16.ram.contains("24")) - XCTAssertTrue(fp16.ram.lowercased().contains("paus")) // "PAUSES the chat model" - } - - // MARK: - 5.3 Gating — the contributor offers the image tool only under the flags - - func testImageToolGatedBehindMasterAndMediaFlags() { - // OFF: master off → no media tools, so no image capability offered. - let off = MediaToolAvailability(isFullPotentialEnabled: { false }, isMediaGenEnabled: { true }) - XCTAssertFalse(off.mediaActive) - - // media flag off under master on → still off. - let mediaOff = MediaToolAvailability(isFullPotentialEnabled: { true }, isMediaGenEnabled: { false }) - XCTAssertFalse(mediaOff.mediaActive) - - // both on → media active (the contributor will then offer generate_image if an image runtime - // advertises .image — exercised in MediaRuntimeTests; here we assert the gate floor). - let on = MediaToolAvailability(isFullPotentialEnabled: { true }, isMediaGenEnabled: { true }) - XCTAssertTrue(on.mediaActive) - } - - func testDiffusionRoleRoutesToGPULane() { - // The gen role→lane policy: mediaDiffusion → .gpu (consumed from ai-compute-tiers). - XCTAssertEqual(DefaultLaneRouting().lane(for: .mediaDiffusion), .gpu) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/MediaRuntimeTests.swift b/Tests/ThreeFingerSwitcherTests/MediaRuntimeTests.swift deleted file mode 100644 index 00ea03b..0000000 --- a/Tests/ThreeFingerSwitcherTests/MediaRuntimeTests.swift +++ /dev/null @@ -1,447 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// `ai-media-runtime` — the seam + value types, the two tools + contributor, the `MediaGenSink`, the seed -/// path, the gallery + `.fileEntry`, the canvas state/resolve model, the parked feed, residency, and -/// `MediaError`. All MLX-free Core, driven against `StubMediaRuntime` (no weights). -final class MediaRuntimeTests: XCTestCase { - - // MARK: - Helpers - - private func pngBytes() -> Data { - Data(MediaSeedValidation.pngMagic + [0x00, 0x01, 0x02]) - } - - private func tmpGalleryRoot() -> URL { - FileManager.default.temporaryDirectory - .appendingPathComponent("MediaGalleryTests-\(UUID().uuidString)", isDirectory: true) - } - - private func imageAsset(in root: URL) -> MediaAsset { - MediaAsset(url: root.appendingPathComponent("x.png"), kind: .image, width: 512, height: 512) - } - - private func route(_ tool: String, argsJSON: String = "{}", userText: String = "draw a cat") -> RoutedCall { - let desc = (tool == MediaTool.generateImage) ? MediaToolContributor.imageDescriptor - : ToolDescriptor(name: tool, summary: "", argsSchema: StructuredSchema(name: tool, json: "{}"), - writePolicy: .dangerous) - return RoutedCall(descriptor: desc, - route: ToolRoute(tool: tool, argumentsJSON: argsJSON), - userText: userText, - source: TaskSource()) - } - - /// A scripted gate (mirrors the routing tests' pattern). - struct ScriptedGate: ApprovalGate { - let decision: ApprovalDecision - func awaitDecision(for review: TaskReview) async -> ApprovalDecision { decision } - } - - /// A recording observer. - final class SpyObserver: MediaJobObserving, @unchecked Sendable { - var steps = 0 - var finished: MediaAsset? - var failedHeadline: String? - var cancelled = false - var residency: MediaResidencyNote? - func didDecideResidency(_ note: MediaResidencyNote) { residency = note } - func didStep(index: Int, total: Int, preview: Data?) { steps += 1 } - func didFinish(_ asset: MediaAsset) { finished = asset } - func didFail(headline: String) { failedHeadline = headline } - func didCancel() { cancelled = true } - } - - // MARK: - 1.1 Value types: Codable round-trip; seed shapes; durationMs video-only - - func testMediaAssetCodableRoundTrip() throws { - let asset = MediaAsset(url: URL(fileURLWithPath: "/tmp/a.mp4"), kind: .video, - width: 1280, height: 720, durationMs: 4000) - let data = try JSONEncoder().encode(asset) - let back = try JSONDecoder().decode(MediaAsset.self, from: data) - XCTAssertEqual(asset, back) - XCTAssertEqual(back.durationMs, 4000) - } - - func testMediaParametersDurationVideoOnlyByConvention() { - let img = MediaArgs(prompt: "p", durationMs: 5000).request(kind: .image, seed: nil) - XCTAssertNil(img.parameters.durationMs, "image requests carry no duration") - let vid = MediaArgs(prompt: "p", durationMs: 5000).request(kind: .video, seed: nil) - XCTAssertEqual(vid.parameters.durationMs, 5000) - } - - func testRequestSeedPresentAndAbsentShapes() { - let none = MediaRequest(prompt: "p", kind: .image) - XCTAssertNil(none.seed) - let seeded = MediaRequest(prompt: "p", seed: pngBytes(), kind: .image) - XCTAssertNotNil(seeded.seed) - } - - // MARK: - 1.3 StubMediaRuntime scripts drive a deterministic stream - - func testStubSuccessWithPreviews() async throws { - let asset = imageAsset(in: tmpGalleryRoot()) - let stub = StubMediaRuntime(capabilities: [.image], - script: .successWithPreviews(count: 3, total: 3, preview: pngBytes(), asset: asset)) - var steps = 0 - var finished: MediaAsset? - for try await p in stub.generate(MediaRequest(prompt: "p", kind: .image)) { - switch p { - case .step: steps += 1 - case let .finished(a): finished = a - } - } - XCTAssertEqual(steps, 3) - XCTAssertEqual(finished, asset) - } - - func testStubFailMidFlightThrows() async { - let stub = StubMediaRuntime(script: .failMidFlight(steps: 2, headline: "boom")) - do { - for try await _ in stub.generate(MediaRequest(prompt: "p", kind: .image)) {} - XCTFail("expected throw") - } catch { - guard case let MediaError.generationFailed(h) = error else { return XCTFail("wrong error \(error)") } - XCTAssertEqual(h, "boom") - } - } - - // MARK: - 2.1 Descriptors: names + tiers + argsSchema validates - - func testImageDescriptorIsConfirmTier() { - XCTAssertEqual(MediaToolContributor.imageDescriptor.name, "generate_image") - XCTAssertEqual(MediaToolContributor.imageDescriptor.writePolicy, .confirm) - } - - func testImageArgsSchemaIsValidJSON() throws { - let data = MediaToolContributor.imageArgsSchemaJSON.data(using: .utf8)! - let obj = try JSONSerialization.jsonObject(with: data) as? [String: Any] - XCTAssertNotNil(obj?["properties"]) - } - - func testCloudVideoDescriptorIsDangerous() { - let contributor = makeContributor(videoCloud: true, cloudOn: true, budget: 5, hasVideoProvider: true) - let video = contributor.videoDescriptor - XCTAssertEqual(video.name, "generate_video") - XCTAssertEqual(video.writePolicy, .dangerous) - } - - // MARK: - 2.2 Availability gating (D11) - - func testNothingContributedWhenMediaDisabled() { - let contributor = makeContributor(mediaOn: false, fullPotentialOn: true) - XCTAssertTrue(contributor.descriptors().isEmpty) - } - - func testNothingContributedWhenFullPotentialOff() { - let contributor = makeContributor(mediaOn: true, fullPotentialOn: false) - XCTAssertTrue(contributor.descriptors().isEmpty) - } - - func testImageOfferedOnlyWhenImageRuntimeAdvertisesImage() { - let withImage = makeContributor(imageCaps: [.image]) - XCTAssertTrue(withImage.descriptors().contains { $0.name == "generate_image" }) - let withoutImage = makeContributor(imageCaps: []) - XCTAssertFalse(withoutImage.descriptors().contains { $0.name == "generate_image" }) - } - - func testVideoOmittedWhenNoProviderOrCloudOffOrBudgetOut() { - // No provider → omitted. - XCTAssertFalse(makeContributor(videoCaps: [.video], hasVideoProvider: false) - .descriptors().contains { $0.name == "generate_video" }) - // Cloud provider, cloud escalation OFF → omitted. - XCTAssertFalse(makeContributor(videoCaps: [.video], videoCloud: true, cloudOn: false, hasVideoProvider: true) - .descriptors().contains { $0.name == "generate_video" }) - // Cloud provider on, budget 0 → omitted. - XCTAssertFalse(makeContributor(videoCaps: [.video], videoCloud: true, cloudOn: true, budget: 0, hasVideoProvider: true) - .descriptors().contains { $0.name == "generate_video" }) - // Cloud provider on, budget left → present. - XCTAssertTrue(makeContributor(videoCaps: [.video], videoCloud: true, cloudOn: true, budget: 3, hasVideoProvider: true) - .descriptors().contains { $0.name == "generate_video" }) - } - - // MARK: - 3.1 Approval precedes compute; skip / approve - - func testApproveDrivesGenerationAndReturnsDoneWithGalleryPath() async throws { - let root = tmpGalleryRoot() - let gallery = MediaGallery(root: root) - let stub = StubMediaRuntime(capabilities: [.image], - script: .successWithPreviews(count: 2, total: 2, preview: pngBytes(), - asset: imageAsset(in: root))) - let observer = SpyObserver() - let sink = MediaGenSink(imageRuntime: stub, videoRuntime: nil, gallery: gallery, - budget: PerDayVideoBudget(cap: { 0 }), observer: observer) - let result = await sink.run(route(MediaTool.generateImage), gate: ScriptedGate(decision: .approve)) - XCTAssertEqual(result.status, .done) - XCTAssertNotNil(observer.finished) - XCTAssertEqual(observer.steps, 2) - } - - func testSkipAppliesNothing() async { - let root = tmpGalleryRoot() - let stub = StubMediaRuntime(capabilities: [.image], script: .success(imageAsset(in: root))) - let observer = SpyObserver() - let sink = MediaGenSink(imageRuntime: stub, videoRuntime: nil, gallery: MediaGallery(root: root), - budget: PerDayVideoBudget(cap: { 0 }), observer: observer) - let result = await sink.run(route(MediaTool.generateImage), gate: ScriptedGate(decision: .skip)) - guard case .declined = result.status else { return XCTFail("expected declined") } - XCTAssertNil(observer.finished, "no compute on skip") - XCTAssertEqual(stub.receivedRequests.count, 0, "the runtime is never called on skip") - } - - func testCancelIsNotAFailure() async { - let root = tmpGalleryRoot() - let stub = StubMediaRuntime(capabilities: [.image], script: .success(imageAsset(in: root))) - let observer = SpyObserver() - let sink = MediaGenSink(imageRuntime: stub, videoRuntime: nil, gallery: MediaGallery(root: root), - budget: PerDayVideoBudget(cap: { 0 }), observer: observer) - let result = await sink.run(route(MediaTool.generateImage), gate: ScriptedGate(decision: .cancel)) - guard case let .declined(reason) = result.status else { return XCTFail("expected declined") } - XCTAssertEqual(reason, MediaGenSink.cancelledReason) - XCTAssertTrue(observer.cancelled) - XCTAssertNil(observer.failedHeadline, "cancel is never a failure") - } - - // MARK: - 3.3 Budget cap enforced before spend - - func testExhaustedBudgetResolvesBeforeAnyRuntimeCall() async { - let stub = StubMediaRuntime(capabilities: [.video], script: .success(imageAsset(in: tmpGalleryRoot()))) - let videoDesc = ToolDescriptor(name: MediaTool.generateVideo, summary: "", - argsSchema: StructuredSchema(name: "v", json: "{}"), writePolicy: .dangerous) - let call = RoutedCall(descriptor: videoDesc, route: ToolRoute(tool: MediaTool.generateVideo), - userText: "make a clip", source: TaskSource()) - let sink = MediaGenSink(imageRuntime: nil, videoRuntime: stub, gallery: MediaGallery(root: tmpGalleryRoot()), - budget: PerDayVideoBudget(cap: { 0 })) // 0 → exhausted - let result = await sink.run(call, gate: ScriptedGate(decision: .approve)) - guard case let .failed(headline) = result.status else { return XCTFail("expected failed") } - XCTAssertEqual(headline, MediaError.cloudBudgetExhausted.errorDescription) - XCTAssertEqual(stub.receivedRequests.count, 0, "budget-out → no runtime call, no spend") - } - - // MARK: - 3.4 Residency: eviction → busy painting - - func testResidencyEvictionSurfacesBusyPainting() async throws { - let root = tmpGalleryRoot() - let stub = StubMediaRuntime(capabilities: [.image], script: .success(imageAsset(in: root))) - let registry = EvictingStubRegistry() // chat resident before, evicted after ensureResident - let residency = MediaResidencyCoordinator(registry: registry) - let observer = SpyObserver() - let sink = MediaGenSink(imageRuntime: stub, videoRuntime: nil, gallery: MediaGallery(root: root), - budget: PerDayVideoBudget(cap: { 0 }), residency: residency, - observer: observer, imageModelID: "image-model") - _ = await sink.run(route(MediaTool.generateImage), gate: ScriptedGate(decision: .approve)) - XCTAssertEqual(observer.residency, .busyPainting) - } - - // MARK: - 4 Seed path - - func testSeedWiredIntoRequest() async throws { - let root = tmpGalleryRoot() - let stub = StubMediaRuntime(capabilities: [.image], script: .success(imageAsset(in: root))) - let sink = MediaGenSink(imageRuntime: stub, videoRuntime: nil, seed: CapturedSeed(png: pngBytes()), - gallery: MediaGallery(root: root), budget: PerDayVideoBudget(cap: { 0 })) - _ = await sink.run(route(MediaTool.generateImage, argsJSON: "{\"seedImage\":\"clipboardImage\"}"), - gate: ScriptedGate(decision: .approve)) - XCTAssertEqual(stub.receivedRequests.first?.seed, pngBytes()) - } - - func testMissingRequiredSeedFailsCleanlyWithNoCompute() async { - let stub = StubMediaRuntime(capabilities: [.image], script: .success(imageAsset(in: tmpGalleryRoot()))) - let sink = MediaGenSink(imageRuntime: stub, videoRuntime: nil, seed: NoSeed(), - gallery: MediaGallery(root: tmpGalleryRoot()), budget: PerDayVideoBudget(cap: { 0 })) - let result = await sink.run(route(MediaTool.generateImage, argsJSON: "{\"seedImage\":\"screenRegion\"}"), - gate: ScriptedGate(decision: .approve)) - guard case let .failed(h) = result.status else { return XCTFail("expected failed") } - XCTAssertEqual(h, MediaError.seedRequired.errorDescription) - XCTAssertEqual(stub.receivedRequests.count, 0) - } - - func testInvalidSeedFails() async { - let stub = StubMediaRuntime(capabilities: [.image], script: .success(imageAsset(in: tmpGalleryRoot()))) - let badSeed = CapturedSeed(png: Data([0x00, 0x01, 0x02, 0x03])) // not a PNG signature - let sink = MediaGenSink(imageRuntime: stub, videoRuntime: nil, seed: badSeed, - gallery: MediaGallery(root: tmpGalleryRoot()), budget: PerDayVideoBudget(cap: { 0 })) - let result = await sink.run(route(MediaTool.generateImage, argsJSON: "{\"seedImage\":\"clipboardImage\"}"), - gate: ScriptedGate(decision: .approve)) - guard case let .failed(h) = result.status else { return XCTFail("expected failed") } - XCTAssertEqual(h, MediaError.seedInvalid.errorDescription) - } - - // MARK: - 5 Gallery output + .fileEntry - - func testGalleryWritePersistsAndSurvivesRelaunchRead() throws { - let root = tmpGalleryRoot() - let gallery = MediaGallery(root: root) - let asset = try gallery.write(pngBytes(), kind: .image, width: 256, height: 256, durationMs: nil) - XCTAssertTrue(FileManager.default.fileExists(atPath: asset.url.path)) - // A "relaunch" read: a fresh gallery instance over the same root still finds the file. - let reread = try Data(contentsOf: asset.url) - XCTAssertEqual(reread, pngBytes()) - } - - func testAssetMapsToFileEntryWithPathStableIdentity() { - let url = tmpGalleryRoot().appendingPathComponent("a.png") - let asset = MediaAsset(url: url, kind: .image, width: 1, height: 1) - let e1 = asset.fileEntry() - let e2 = asset.fileEntry() - XCTAssertEqual(e1.id, e2.id, "path-stable identity — no strobe") - XCTAssertEqual(e1.kind, .image) - XCTAssertFalse(e1.isDirectory) - } - - // MARK: - 6.1 Canvas state model - - func testJobStateAdvancesAndTerminates() { - var s = MediaJobState.idle - s.advance(.step(index: 0, total: 4, preview: pngBytes())) - guard case .generating = s else { return XCTFail() } - XCTAssertFalse(s.isTerminal) - let asset = imageAsset(in: tmpGalleryRoot()) - s.advance(.finished(asset)) - XCTAssertEqual(s.asset, asset) - XCTAssertTrue(s.isTerminal) - } - - func testJobStateFailVsCancel() { - var failed = MediaJobState.generating(index: 0, total: 1, preview: nil) - failed.fail(with: MediaError.generationFailed(headline: "nope")) - guard case let .failed(h) = failed else { return XCTFail() } - XCTAssertEqual(h, "nope") - - var cancelled = MediaJobState.generating(index: 0, total: 1, preview: nil) - cancelled.fail(with: CancellationError()) - XCTAssertEqual(cancelled, .cancelled) - } - - // MARK: - 6.3 Canvas resolve compass - - func testResolveDownAtTopExtracts() { - let s = MediaJobState.finished(imageAsset(in: tmpGalleryRoot())) - XCTAssertEqual(MediaCanvasResolver.resolve(dx: 0, dy: 0.5, atTop: true, state: s), .extract) - } - - func testResolveDownNotAtTopDoesNotExtract() { - let s = MediaJobState.finished(imageAsset(in: tmpGalleryRoot())) - XCTAssertEqual(MediaCanvasResolver.resolve(dx: 0, dy: 0.5, atTop: false, state: s), .none) - } - - func testResolveRightDiscards() { - let s = MediaJobState.finished(imageAsset(in: tmpGalleryRoot())) - XCTAssertEqual(MediaCanvasResolver.resolve(dx: 0.5, dy: 0, atTop: true, state: s), .discard) - } - - func testSubThresholdScrollDoesNotResolve() { - let s = MediaJobState.finished(imageAsset(in: tmpGalleryRoot())) - XCTAssertEqual(MediaCanvasResolver.resolve(dx: 0.05, dy: 0.05, atTop: true, state: s), .none) - } - - func testNonTerminalNeverResolves() { - let s = MediaJobState.generating(index: 0, total: 1, preview: nil) - XCTAssertEqual(MediaCanvasResolver.resolve(dx: 0, dy: 0.9, atTop: true, state: s), .none) - } - - // MARK: - 7 Parked feed - - func testPaintingThenFinishedFeedsScheduler() { - let id = AgentSessionID() - let row = ParkedSession(id: id, title: "gen", state: .parked) - let scheduler = SerialParkScheduler(sessions: [row]) - let feed = MediaParkFeed(scheduler: scheduler) - feed.reportPainting(id, tool: MediaTool.generateImage) - feed.reportFinished(id, tool: MediaTool.generateImage, asset: imageAsset(in: tmpGalleryRoot())) - let after = scheduler.snapshot().first { $0.id == id } - XCTAssertEqual(after?.state, .idle, - "a finished gen idles with its unseen result — never a terminal state that removes it") - XCTAssertGreaterThanOrEqual(after?.badgeCount ?? 0, 1, "unseen count bumped") - } - - func testDangerousVideoParkedEscalates() { - let id = AgentSessionID() - let scheduler = SerialParkScheduler(sessions: [ParkedSession(id: id, title: "v", state: .parked)]) - let feed = MediaParkFeed(scheduler: scheduler) - feed.reportNeedsYou(id, reason: "Cloud video spends from today's budget.") - XCTAssertEqual(scheduler.snapshot().first { $0.id == id }?.state, .needsYou) - } - - // MARK: - 8 MediaError taxonomy + translator - - func testEveryMediaErrorCaseHasCleanNonEmptyDescription() { - let cases: [MediaError] = [ - .noCapableBackend(kind: .image), .noCapableBackend(kind: .video), - .seedRequired, .seedInvalid, .generationFailed(headline: "x"), - .outputWriteFailed(detail: "io"), .cloudBudgetExhausted, .cloudUnavailable - ] - for c in cases { - XCTAssertFalse((c.errorDescription ?? "").isEmpty, "\(c) has empty description") - } - } - - func testMediaErrorRoutesThroughTranslatorRawTextOnlyInDetails() { - let err = MediaError.outputWriteFailed(detail: "POSIX error 13: Permission denied") - let presented = AIError.message(for: err) - XCTAssertEqual(presented.headline, err.errorDescription) - XCTAssertFalse(presented.headline.contains("POSIX"), "raw text never in headline") - XCTAssertEqual(presented.details, "POSIX error 13: Permission denied") - } - - func testCancellationDistinctFromFailureInTranslator() { - let presented = AIError.message(for: CancellationError()) - XCTAssertEqual(presented.headline, RuntimeError.cancelled.errorDescription) - } - - // MARK: - 8.3 Audit - - func testOneAuditRecordPerTerminalOutcome() async { - let root = tmpGalleryRoot() - let stub = StubMediaRuntime(capabilities: [.image], script: .success(imageAsset(in: root))) - let audit = InMemoryAuditLog() - let id = AgentSessionID() - let sink = MediaGenSink(imageRuntime: stub, videoRuntime: nil, gallery: MediaGallery(root: root), - budget: PerDayVideoBudget(cap: { 0 }), audit: audit, sessionID: id) - _ = await sink.run(route(MediaTool.generateImage), gate: ScriptedGate(decision: .approve)) - let records = audit.recent(limit: 10) - XCTAssertEqual(records.count, 1) - XCTAssertEqual(records.first?.tool, MediaTool.generateImage) - XCTAssertEqual(records.first?.policy, .confirm) - XCTAssertEqual(records.first?.outcome, .done) - } - - // MARK: - Cloud-video dangerous tier is never lowered - - func testDangerousNeverLoweredByDescriptorResolver() { - let videoDesc = ToolDescriptor(name: MediaTool.generateVideo, summary: "", - argsSchema: StructuredSchema(name: "v", json: "{}"), writePolicy: .dangerous) - XCTAssertEqual(DescriptorWritePolicy().effectiveTier(for: videoDesc), .dangerous) - } - - // MARK: - Builders - - private func makeContributor(mediaOn: Bool = true, fullPotentialOn: Bool = true, - imageCaps: Set = [.image], videoCaps: Set = [.video], - videoCloud: Bool = true, cloudOn: Bool = false, budget: Int = 0, - hasVideoProvider: Bool = false) -> MediaToolContributor { - let availability = MediaToolAvailability( - isFullPotentialEnabled: { fullPotentialOn }, - isMediaGenEnabled: { mediaOn }, - isCloudEscalationEnabled: { cloudOn }, - hasVideoProvider: { hasVideoProvider }, - videoProviderIsCloud: { videoCloud }) - let img = StubMediaRuntime(capabilities: imageCaps, script: .success(imageAsset(in: tmpGalleryRoot()))) - let vid = StubMediaRuntime(capabilities: videoCaps, script: .success(imageAsset(in: tmpGalleryRoot()))) - let budgetObj = PerDayVideoBudget(cap: { budget }) - let sink = MediaGenSink(imageRuntime: img, videoRuntime: vid, gallery: MediaGallery(root: tmpGalleryRoot()), - budget: budgetObj) - return MediaToolContributor(availability: availability, imageRuntime: img, videoRuntime: vid, - budget: budgetObj, sink: sink) - } -} - -/// A stub registry where chat is resident BEFORE `ensureResident` and EVICTED after (models a heavy gen). -private final class EvictingStubRegistry: ModelRegistry, @unchecked Sendable { - private var residentChat = true - private let chat = ModelDescriptor(id: "chat", displayName: "Chat", sizeBytes: 1, integritySHA: "x", - downloadURL: URL(string: "https://example.com")!, capabilities: [.text], - quantization: .qat4bit, role: .chat) - func descriptors() -> [ModelDescriptor] { [chat] } - func resident() -> [ModelDescriptor] { residentChat ? [chat] : [] } - func ensureResident(_ id: String) async throws { residentChat = false } // the gen evicts chat -} diff --git a/Tests/ThreeFingerSwitcherTests/MemoryTests.swift b/Tests/ThreeFingerSwitcherTests/MemoryTests.swift deleted file mode 100644 index 0919821..0000000 --- a/Tests/ThreeFingerSwitcherTests/MemoryTests.swift +++ /dev/null @@ -1,376 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for the `ai-agent-memory` slice (tasks §1–§7): the pure `MemoryDocument` parse/serialize + -/// cap/evict (1.x), `MemorySubfile` parse/serialize + malformed boundary (2.x), reconciliation (3.x), -/// the `MemoryStore` IO/containment/write/forget/reconcile (4.x), the shared-index contribution (5.x), -/// the `MemoryToolProvider` descriptors + invoke + audit (6.x), and `MemoryError` + the one translator -/// (7.x). Everything is MLX-free Core, driven against temp dirs + fakes — no model. -@MainActor -final class MemoryTests: XCTestCase { - - private func tempDir() -> URL { - let url = FileManager.default.temporaryDirectory - .appendingPathComponent("tfs-memory-test-\(UUID().uuidString)", isDirectory: true) - return url - } - - // MARK: - 1. CORE document model - - func testCoreDocumentRoundTripIsAFixedPoint() { - let doc = MemoryDocument( - facts: [MemoryFact("I work at Acme"), MemoryFact("My partner is Dana")], - contents: [MemoryTOCEntry(name: "acme-migration", summary: "Notes on the Acme migration.")]) - let reparsed = MemoryDocument.parse(doc.serialized()) - XCTAssertEqual(doc, reparsed) - // serialize is a fixed point. - XCTAssertEqual(doc.serialized(), reparsed.serialized()) - } - - func testEmptyAndPartialDocumentsParse() { - XCTAssertEqual(MemoryDocument.parse(""), MemoryDocument()) - let factsOnly = MemoryDocument(facts: [MemoryFact("a")]) - XCTAssertEqual(MemoryDocument.parse(factsOnly.serialized()), factsOnly) - let tocOnly = MemoryDocument(contents: [MemoryTOCEntry(name: "n", summary: "s")]) - XCTAssertEqual(MemoryDocument.parse(tocOnly.serialized()), tocOnly) - } - - // MARK: - 1.2 Cap + eviction - - func testFactCapBindsAndEvicts() { - let cap = MemoryCap(maxBytes: 100_000, maxFacts: 3) - var doc = MemoryDocument(facts: (0..<3).map { MemoryFact("fact-\($0)") }) - XCTAssertFalse(doc.exceeds(cap)) - doc.facts.append(MemoryFact("fact-4-and-this-is-the-longest-one-so-it-evicts")) - XCTAssertTrue(doc.exceeds(cap)) - let (kept, evicted) = doc.evicting(toFit: cap) - XCTAssertFalse(kept.exceeds(cap)) - XCTAssertEqual(evicted.count, 1) - // Longest fact is the eviction victim (oldest-among-longest). - XCTAssertEqual(evicted.first?.text, "fact-4-and-this-is-the-longest-one-so-it-evicts") - } - - func testByteCapBindsBeforeFactCap() { - let cap = MemoryCap(maxBytes: 80, maxFacts: 1000) - let doc = MemoryDocument(facts: [MemoryFact(String(repeating: "x", count: 200))]) - XCTAssertTrue(doc.exceeds(cap)) - // A single fact larger than the whole cap cannot be made to fit by eviction. - let (kept, evicted) = doc.evicting(toFit: cap) - XCTAssertTrue(kept.facts.isEmpty) - XCTAssertEqual(evicted.count, 1) - } - - func testEvictionIsDeterministicOldestAmongLongest() { - let cap = MemoryCap(maxBytes: 100_000, maxFacts: 2) - // Two equally-long facts; the OLDER (earlier index) evicts first. - let doc = MemoryDocument(facts: [MemoryFact("AAAA"), MemoryFact("BBBB"), MemoryFact("c")]) - let (kept, evicted) = doc.evicting(toFit: cap) - XCTAssertEqual(evicted.first?.text, "AAAA") - XCTAssertEqual(kept.facts.map(\.text), ["BBBB", "c"]) - } - - func testWouldExceedCap() { - let cap = MemoryCap(maxBytes: 100_000, maxFacts: 1) - let doc = MemoryDocument(facts: [MemoryFact("one")]) - XCTAssertTrue(doc.wouldExceedCap(addingFact: MemoryFact("two"), cap: cap)) - } - - // MARK: - 2. Subfile model - - func testSubfileRoundTrip() { - let sub = MemorySubfile(name: "acme-migration", summary: "Notes on the Acme migration.", - keywords: ["acme", "migration"], - body: "The Acme migration is targeted for Q3.", - updatedAt: Date(timeIntervalSince1970: 1_700_000_000)) - guard case let .success(reparsed) = MemorySubfile.parse(sub.serialized(), fallbackName: "x") else { - return XCTFail("subfile should re-parse") - } - XCTAssertEqual(reparsed.name, sub.name) - XCTAssertEqual(reparsed.summary, sub.summary) - XCTAssertEqual(reparsed.keywords, sub.keywords) - XCTAssertEqual(reparsed.body, sub.body) - } - - func testMalformedSubfileIsBoundedProblem() { - // Missing front-matter delimiters. - if case let .failure(e) = MemorySubfile.parse("just a body", fallbackName: "bad") { - XCTAssertEqual(e, .malformedSubfile(name: "bad", detail: "Missing the `---` front-matter delimiters.")) - } else { XCTFail("should be malformed") } - // Missing required summary. - let noSummary = "---\nname: x\n---\nbody" - if case let .failure(e) = MemorySubfile.parse(noSummary, fallbackName: "x") { - if case .malformedSubfile = e {} else { XCTFail("expected malformedSubfile") } - } else { XCTFail("should be malformed") } - } - - // MARK: - 3. Reconciliation - - func testReconcileDropsStaleTOCAddsMissing() { - let doc = MemoryDocument( - facts: [MemoryFact("keep me")], - contents: [MemoryTOCEntry(name: "ghost", summary: "no backing file"), - MemoryTOCEntry(name: "real", summary: "stale summary")]) - let subfiles = [ - MemorySubfile(name: "real", summary: "fresh summary", body: "b"), - MemorySubfile(name: "orphan", summary: "no toc entry", body: "b")] - let result = doc.reconciled(withSubfiles: subfiles) - XCTAssertEqual(result.facts, doc.facts) // facts untouched - let names = result.contents.map(\.name) - XCTAssertFalse(names.contains("ghost")) // dropped - XCTAssertTrue(names.contains("real")) - XCTAssertTrue(names.contains("orphan")) // re-added - // summary refreshed from the subfile front-matter. - XCTAssertEqual(result.contents.first { $0.name == "real" }?.summary, "fresh summary") - } - - func testReconcileConsistentIsNoOp() { - let subfiles = [MemorySubfile(name: "a", summary: "s", body: "b")] - let doc = MemoryDocument(contents: [MemoryTOCEntry(name: "a", summary: "s")]) - XCTAssertEqual(doc.reconciled(withSubfiles: subfiles).contents, - [MemoryTOCEntry(name: "a", summary: "s")]) - } - - // MARK: - 4. MemoryStore IO + containment - - func testStoreCreatesDirectoryAndContainsHostileName() throws { - let dir = tempDir() - let store = MemoryStore(directory: dir) - _ = try store.write(scope: .subfile, name: "../../escape/etc", summary: "s", content: "secret") - // The subfile landed INSIDE the memory folder (containment), not at the traversal target. - let subfilesDir = dir.appendingPathComponent("subfiles") - let files = try FileManager.default.contentsOfDirectory(at: subfilesDir, includingPropertiesForKeys: nil) - XCTAssertEqual(files.count, 1) - // The file is rooted under the subfiles folder (compare resolved paths to dodge the - // /var ↔ /private/var temp-dir symlink). - XCTAssertTrue(files[0].resolvingSymlinksInPath().path - .hasPrefix(subfilesDir.resolvingSymlinksInPath().path)) - XCTAssertFalse(files[0].path.contains("/etc")) - } - - func testFactWriteOverCapEvictsToSubfile() throws { - let dir = tempDir() - let store = MemoryStore(directory: dir) - let cap = MemoryCap(maxBytes: 100_000, maxFacts: 2) - _ = try store.write(scope: .fact, name: nil, summary: nil, content: "short identity fact", cap: cap) - _ = try store.write(scope: .fact, name: nil, summary: nil, content: "another short fact", cap: cap) - let outcome = try store.write(scope: .fact, name: nil, summary: nil, - content: "a third much longer detail-bearing fact line here", cap: cap) - XCTAssertNotNil(outcome.evictedToSubfile) - let core = try store.loadCore() - XCTAssertFalse(core.exceeds(cap)) - // The eviction subfile exists and shows in the TOC. - XCTAssertTrue(core.contents.contains { $0.name == MemoryStore.evictionSubfileName }) - } - - func testSingleFactOverCapPromoteIsCapExceeded() { - let dir = tempDir() - let store = MemoryStore(directory: dir) - let cap = MemoryCap(maxBytes: 50, maxFacts: 100) - XCTAssertThrowsError(try store.promote(content: String(repeating: "x", count: 200), cap: cap)) { err in - XCTAssertEqual(err as? MemoryError, .capExceeded) - } - } - - func testSingleFactOverCapWriteRoutesToSubfile() throws { - let dir = tempDir() - let store = MemoryStore(directory: dir) - let cap = MemoryCap(maxBytes: 50, maxFacts: 100) - let outcome = try store.write(scope: .fact, name: nil, summary: nil, - content: String(repeating: "y", count: 200), cap: cap) - XCTAssertNotNil(outcome.evictedToSubfile) // routed to a subfile, NOT kept in core - XCTAssertTrue(try store.loadCore().facts.isEmpty) - } - - func testSubfileWriteAddsTOCEntry() throws { - let dir = tempDir() - let store = MemoryStore(directory: dir) - _ = try store.write(scope: .subfile, name: "acme", summary: "Acme notes", content: "detail") - let core = try store.loadCore() - XCTAssertTrue(core.contents.contains { $0.name == "acme" && $0.summary == "Acme notes" }) - } - - func testUpdateMissingSubfileThrows() { - let store = MemoryStore(directory: tempDir()) - XCTAssertThrowsError(try store.update(name: "nope", content: "x", summary: nil)) { err in - XCTAssertEqual(err as? MemoryError, .subfileNotFound(name: "nope")) - } - } - - func testForgetSingleSubfileAndBulkDangerous() throws { - let dir = tempDir() - let store = MemoryStore(directory: dir) - _ = try store.write(scope: .subfile, name: "acme-one", summary: "acme thing", content: "a") - _ = try store.write(scope: .subfile, name: "acme-two", summary: "acme thing", content: "b") - _ = try store.write(scope: .subfile, name: "dana", summary: "partner", content: "c") - - // Single targeted forget — not dangerous. - let single = try store.forget(scope: .subfile, name: "dana", match: nil) - XCTAssertEqual(single.removedCount, 1) - XCTAssertFalse(single.dangerous) - XCTAssertFalse(try store.loadCore().contents.contains { $0.name == "dana" }) - - // Broad match hitting many → dangerous. - let bulk = try store.forget(scope: nil, name: nil, match: "acme") - XCTAssertEqual(bulk.removedCount, 2) - XCTAssertTrue(bulk.dangerous) - - // No-match forget is a clean no-op. - let none = try store.forget(scope: nil, name: nil, match: "zzz-nothing") - XCTAssertEqual(none.removedCount, 0) - XCTAssertFalse(none.dangerous) - } - - func testWatchReloadPicksUpExternalEdit() throws { - let dir = tempDir() - let store = MemoryStore(directory: dir) - _ = try store.write(scope: .fact, name: nil, summary: nil, content: "first") - // Simulate an out-of-band user hand-edit of core.md. - let core = dir.appendingPathComponent("core.md") - let edited = MemoryDocument(facts: [MemoryFact("first"), MemoryFact("hand added")]) - try Data(edited.serialized().utf8).write(to: core, options: .atomic) - // A fresh load reflects the external change. - XCTAssertTrue(try store.loadCore().facts.contains(MemoryFact("hand added"))) - } - - // MARK: - 5. Shared-index contribution - - func testMemoryDocsMergeWithSkillsInOneIndex() throws { - let dir = tempDir() - let store = MemoryStore(directory: dir) - _ = try store.write(scope: .fact, name: nil, summary: nil, content: "I work at Acme") - _ = try store.write(scope: .subfile, name: "acme-migration", - summary: "Notes on the Acme migration project deadline.", - content: "Targeted for Q3. Dana owns schema.") - let (memDocs, memBodies) = try store.indexedDocs() - - // A skill doc, merged into the SAME snapshot (no second retriever). - let skillDoc = IndexedDoc(id: "fix-grammar", title: "Fix Grammar", - summary: "Correct grammar.", keywords: ["grammar"], - kind: .skill, bodyPath: dir) - var allDocs = memDocs; allDocs.append(skillDoc) - var allBodies = memBodies; allBodies["fix-grammar"] = "fix it" - let index = InMemoryDocIndex(docs: allDocs, bodies: allBodies) - - // The memory core + subfile both appear in the one TOC. - let kinds = Set(index.allSummaries().map(\.kind)) - XCTAssertTrue(kinds.contains(.memoryCore)) - XCTAssertTrue(kinds.contains(.memorySubfile)) - XCTAssertTrue(kinds.contains(.skill)) - - // A memory subfile is selectable + body-loadable through the one retriever. - let hits = index.retrieve(query: "acme migration deadline", limit: 5) - let subID = MemorySubfile.docID(for: "acme-migration") - XCTAssertTrue(hits.contains { $0.id == subID }) - XCTAssertEqual(try index.body(of: subID), "Targeted for Q3. Dana owns schema.") - - // Namespaced ids: a memory subfile named "fix-grammar" cannot collide with the skill id. - XCTAssertNotEqual(MemorySubfile.docID(for: "fix-grammar"), "fix-grammar") - } - - // MARK: - 6. MemoryToolProvider descriptors + invoke + audit - - func testDescriptorTiersMatchTable() { - let descriptors = MemoryToolProvider.allDescriptors - func tier(_ name: String) -> WritePolicyTier? { descriptors.first { $0.name == name }?.writePolicy } - XCTAssertEqual(tier(MemoryToolProvider.read), .auto) // read is free - XCTAssertEqual(tier(MemoryToolProvider.write), .confirm) // default confirm (whitelist → auto) - XCTAssertEqual(tier(MemoryToolProvider.update), .confirm) - XCTAssertEqual(tier(MemoryToolProvider.forget), .confirm) - XCTAssertEqual(tier(MemoryToolProvider.promote), .confirm) - XCTAssertEqual(descriptors.count, 5) - } - - func testInvokeReadIsFreeAndAudited() async throws { - let dir = tempDir() - let store = MemoryStore(directory: dir) - _ = try store.write(scope: .fact, name: nil, summary: nil, content: "I prefer metric units") - let audit = RecordingAudit() - let provider = MemoryToolProvider(store: store, audit: audit, isBackground: true) - let result = await provider.invoke(tool: MemoryToolProvider.read, argumentsJSON: "{}") - XCTAssertEqual(result.status, .done) - XCTAssertTrue(result.summary.contains("I prefer metric units")) - XCTAssertEqual(audit.records.count, 1) - XCTAssertEqual(audit.records[0].policy, .auto) - XCTAssertTrue(audit.records[0].wasBackground) - } - - func testInvokeWriteAppliesAndRedactsAudit() async throws { - let dir = tempDir() - let store = MemoryStore(directory: dir) - let audit = RecordingAudit() - let provider = MemoryToolProvider(store: store, audit: audit) - let args = "{\"scope\":\"subfile\",\"name\":\"secret-note\",\"content\":\"the password is hunter2\"}" - let result = await provider.invoke(tool: MemoryToolProvider.write, argumentsJSON: args) - XCTAssertEqual(result.status, .done) - XCTAssertTrue(try store.loadCore().contents.contains { $0.name == "secret-note" }) - // The audit summary is REDACTED — the raw secret never appears. - let summary = audit.records[0].argumentsSummary - XCTAssertFalse(summary.contains("hunter2")) - XCTAssertTrue(summary.contains("content len=")) - } - - func testInvokeFailedWriteIsFailedNeverFalseDone() async { - // A store rooted at a path that cannot be created (a file masquerading as the parent dir). - let badParent = FileManager.default.temporaryDirectory - .appendingPathComponent("tfs-mem-blocked-\(UUID().uuidString)") - try? Data("x".utf8).write(to: badParent) // a FILE where the store wants a DIRECTORY - let store = MemoryStore(directory: badParent.appendingPathComponent("memory")) - let audit = RecordingAudit() - let provider = MemoryToolProvider(store: store, audit: audit) - let args = "{\"scope\":\"subfile\",\"name\":\"x\",\"content\":\"y\"}" - let result = await provider.invoke(tool: MemoryToolProvider.write, argumentsJSON: args) - if case .failed = result.status {} else { XCTFail("a non-landing write must be .failed") } - if case .failed = audit.records.last?.outcome {} else { XCTFail("the failure must be audited") } - } - - func testInvokeMalformedArgsDeclines() async { - let store = MemoryStore(directory: tempDir()) - let provider = MemoryToolProvider(store: store) - let result = await provider.invoke(tool: MemoryToolProvider.write, argumentsJSON: "{}") - if case .declined = result.status {} else { XCTFail("missing content should decline") } - } - - func testInvokeBulkForgetIsDangerousPolicyInAudit() async throws { - let dir = tempDir() - let store = MemoryStore(directory: dir) - _ = try store.write(scope: .subfile, name: "acme-a", summary: "acme", content: "a") - _ = try store.write(scope: .subfile, name: "acme-b", summary: "acme", content: "b") - let audit = RecordingAudit() - let provider = MemoryToolProvider(store: store, audit: audit) - let result = await provider.invoke(tool: MemoryToolProvider.forget, argumentsJSON: "{\"match\":\"acme\"}") - XCTAssertEqual(result.status, .done) - XCTAssertEqual(audit.records.last?.policy, .dangerous) - } - - // MARK: - 7. Error taxonomy + translator - - func testMemoryErrorDescriptionsAreClean() { - let errors: [MemoryError] = [ - .unreadableCore(detail: "NSError raw blah"), - .writeFailed(detail: "EPERM raw blah"), - .subfileNotFound(name: "acme"), - .capExceeded, - .malformedSubfile(name: "bad", detail: "YAML exploded raw blah")] - for e in errors { - let headline = e.errorDescription ?? "" - XCTAssertFalse(headline.isEmpty) - XCTAssertFalse(headline.contains("raw blah")) // raw text never in the headline - } - } - - func testTranslatorRoutesMemoryError() { - let presented = AIError.message(for: MemoryError.writeFailed(detail: "EPERM raw detail")) - XCTAssertEqual(presented.headline, "That memory couldn't be saved.") - XCTAssertFalse(presented.headline.contains("EPERM")) - XCTAssertEqual(presented.details, "EPERM raw detail") // raw text only in opt-in details - // A clean case with no extra detail. - let clean = AIError.message(for: MemoryError.capExceeded) - XCTAssertNil(clean.details) - } -} - -/// A recording `MemoryAuditing` fake (the durable conformer is owned by `ai-background-autonomy`). -private final class RecordingAudit: MemoryAuditing, @unchecked Sendable { - private(set) var records: [MemoryAuditRecord] = [] - func record(_ record: MemoryAuditRecord) { records.append(record) } -} diff --git a/Tests/ThreeFingerSwitcherTests/ModelManagerFleetTests.swift b/Tests/ThreeFingerSwitcherTests/ModelManagerFleetTests.swift deleted file mode 100644 index f4e6eed..0000000 --- a/Tests/ThreeFingerSwitcherTests/ModelManagerFleetTests.swift +++ /dev/null @@ -1,162 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// `ModelManager` consuming the fleet registry + planner around the EXISTING provisioner/runtimeFactory -/// (tasks 5.1–5.3, 6.2). Dev-stub path — no real weights. `StubLLMRuntime`-backed. -@MainActor -final class ModelManagerFleetTests: XCTestCase { - - private let GB: UInt64 = 1024 * 1024 * 1024 - - // A downloader that's never used on the fleet path (loadDescriptor uses the runtimeFactory). - private struct UnusedDownloader: ModelDownloading { - func download(_ d: ModelDescriptor, to: URL, progress: @Sendable (Double) -> Void) async throws -> Data { - Data() - } - } - - /// Records every descriptor id the runtimeFactory builds, so a test can assert load order. - private final class LoadRecorder: @unchecked Sendable { - private let lock = NSLock() - private(set) var loaded: [String] = [] - func record(_ id: String) { lock.lock(); loaded.append(id); lock.unlock() } - var ids: [String] { lock.lock(); defer { lock.unlock() }; return loaded } - } - - private func descriptor(_ id: String, role: ModelRole, lane: ComputeLane?, gb: UInt64, - provider: ModelProvider = .onDevice) -> ModelDescriptor { - ModelDescriptor(id: id, displayName: id, sizeBytes: Int64(gb * GB), integritySHA: "x", - downloadURL: URL(string: "https://example.com/\(id)")!, - capabilities: [.text], quantization: .qat4bit, - role: role, lane: lane, provider: provider, residencyBytes: gb * GB) - } - - private func chat() -> ModelDescriptor { descriptor("chat", role: .chat, lane: .gpu, gb: 17) } - private func ternary() -> ModelDescriptor { descriptor("ternary", role: .ternaryChat, lane: .cpuTernary, gb: 1) } - private func video() -> ModelDescriptor { descriptor("video", role: .video, lane: .gpu, gb: 24) } - private func cloud() -> ModelDescriptor { descriptor("cloud", role: .cloudEscalation, lane: nil, gb: 0, provider: .cloud) } - - private func manager(members: [ModelDescriptor], - recorder: LoadRecorder, - cloudGate: FleetCloudGate = FleetCloudGate(), - freeBytes: @escaping @Sendable () -> UInt64 = { 48 * 1024 * 1024 * 1024 }) -> ModelManager { - let fleet = FleetRoster(members: members, freeBytesProbe: freeBytes) - let m = ModelManager( - downloader: UnusedDownloader(), - optedIn: true, - fleet: fleet, - fleetFreeBytes: freeBytes, - cloudGate: cloudGate, - runtimeFactory: { d in - recorder.record(d.id) - return StubLLMRuntime(capabilities: d.capabilities) - }) - return m - } - - // MARK: - 5.1 evict-then-load - - func testEnsureResidentEvictsThenLoadsTarget() async throws { - let rec = LoadRecorder() - let m = manager(members: [chat(), ternary(), video()], recorder: rec) - - // Load chat resident first. - try await m.ensureResident("chat") - XCTAssertEqual(rec.ids, ["chat"]) - XCTAssertTrue(m.isResident) - - // Admitting video must evict chat (the resident runtime) then load video. - try await m.ensureResident("video") - XCTAssertEqual(rec.ids, ["chat", "video"], "video loaded after chat was evicted") - // The fleet's resident view no longer lists chat. - XCTAssertFalse(m.fleetResident().map(\.id).contains("chat")) - XCTAssertTrue(m.fleetResident().map(\.id).contains("video")) - } - - /// A cannotAdmit plan throws FleetError.cannotAdmit and leaves state .failed, never a false .loaded. - func testCannotAdmitThrowsAndFails() async throws { - let rec = LoadRecorder() - let huge = descriptor("huge", role: .video, lane: .gpu, gb: 60) - let m = manager(members: [chat(), ternary(), huge], recorder: rec) - try await m.ensureResident("chat") - - do { - try await m.ensureResident("huge") - XCTFail("expected cannotAdmit") - } catch let error as FleetError { - guard case .cannotAdmit = error else { return XCTFail("wrong case: \(error)") } - } - if case .failed = m.state {} else { XCTFail("state must be .failed, not a false .loaded") } - } - - // MARK: - 5.2 fleet-of-one - - func testFleetOfOneLoadsLikeSingleModel() async throws { - let rec = LoadRecorder() - let m = manager(members: [chat()], recorder: rec) - try await m.ensureResident("chat") - XCTAssertEqual(rec.ids, ["chat"], "no eviction, a single lazy load") - XCTAssertTrue(m.isResident) - if case .loaded = m.state {} else { XCTFail("fleet-of-one settles .loaded") } - // A second ensureResident of the same model is a warm hit (no re-load). - try await m.ensureResident("chat") - XCTAssertEqual(rec.ids, ["chat"], "warm hit re-loads nothing") - } - - // MARK: - 5.3 cloud no-op - - func testCloudEnsureResidentIsResidencyNoOp() async throws { - let rec = LoadRecorder() - // Cloud enabled so select() routes (no throw); residency must still be untouched. - let gate = FleetCloudGate(isEnabled: { true }) - let m = manager(members: [chat(), cloud()], recorder: rec, cloudGate: gate) - try await m.ensureResident("chat") - let before = m.fleetResident().map(\.id) - - try await m.ensureResident("cloud") - XCTAssertEqual(rec.ids, ["chat"], "cloud never loads weights") - XCTAssertEqual(m.fleetResident().map(\.id), before, "resident() unchanged by a cloud admission") - } - - // MARK: - 6.2 cloud gating - - func testCloudDisabledThrowsCloudDisabled() async throws { - let rec = LoadRecorder() - let gate = FleetCloudGate(isEnabled: { false }) // cloud OFF - let m = manager(members: [chat(), cloud()], recorder: rec, cloudGate: gate) - do { - try await m.ensureResident("cloud") - XCTFail("expected cloudDisabled") - } catch let error as FleetError { - guard case .cloudDisabled = error else { return XCTFail("wrong case: \(error)") } - } - XCTAssertTrue(rec.ids.isEmpty, "nothing loaded for a gated-off cloud member") - } - - func testCloudEnabledRoutesToHandoffSpy() async throws { - final class EscalationSpy: FleetCloudEscalating, @unchecked Sendable { - let lock = NSLock(); private(set) var routed: [String] = [] - func escalate(to descriptor: ModelDescriptor) async throws { - lock.lock(); routed.append(descriptor.id); lock.unlock() - } - var ids: [String] { lock.lock(); defer { lock.unlock() }; return routed } - } - let spy = EscalationSpy() - let gate = FleetCloudGate(isEnabled: { true }, escalator: spy) - let rec = LoadRecorder() - let m = manager(members: [chat(), cloud()], recorder: rec, cloudGate: gate) - try await m.ensureResident("cloud") - XCTAssertEqual(spy.ids, ["cloud"], "an enabled cloud selection routes to the escalation seam") - XCTAssertTrue(rec.ids.isEmpty, "still loads no weights") - } - - // MARK: - 6.2 selectableDescriptors gating - - func testSelectableDescriptorsHidesCloudWhenOff() { - let descs = [chat(), cloud()] - let off = FleetCloudGate(isEnabled: { false }) - XCTAssertEqual(off.selectableDescriptors(from: descs).map(\.id), ["chat"]) - let on = FleetCloudGate(isEnabled: { true }) - XCTAssertEqual(Set(on.selectableDescriptors(from: descs).map(\.id)), ["chat", "cloud"]) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/ModelManagerTests.swift b/Tests/ThreeFingerSwitcherTests/ModelManagerTests.swift deleted file mode 100644 index c925501..0000000 --- a/Tests/ThreeFingerSwitcherTests/ModelManagerTests.swift +++ /dev/null @@ -1,698 +0,0 @@ -import XCTest -import Combine -@testable import ThreeFingerSwitcherCore - -/// Tests for the model lifecycle (spec: "Model lifecycle management") against a fake downloader — -/// NO real network: no download while the opt-in is off, integrity (SHA) verified before load with a -/// corrupt download rejected (never loaded), residency across two calls (no re-load), and eviction -/// (on demand / opt-in off) unloads the resident runtime. -@MainActor -final class ModelManagerTests: XCTestCase { - - // MARK: - Fakes - - /// A fake downloader that fabricates the bytes it "downloads" — never touches the network. - /// Counts calls so residency tests can prove a second run did NOT re-download. - private final class FakeDownloader: ModelDownloading, @unchecked Sendable { - /// The bytes to hand back on download (default: a fixed payload). - let payload: Data - private(set) var downloadCount = 0 - private let lock = NSLock() - - init(payload: Data = Data("gemma-weights".utf8)) { - self.payload = payload - } - - func download(_ descriptor: ModelDescriptor, to destination: URL, - progress: @Sendable (Double) -> Void) async throws -> Data { - bumpCount() - progress(0.5) - progress(1.0) - return payload - } - - private func bumpCount() { lock.lock(); downloadCount += 1; lock.unlock() } - var count: Int { lock.lock(); defer { lock.unlock() }; return downloadCount } - } - - // MARK: - Helpers - - private func tempRoot() -> URL { - let url = URL(fileURLWithPath: NSTemporaryDirectory()) - .appendingPathComponent("tfs-model-tests-\(UUID().uuidString)", isDirectory: true) - try? FileManager.default.createDirectory(at: url, withIntermediateDirectories: true) - return url - } - - /// A registry with a single descriptor whose `integritySHA` matches `payload`, so the happy path - /// verifies. Capabilities default to text+vision. - private func registry(matching payload: Data, - capabilities: Set = [.text, .vision], - id: String = "test-model") -> ModelCatalog { - ModelCatalog( - models: [ModelDescriptor( - id: id, - displayName: "Test Model", - sizeBytes: Int64(payload.count), - integritySHA: ModelManager.sha256Hex(payload), - downloadURL: URL(string: "https://models.invalid/\(id)")!, - capabilities: capabilities, - quantization: .qat4bit - )], - defaultModelID: id - ) - } - - // MARK: - No download when opt-in off - - func testNoDownloadWhenOptInOff() async { - let payload = Data("w".utf8) - let downloader = FakeDownloader(payload: payload) - let manager = ModelManager(registry: registry(matching: payload), - downloader: downloader, - optedIn: false, - storageRoot: tempRoot()) - let descriptor = manager.registry.models[0] - do { - try await manager.downloadAndVerify(descriptor) - XCTFail("download must be refused while the opt-in is off") - } catch let e as RuntimeError { - guard case .unavailable = e else { return XCTFail("expected .unavailable, got \(e)") } - } catch { - XCTFail("unexpected error: \(error)") - } - XCTAssertEqual(downloader.count, 0, "no bytes are fetched while opted out") - XCTAssertEqual(manager.state, .notDownloaded) - } - - // MARK: - Corrupt-hash rejection - - func testCorruptDownloadIsRejectedAndNeverLoaded() async { - let realPayload = Data("good-weights".utf8) - // The descriptor declares the hash of the REAL payload, but the downloader returns garbage. - let manager = ModelManager(registry: registry(matching: realPayload), - downloader: FakeDownloader(payload: Data("corrupted".utf8)), - optedIn: true, - storageRoot: tempRoot()) - let descriptor = manager.registry.models[0] - do { - try await manager.downloadAndVerify(descriptor) - XCTFail("a corrupt download must not verify") - } catch let e as RuntimeError { - XCTAssertEqual(e, .integrityFailed) - } catch { - XCTFail("unexpected error: \(error)") - } - guard case .failed = manager.state else { - return XCTFail("corrupt download leaves the manager in .failed, got \(manager.state)") - } - // And it must not be loadable. - do { - _ = try await manager.loadIfNeeded() - XCTFail("a corrupt model must never load") - } catch let e as RuntimeError { - XCTAssertEqual(e, .modelMissing, "no verified weights → modelMissing") - } catch { - XCTFail("unexpected error: \(error)") - } - } - - // MARK: - Happy path: verify then load - - func testVerifyThenLoadReachesLoadedState() async throws { - let payload = Data("good".utf8) - let manager = ModelManager(registry: registry(matching: payload), - downloader: FakeDownloader(payload: payload), - optedIn: true, - storageRoot: tempRoot()) - let descriptor = manager.registry.models[0] - try await manager.downloadAndVerify(descriptor) - XCTAssertEqual(manager.state, .ready, "verified weights leave the manager .ready") - - let runtime = try await manager.loadIfNeeded() - XCTAssertEqual(manager.state, .loaded) - XCTAssertTrue(runtime.capabilities.contains(.vision), "the resolved runtime mirrors capabilities") - XCTAssertTrue(manager.isResident) - } - - // MARK: - Residency across two calls (no re-load / no re-download) - - func testResidencyAcrossTwoCallsDoesNotReload() async throws { - let payload = Data("good".utf8) - let downloader = FakeDownloader(payload: payload) - let manager = ModelManager(registry: registry(matching: payload), - downloader: downloader, - optedIn: true, - storageRoot: tempRoot()) - let descriptor = manager.registry.models[0] - try await manager.downloadAndVerify(descriptor) - - let first = try await manager.loadIfNeeded() - let second = try await manager.loadIfNeeded() - XCTAssertTrue((first as AnyObject) === (second as AnyObject), - "the same resident runtime is reused (no cold re-load)") - XCTAssertEqual(downloader.count, 1, "residency means no second download") - XCTAssertEqual(manager.state, .loaded) - } - - func testRuntimeForCapabilityResolvesResident() async throws { - let payload = Data("good".utf8) - let manager = ModelManager(registry: registry(matching: payload), - downloader: FakeDownloader(payload: payload), - optedIn: true, - storageRoot: tempRoot()) - try await manager.downloadAndVerify(manager.registry.models[0]) - let runtime = try await manager.runtime(requiring: [.vision]) - XCTAssertTrue(runtime.capabilities.contains(.vision)) - XCTAssertTrue(manager.isResident) - } - - func testRuntimeForCapabilityReportsMissingWhenNotDownloaded() async { - let payload = Data("good".utf8) - let manager = ModelManager(registry: registry(matching: payload), - downloader: FakeDownloader(payload: payload), - optedIn: true, - storageRoot: tempRoot()) - do { - _ = try await manager.runtime(requiring: [.text]) - XCTFail("a non-downloaded model must report missing, not silently fetch") - } catch let e as RuntimeError { - XCTAssertEqual(e, .modelMissing) - } catch { - XCTFail("unexpected error: \(error)") - } - } - - // MARK: - Eviction - - func testEvictUnloadsButKeepsVerifiedWeights() async throws { - let payload = Data("good".utf8) - let downloader = FakeDownloader(payload: payload) - let manager = ModelManager(registry: registry(matching: payload), - downloader: downloader, - optedIn: true, - storageRoot: tempRoot()) - try await manager.downloadAndVerify(manager.registry.models[0]) - _ = try await manager.loadIfNeeded() - XCTAssertTrue(manager.isResident) - - manager.evict() - XCTAssertFalse(manager.isResident, "eviction unloads the resident runtime") - XCTAssertEqual(manager.state, .ready, "verified weights remain (a warm re-load, not re-download)") - - // Re-load is warm: no second download. - _ = try await manager.loadIfNeeded() - XCTAssertTrue(manager.isResident) - XCTAssertEqual(downloader.count, 1, "eviction does not force a re-download") - } - - func testOptingOutEvictsAndResets() async throws { - let payload = Data("good".utf8) - let manager = ModelManager(registry: registry(matching: payload), - downloader: FakeDownloader(payload: payload), - optedIn: true, - storageRoot: tempRoot()) - try await manager.downloadAndVerify(manager.registry.models[0]) - _ = try await manager.loadIfNeeded() - XCTAssertTrue(manager.isResident) - - manager.setOptedIn(false) - XCTAssertFalse(manager.isResident, "turning the opt-in off evicts immediately") - XCTAssertEqual(manager.state, .notDownloaded, "opt-out resets lifecycle state") - } - - // MARK: - Loading-state observability - - /// The lifecycle must be a VISIBLE sequence of states the UI can bind to (design D4), never a - /// silent block. We subscribe to `$state` across a full download → verify → load cycle and assert - /// the intermediate states surface: `.downloading(progress:)` (including the fake's 0.5 and 1.0 - /// progress values), `.verifying`, `.loading`, and the terminal `.loaded`. - func testStatePublishesIntermediateLoadingSequence() async throws { - let payload = Data("good".utf8) - let manager = ModelManager(registry: registry(matching: payload), - downloader: FakeDownloader(payload: payload), - optedIn: true, - storageRoot: tempRoot()) - - // Collect every published state. `$state` emits the current value on subscription and then on - // each change; the manager is @MainActor so synchronous transitions arrive in order. - var observed: [ModelLifecycleState] = [] - let cancellable = manager.$state.sink { observed.append($0) } - defer { cancellable.cancel() } - - try await manager.downloadAndVerify(manager.registry.models[0]) - try await manager.loadIfNeeded() - - // The progress callbacks (0.5, 1.0) are surfaced via `Task { @MainActor … }`, so drain the main - // queue with a bounded poll until both progress values appear (no real sleep, no wall clock). - for _ in 0..<200 { - let hasHalf = observed.contains(.downloading(progress: 0.5)) - let hasFull = observed.contains(.downloading(progress: 1.0)) - if hasHalf && hasFull { break } - await Task.yield() - } - - XCTAssertTrue(observed.contains(.downloading(progress: 0)), - "the download phase surfaces an initial .downloading(progress: 0)") - XCTAssertTrue(observed.contains(.downloading(progress: 0.5)), - "the fake downloader's 0.5 progress is surfaced as observable state") - XCTAssertTrue(observed.contains(.downloading(progress: 1.0)), - "the fake downloader's 1.0 progress is surfaced as observable state") - XCTAssertTrue(observed.contains(.verifying), "the integrity check is a visible .verifying state") - XCTAssertTrue(observed.contains(.loading), "the lazy-load is a visible .loading state") - XCTAssertTrue(observed.contains(.loaded), "the cycle surfaces the terminal .loaded state") - XCTAssertEqual(manager.state, .loaded, "the manager settles resident in .loaded") - } - - // MARK: - Strong-hardware-only guard - - func testUnsupportedHardwareReportsUnavailable() async { - let payload = Data("good".utf8) - let manager = ModelManager(registry: registry(matching: payload), - downloader: FakeDownloader(payload: payload), - optedIn: true, - storageRoot: tempRoot(), - hardwareSupports: { _ in false }) - do { - try await manager.downloadAndVerify(manager.registry.models[0]) - XCTFail("unsupported hardware must report unavailable, not degrade") - } catch let e as RuntimeError { - guard case .unavailable = e else { return XCTFail("expected .unavailable, got \(e)") } - } catch { - XCTFail("unexpected error: \(error)") - } - guard case .failed = manager.state else { - return XCTFail("unsupported hardware leaves the manager .failed, got \(manager.state)") - } - } - - // MARK: - A non-cancel download error resolves the state (never stuck .downloading) - - /// A downloader that throws a scripted error instead of returning bytes. - private final class ThrowingDownloader: ModelDownloading, @unchecked Sendable { - let error: Error - init(_ error: Error) { self.error = error } - func download(_ descriptor: ModelDescriptor, to destination: URL, - progress: @Sendable (Double) -> Void) async throws -> Data { - progress(0.25) - throw error - } - } - - func testNonCancelDownloadErrorEndsFailedWithCleanHeadline() async { - let payload = Data("good".utf8) - // A bare offline NSError — the byte path's generic catch must convert it to a clean .failed. - let offline = NSError(domain: NSURLErrorDomain, code: NSURLErrorNotConnectedToInternet) - let manager = ModelManager(registry: registry(matching: payload), - downloader: ThrowingDownloader(offline), - optedIn: true, - storageRoot: tempRoot()) - do { - try await manager.downloadAndVerify(manager.registry.models[0]) - XCTFail("a failing download must throw") - } catch { - // any error type is fine; the STATE is the assertion below - } - guard case let .failed(reason, _) = manager.state else { - return XCTFail("a non-cancel download error must end .failed (not stuck .downloading), got \(manager.state)") - } - XCTAssertEqual(reason, RuntimeError.offline.errorDescription, - "the headline is the clean connectivity message") - XCTAssertFalse(reason.contains("Domain="), "the headline never carries the raw NSError dump") - } - - func testCancelledDownloadReturnsToNotDownloadedNotFailed() async { - let payload = Data("good".utf8) - let manager = ModelManager(registry: registry(matching: payload), - downloader: ThrowingDownloader(CancellationError()), - optedIn: true, - storageRoot: tempRoot()) - do { - try await manager.downloadAndVerify(manager.registry.models[0]) - XCTFail("a cancelled download throws") - } catch let e as RuntimeError { - XCTAssertEqual(e, .cancelled, "cancellation is surfaced as RuntimeError.cancelled") - } catch { - XCTFail("unexpected error: \(error)") - } - XCTAssertEqual(manager.state, .notDownloaded, - "cancellation returns to the resting state, never .failed") - } - - // MARK: - Provisioner path: rediscover an already-downloaded model (no re-download) - - /// Thread-safe call counter for the fake provisioner (it may be touched off the main actor). - private final class ProvisionCounter: @unchecked Sendable { - private let lock = NSLock(); private var n = 0 - func bump() { lock.lock(); n += 1; lock.unlock() } - var count: Int { lock.lock(); defer { lock.unlock() }; return n } - } - - /// A provisioner-backed manager (the real-runtime shape) whose disk-probe reports `onDisk` and whose - /// provisioner returns a stub (counting invocations) — so rediscovery + lazy-load are testable with - /// no network and no real MLX. - private func provisionerManager(onDisk: Bool, - counter: ProvisionCounter, - optedIn: Bool = true) -> ModelManager { - let payload = Data("w".utf8) - return ModelManager( - registry: registry(matching: payload), - downloader: FakeDownloader(payload: payload), - optedIn: optedIn, - storageRoot: tempRoot(), - provisioner: { descriptor, progress in - counter.bump() - progress(1.0) - return StubLLMRuntime(capabilities: descriptor.capabilities) - }, - provisionedOnDisk: { _ in onDisk } - ) - } - - func testReconcileDiscoversOnDiskModelAsReady() { - let counter = ProvisionCounter() - let manager = provisionerManager(onDisk: true, counter: counter) - XCTAssertEqual(manager.state, .notDownloaded, "fresh manager starts not-downloaded") - manager.reconcileWithDisk() - XCTAssertEqual(manager.state, .ready, - "an already-downloaded model is rediscovered as .ready (no Download click needed)") - XCTAssertFalse(manager.isResident, "rediscovery does NOT eagerly load (load stays lazy)") - XCTAssertEqual(counter.count, 0, "rediscovery is a pure disk probe — the provisioner never runs") - } - - func testReconcileIsNoOpWhenNothingOnDisk() { - let counter = ProvisionCounter() - let manager = provisionerManager(onDisk: false, counter: counter) - manager.reconcileWithDisk() - XCTAssertEqual(manager.state, .notDownloaded, "nothing on disk → stays not-downloaded") - } - - func testReconcileIsNoOpWhenNotOptedIn() { - let counter = ProvisionCounter() - let manager = provisionerManager(onDisk: true, counter: counter, optedIn: false) - manager.reconcileWithDisk() - XCTAssertEqual(manager.state, .notDownloaded, "no rediscovery while opted out") - } - - /// The real AppCoordinator flow when a user enables AI commands: opt-in flips on, THEN reconcile - /// runs — an already-downloaded model must surface as .ready without a Download click. - func testReconcileAfterOptingInRediscoversOnDiskModel() { - let counter = ProvisionCounter() - let manager = provisionerManager(onDisk: true, counter: counter, optedIn: false) - manager.reconcileWithDisk() - XCTAssertEqual(manager.state, .notDownloaded, "no rediscovery while still opted out") - - manager.setOptedIn(true) - manager.reconcileWithDisk() // mirrors observeAICommandsToggle's on-enable reconcile - XCTAssertEqual(manager.state, .ready, "enabling AI commands rediscovers the on-disk model as .ready") - XCTAssertEqual(counter.count, 0, "rediscovery still never downloads/loads") - } - - func testReconcileNeverRegressesAResolvedState() async throws { - // Guard: reconcile must not clobber an already-resolved lifecycle. Once loaded, a stray - // reconcile (e.g. a second opt-in toggle) must leave .loaded untouched — not drop to .ready. - let counter = ProvisionCounter() - let manager = provisionerManager(onDisk: true, counter: counter) - _ = try await manager.runtime(requiring: [.text]) - XCTAssertEqual(manager.state, .loaded) - - manager.reconcileWithDisk() - XCTAssertEqual(manager.state, .loaded, "reconcile is a no-op on .loaded (never regresses it)") - XCTAssertEqual(counter.count, 1, "the stray reconcile triggered no extra provision") - } - - func testRediscoveredModelLazyLoadsAsLoadingNotDownloading() async throws { - let counter = ProvisionCounter() - let manager = provisionerManager(onDisk: true, counter: counter) - manager.reconcileWithDisk() - XCTAssertEqual(manager.state, .ready) - - var observed: [ModelLifecycleState] = [] - let c = manager.$state.sink { observed.append($0) } - defer { c.cancel() } - - let runtime = try await manager.runtime(requiring: [.text]) - XCTAssertTrue(runtime.capabilities.contains(.text)) - XCTAssertTrue(manager.isResident, "first use lazy-loads the rediscovered model resident") - XCTAssertEqual(manager.state, .loaded) - XCTAssertEqual(counter.count, 1, "the provisioner ran exactly once — to LOAD, not re-download") - XCTAssertTrue(observed.contains(.loading), "a rediscovered load surfaces as .loading") - XCTAssertFalse(observed.contains(where: { if case .downloading = $0 { return true }; return false }), - "loading an already-present model never shows a (misleading) download bar") - } - - func testRuntimeRequestLoadsOnDiskModelWithoutPriorReconcileOrDownload() async throws { - let counter = ProvisionCounter() - let manager = provisionerManager(onDisk: true, counter: counter) - // Straight to a command's runtime request — no reconcile, no downloadAndVerify. - let runtime = try await manager.runtime(requiring: [.text]) - XCTAssertTrue(manager.isResident) - XCTAssertEqual(manager.state, .loaded) - XCTAssertEqual(counter.count, 1, "the on-disk model is loaded on demand, not re-downloaded") - } - - func testRuntimeRequestReportsMissingWhenNothingOnDisk() async { - let counter = ProvisionCounter() - let manager = provisionerManager(onDisk: false, counter: counter) - do { - _ = try await manager.runtime(requiring: [.text]) - XCTFail("nothing on disk must report missing, not silently load") - } catch let e as RuntimeError { - XCTAssertEqual(e, .modelMissing) - } catch { - XCTFail("unexpected error: \(error)") - } - XCTAssertEqual(counter.count, 0, "no provisioner call when nothing is on disk") - } - - func testProvisionerEvictFallsBackToReadyAndReloadsWarm() async throws { - let counter = ProvisionCounter() - let manager = provisionerManager(onDisk: true, counter: counter) - _ = try await manager.runtime(requiring: [.text]) // load resident - XCTAssertEqual(manager.state, .loaded) - XCTAssertEqual(counter.count, 1) - - manager.evict() - XCTAssertFalse(manager.isResident, "evict unloads the resident runtime") - XCTAssertEqual(manager.state, .ready, - "provisioner-path evict falls back to .ready (weights remain on disk), not stuck .loaded") - - // A subsequent request reloads from disk (no re-download): the provisioner runs again to LOAD. - _ = try await manager.runtime(requiring: [.text]) - XCTAssertTrue(manager.isResident) - XCTAssertEqual(manager.state, .loaded) - XCTAssertEqual(counter.count, 2, "the warm reload re-runs the provisioner (load), still no byte fetch path") - } - - // MARK: - Single-flight loading (fix-model-load-coalescing-and-gpu-cache) - - /// A provisioner-backed manager whose provisioner SLEEPS (so racing callers genuinely overlap the - /// multi-second load window the real ~17 GB load has) and counts invocations; optionally fails. - private func slowProvisionerManager(counter: ProvisionCounter, - delayNanos: UInt64 = 50_000_000, - failing: Bool = false) -> ModelManager { - let payload = Data("w".utf8) - return ModelManager( - registry: registry(matching: payload), - downloader: FakeDownloader(payload: payload), - optedIn: true, - storageRoot: tempRoot(), - provisioner: { descriptor, progress in - counter.bump() - try await Task.sleep(nanoseconds: delayNanos) - if failing { throw RuntimeError.modelLoadFailed(detail: "scripted load failure") } - progress(1.0) - return StubLLMRuntime(capabilities: descriptor.capabilities) - }, - provisionedOnDisk: { _ in true } - ) - } - - /// THE double-load regression (caught live in the unified log: two overlapping `prepare` runs → - /// two full weight sets resident): racing loads must coalesce into ONE provisioner run. - func testConcurrentLoadIfNeededRunsProvisionerExactlyOnce() async throws { - let counter = ProvisionCounter() - let manager = slowProvisionerManager(counter: counter) - async let first = manager.loadIfNeeded() - async let second = manager.loadIfNeeded() - let (a, b) = try await (first, second) - XCTAssertEqual(counter.count, 1, "racing loads coalesce into ONE provisioner run") - XCTAssertTrue(a as AnyObject === b as AnyObject, "both racers receive the SAME loaded runtime") - XCTAssertTrue(manager.isResident) - XCTAssertEqual(manager.state, .loaded) - } - - /// The real-world racing pair — a capability-resolved turn and a plain load (e.g. a user send and - /// a background advance during the load window) — shares one load too. - func testRuntimeRequestRacingLoadIfNeededSharesOneLoad() async throws { - let counter = ProvisionCounter() - let manager = slowProvisionerManager(counter: counter) - async let viaCapability = manager.runtime(requiring: [.text]) - async let viaPlainLoad = manager.loadIfNeeded() - _ = try await (viaCapability, viaPlainLoad) - XCTAssertEqual(counter.count, 1, "a turn racing an advance pays the cold-load cost exactly once") - XCTAssertTrue(manager.isResident) - } - - /// A failed shared load fails EVERY joiner (never a hang, never a stub) and clears the in-flight - /// task so a later retry starts a fresh load. - func testFailedSharedLoadFailsBothJoinersAndRetryStartsFresh() async { - let counter = ProvisionCounter() - let manager = slowProvisionerManager(counter: counter, failing: true) - async let first = manager.loadIfNeeded() - async let second = manager.loadIfNeeded() - var failures = 0 - do { _ = try await first; XCTFail("the shared failure must reach the starter") } catch { failures += 1 } - do { _ = try await second; XCTFail("the shared failure must reach the joiner") } catch { failures += 1 } - XCTAssertEqual(failures, 2) - XCTAssertEqual(counter.count, 1, "one failing run — the joiner never triggers a second") - - do { _ = try await manager.loadIfNeeded() } catch { /* fails again; only the count matters */ } - XCTAssertEqual(counter.count, 2, "a retry after failure starts FRESH (the in-flight task was cleared)") - } - - /// Evicting mid-load (opt-in off, a fleet plan) cancels the in-flight load — a completing load must - /// never resurrect a resident model behind the eviction. - func testEvictMidLoadCancelsTheInFlightLoad() async { - let counter = ProvisionCounter() - let manager = slowProvisionerManager(counter: counter, delayNanos: 200_000_000) - async let load = manager.loadIfNeeded() - try? await Task.sleep(nanoseconds: 20_000_000) // let the load get in flight - manager.evict() - do { - _ = try await load - XCTFail("an evicted in-flight load must cancel, not complete") - } catch { /* cancelled — the expected outcome */ } - XCTAssertFalse(manager.isResident, "the cancelled load never resurrects a resident model") - } - - // MARK: - Per-model status + delete - - /// A mutable on-disk model set the provisioner-path probe & delete read/write, so a test can assert a - /// delete actually removes the weights (and a re-probe then reads "not downloaded"). - private final class DiskState: @unchecked Sendable { - private let lock = NSLock() - private var present: Set - private(set) var deleted: [String] = [] - init(_ present: Set) { self.present = present } - func contains(_ id: String) -> Bool { lock.lock(); defer { lock.unlock() }; return present.contains(id) } - func remove(_ id: String) { lock.lock(); present.remove(id); deleted.append(id); lock.unlock() } - } - - /// A two-model registry (default = "model-a") so per-model selection/status is exercisable. - private func twoModelRegistry() -> ModelCatalog { - func descriptor(_ id: String) -> ModelDescriptor { - ModelDescriptor(id: id, displayName: id, sizeBytes: 1, integritySHA: "sha-\(id)", - downloadURL: URL(string: "https://models.invalid/\(id)")!, - capabilities: [.text, .vision], quantization: .qat4bit) - } - return ModelCatalog(models: [descriptor("model-a"), descriptor("model-b")], defaultModelID: "model-a") - } - - /// A provisioner-backed manager whose disk state is the mutable `disk` (probe + delete go through it). - private func managerBackedBy(_ disk: DiskState) -> ModelManager { - ModelManager( - registry: twoModelRegistry(), - downloader: FakeDownloader(), - optedIn: true, - storageRoot: tempRoot(), - provisioner: { descriptor, progress in - progress(1.0) - return StubLLMRuntime(capabilities: descriptor.capabilities) - }, - provisionedOnDisk: { disk.contains($0.id) }, - provisionedDelete: { disk.remove($0.id) } - ) - } - - func testShowStatusTracksSelectedModelPerDisk() { - let disk = DiskState(["model-b"]) // B downloaded, A not - let manager = managerBackedBy(disk) - let a = manager.registry.descriptor(id: "model-a")! - let b = manager.registry.descriptor(id: "model-b")! - - manager.showStatus(for: a) - XCTAssertEqual(manager.state, .notDownloaded, "the selected model A is not on disk") - manager.showStatus(for: b) - XCTAssertEqual(manager.state, .ready, "selecting B reflects B's own on-disk status") - manager.showStatus(for: a) - XCTAssertEqual(manager.state, .notDownloaded, - "switching back to A shows A's status, not a stale carry-over from B") - } - - func testShowStatusIsNoOpWhileOptedOut() { - let disk = DiskState(["model-b"]) - let manager = managerBackedBy(disk) - manager.setOptedIn(false) - manager.showStatus(for: manager.registry.descriptor(id: "model-b")!) - XCTAssertEqual(manager.state, .notDownloaded, "no status surfaces while AI is opted out") - } - - func testDeleteFromDiskRemovesWeightsAndResetsSelected() { - let disk = DiskState(["model-b"]) - let manager = managerBackedBy(disk) - let b = manager.registry.descriptor(id: "model-b")! - manager.showStatus(for: b) - XCTAssertEqual(manager.state, .ready) - - manager.deleteFromDisk(b) - XCTAssertFalse(disk.contains("model-b"), "delete removes the provisioned weights from disk") - XCTAssertTrue(disk.deleted.contains("model-b"), "the injected provisioner-delete ran for B") - XCTAssertEqual(manager.state, .notDownloaded, "the on-screen model resets to not-downloaded") - } - - func testDeletedModelDoesNotRediscoverAsDownloaded() { - // The reported bug: after deleting, re-opening the AI section / re-enabling must NOT say - // "Downloaded". With the weights actually gone, every re-probe reads not-downloaded. - let disk = DiskState(["model-b"]) - let manager = managerBackedBy(disk) - let b = manager.registry.descriptor(id: "model-b")! - manager.showStatus(for: b) - manager.deleteFromDisk(b) - - manager.showStatus(for: b) // re-open the AI section - XCTAssertEqual(manager.state, .notDownloaded, "a deleted model never re-discovers as Downloaded") - manager.reconcileWithDisk() // launch / opt-in rediscovery path - XCTAssertEqual(manager.state, .notDownloaded) - } - - func testDeleteResidentModelEvictsIt() async throws { - let disk = DiskState(["model-a"]) - let manager = managerBackedBy(disk) - let a = manager.registry.descriptor(id: "model-a")! - _ = try await manager.runtime(requiring: [.text]) // A resident + .loaded - XCTAssertEqual(manager.state, .loaded) - - manager.deleteFromDisk(a) - XCTAssertFalse(manager.isResident, "deleting the resident model unloads it") - XCTAssertEqual(manager.state, .notDownloaded) - XCTAssertFalse(disk.contains("model-a")) - } - - func testDeleteAllFromDiskClearsEveryModelAndResidency() async throws { - let disk = DiskState(["model-a", "model-b"]) - let manager = managerBackedBy(disk) - _ = try await manager.runtime(requiring: [.text]) // load the default (A) resident - XCTAssertTrue(manager.isResident) - - manager.deleteAllFromDisk() - XCTAssertFalse(manager.isResident, "deleting all drops residency") - XCTAssertEqual(manager.state, .notDownloaded) - XCTAssertFalse(disk.contains("model-a")) - XCTAssertFalse(disk.contains("model-b")) - } - - func testDeleteFromDiskBytePathClearsVerifiedWeights() async throws { - // Dev/byte path: no provisioner — deleting clears the held verified weights and resets state. - let payload = Data("good".utf8) - let manager = ModelManager(registry: registry(matching: payload), - downloader: FakeDownloader(payload: payload), - optedIn: true, - storageRoot: tempRoot()) - try await manager.downloadAndVerify(manager.registry.models[0]) - XCTAssertEqual(manager.state, .ready) - - manager.deleteFromDisk(manager.registry.models[0]) - XCTAssertEqual(manager.state, .notDownloaded, "deleting clears the verified weights → not-downloaded") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/ModelRegistryTests.swift b/Tests/ThreeFingerSwitcherTests/ModelRegistryTests.swift deleted file mode 100644 index da7966d..0000000 --- a/Tests/ThreeFingerSwitcherTests/ModelRegistryTests.swift +++ /dev/null @@ -1,78 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for the model registry and capability-based selection (spec: "Model registry and -/// capability-based selection"): a vision command selects a vision-capable model, an audio command -/// routes to the reserved audio model, the default is preferred when it qualifies, and an impossible -/// requirement fails clearly instead of degrading. -final class ModelRegistryTests: XCTestCase { - - private let registry = ModelCatalog.standard - - func testStandardRegistryHasTheThreeKnownEntries() { - let ids = Set(registry.models.map(\.id)) - XCTAssertEqual(ids, ["gemma-4-31b", "gemma-4-26b-a4b", "gemma-4-12b"]) - XCTAssertEqual(registry.defaultModelID, "gemma-4-31b", "31B is the quality-first default") - } - - func testDefaultDescriptorResolves() { - XCTAssertEqual(registry.defaultDescriptor?.id, "gemma-4-31b") - } - - func testTextOnlySelectionPrefersTheDefault() throws { - let chosen = try registry.selectModel(requiring: [.text]) - XCTAssertEqual(chosen.id, "gemma-4-31b", "the default is preferred when it satisfies the need") - } - - func testVisionCommandSelectsVisionCapableModel() throws { - let chosen = try registry.selectModel(requiring: [.vision]) - XCTAssertTrue(chosen.capabilities.contains(.vision)) - XCTAssertEqual(chosen.id, "gemma-4-31b", "the default 31B is vision-capable and preferred") - } - - func testAudioCommandRoutesToReservedAudioModel() throws { - // Only the 12B carries `.audio`, so an audio requirement must route to it even though it is - // not the default — without any feature-code change (capability routing). - let chosen = try registry.selectModel(requiring: [.audio]) - XCTAssertEqual(chosen.id, "gemma-4-12b") - XCTAssertTrue(chosen.capabilities.contains(.audio)) - } - - func testSelectionFailsClearlyWhenNoModelSatisfies() { - // A registry whose only model is text-only cannot serve a vision command. - let textOnly = ModelCatalog( - models: [ModelDescriptor( - id: "text-only", - displayName: "Text Only", - sizeBytes: 1, - integritySHA: "deadbeef", - downloadURL: URL(string: "https://models.invalid/text-only")!, - capabilities: [.text], - quantization: .qat4bit - )], - defaultModelID: "text-only" - ) - XCTAssertThrowsError(try textOnly.selectModel(requiring: [.vision])) { error in - guard case RuntimeError.unavailable = error else { - return XCTFail("expected .unavailable, got \(error)") - } - } - } - - func testSelectionFallsBackToFirstQualifyingWhenDefaultDisqualified() throws { - // Make the default text-only; a vision requirement should then pick the next qualifying entry - // (the 26B-A4B comes before 12B in curated order). - var r = ModelCatalog.standard - r.defaultModelID = "gemma-4-12b" // default is audio-capable but we ask for something it has too - // Ask for vision: 12B qualifies and is the default, so it is preferred. - let chosen = try r.selectModel(requiring: [.vision]) - XCTAssertEqual(chosen.id, "gemma-4-12b") - } - - func testDefaultSwitchIsAOneLineChange() throws { - var r = ModelCatalog.standard - r.defaultModelID = "gemma-4-26b-a4b" // the documented speed alternative - let chosen = try r.selectModel(requiring: [.text]) - XCTAssertEqual(chosen.id, "gemma-4-26b-a4b", "switching the default re-routes selection") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/NotchFlickGestureTests.swift b/Tests/ThreeFingerSwitcherTests/NotchFlickGestureTests.swift deleted file mode 100644 index 304c300..0000000 --- a/Tests/ThreeFingerSwitcherTests/NotchFlickGestureTests.swift +++ /dev/null @@ -1,246 +0,0 @@ -import XCTest -import CoreGraphics -@testable import ThreeFingerSwitcherCore - -/// Tests for `notch-conversation-gestures`: the shared D4 flick classifier (§1), the recognizer's -/// two-finger notch mode with fall-through (§2), and the audit ledger's single removal operation — -/// the user's per-session purge (§3). ParkController-level purge/wiring lives in `ParkedSessionsTests`. -@MainActor -final class NotchFlickGestureTests: XCTestCase { - - // MARK: - Harness - - /// Records canvas + notch resolves and the switcher lifecycle, so routing tests can assert exactly - /// which grammar consumed a synthetic frame stream. - private final class MockDelegate: GestureRecognizerDelegate { - private(set) var canvasResolves: [(dx: Int, dy: Int)] = [] - private(set) var notchResolves: [(dx: Int, dy: Int)] = [] - private(set) var activateCount = 0 - - func gestureDidActivate() { activateCount += 1 } - func gestureDidStep(_ direction: Int) {} - func gestureDidStepRow(_ direction: Int) {} - func gestureDidTriggerMissionControl(up: Bool) {} - func gestureDidCommit() {} - func gestureDidCancel() {} - func launcherCanvasResolve(dx: Int, dy: Int) { canvasResolves.append((dx, dy)) } - func notchConversationResolve(dx: Int, dy: Int) { notchResolves.append((dx, dy)) } - } - - private func makeSettings() -> AppSettings { - let defaults = UserDefaults(suiteName: "ThreeFingerSwitcherTests.\(UUID().uuidString)")! - return AppSettings(defaults: defaults) // defaults: flick 0.8 vel / 0.12s window, axisLockRatio, exactly-three - } - - private func makeRecognizer(_ settings: AppSettings, delegate: MockDelegate) -> GestureRecognizer { - let r = GestureRecognizer(settings: settings) - r.delegate = delegate - return r - } - - private func frame(_ count: Int, x: CGFloat, y: CGFloat, - vx: CGFloat = 0, vy: CGFloat = 0, t: CFTimeInterval) -> TouchFrame { - TouchFrame(testFingerCount: count, centroid: CGPoint(x: x, y: y), - velocity: CGVector(dx: vx, dy: vy), time: t) - } - - // MARK: - 1. FlickExcursionClassifier (the shared D4 math, direct) - - private func classify(_ c: FlickExcursionClassifier) -> (dx: Int, dy: Int)? { - c.classifyOnLift(travelFloor: 0.12, velocityThreshold: 0.8, liftWindow: 0.12, axisLockRatio: 1.4) - } - - func testClassifierFastFlicksClassifyWithTheRightSign() { - // Up, down, right, left — each a fast excursion with a prompt lift. - let cases: [(dx: CGFloat, dy: CGFloat, vx: CGFloat, vy: CGFloat, want: (Int, Int))] = [ - (0, 0.2, 0, 2.0, (0, 1)), // up - (0, -0.2, 0, 2.0, (0, -1)), // down - (0.2, 0, 2.0, 0, (1, 0)), // right - (-0.2, 0, 2.0, 0, (-1, 0)), // left - ] - for c in cases { - var flick = FlickExcursionClassifier() - flick.begin(at: CGPoint(x: 0.5, y: 0.5), time: 0) - flick.track(centroid: CGPoint(x: 0.5 + c.dx, y: 0.5 + c.dy), - velocity: CGVector(dx: c.vx, dy: c.vy), time: 0.05, - velocityThreshold: 0.8, axisLockRatio: 1.4) - let out = classify(flick) - XCTAssertNotNil(out) - XCTAssertEqual(out?.dx, c.want.0) - XCTAssertEqual(out?.dy, c.want.1) - } - } - - func testClassifierSoftScrubIsNil() { - var flick = FlickExcursionClassifier() - flick.begin(at: CGPoint(x: 0.5, y: 0.3), time: 0) - // Plenty of travel, but the peak velocity never crosses the threshold — a reading-scroll. - for i in 1...6 { - flick.track(centroid: CGPoint(x: 0.5, y: 0.3 + CGFloat(i) * 0.05), - velocity: CGVector(dx: 0, dy: 0.3), time: Double(i) * 0.05, - velocityThreshold: 0.8, axisLockRatio: 1.4) - } - XCTAssertNil(classify(flick), "sub-threshold peak velocity is a SCROLL, never a flick") - } - - func testClassifierDeceleratedLiftIsNil() { - var flick = FlickExcursionClassifier() - flick.begin(at: CGPoint(x: 0.5, y: 0.3), time: 0) - // One genuinely fast frame early… - flick.track(centroid: CGPoint(x: 0.5, y: 0.45), velocity: CGVector(dx: 0, dy: 2.0), time: 0.05, - velocityThreshold: 0.8, axisLockRatio: 1.4) - // …then the fingers decelerate and linger well past the lift window before the lift. - flick.track(centroid: CGPoint(x: 0.5, y: 0.5), velocity: CGVector(dx: 0, dy: 0.1), time: 0.4, - velocityThreshold: 0.8, axisLockRatio: 1.4) - XCTAssertNil(classify(flick), "a pause before lifting means a hold/scroll, not a flick") - } - - func testClassifierTravelUnderFloorIsNil() { - var flick = FlickExcursionClassifier() - flick.begin(at: CGPoint(x: 0.5, y: 0.5), time: 0) - // Fast but tiny — a twitch, not an excursion. - flick.track(centroid: CGPoint(x: 0.5, y: 0.55), velocity: CGVector(dx: 0, dy: 2.0), time: 0.03, - velocityThreshold: 0.8, axisLockRatio: 1.4) - XCTAssertNil(classify(flick), "the travel floor gates twitches out") - } - - func testClassifierAxisLockPicksTheDominantAxis() { - var flick = FlickExcursionClassifier() - flick.begin(at: CGPoint(x: 0.5, y: 0.5), time: 0) - // Diagonal but clearly vertical-dominant (dy well over ratio × dx). - flick.track(centroid: CGPoint(x: 0.55, y: 0.75), velocity: CGVector(dx: 0.4, dy: 2.2), time: 0.05, - velocityThreshold: 0.8, axisLockRatio: 1.4) - let out = classify(flick) - XCTAssertEqual(out?.dx, 0) - XCTAssertEqual(out?.dy, 1, "the dominant axis wins; the resolve is axis-locked") - } - - // MARK: - 2. Recognizer notch mode (two-finger only, falls through) - - func testNotchFastUpFlickEmitsExactlyOnce() { - let delegate = MockDelegate() - let r = makeRecognizer(makeSettings(), delegate: delegate) - r.notchConversationActive = true - - r.feed(frame(2, x: 0.5, y: 0.3, t: 0)) // begin - r.feed(frame(2, x: 0.5, y: 0.48, vy: 2.0, t: 0.05)) // fast travel up - r.feed(frame(0, x: 0, y: 0, t: 0.08)) // prompt lift → flick - r.feed(frame(0, x: 0, y: 0, t: 0.1)) // stray re-lift → no-op - - XCTAssertEqual(delegate.notchResolves.count, 1) - XCTAssertEqual(delegate.notchResolves.first?.dx, 0) - XCTAssertEqual(delegate.notchResolves.first?.dy, 1) - XCTAssertEqual(delegate.activateCount, 0, "no switcher involvement") - } - - func testNotchSoftScrubEmitsNothing() { - let delegate = MockDelegate() - let r = makeRecognizer(makeSettings(), delegate: delegate) - r.notchConversationActive = true - - r.feed(frame(2, x: 0.5, y: 0.3, t: 0)) - for i in 1...8 { // slow reading-scroll - r.feed(frame(2, x: 0.5, y: 0.3 + CGFloat(i) * 0.04, vy: 0.3, t: Double(i) * 0.06)) - } - r.feed(frame(0, x: 0, y: 0, t: 0.6)) - - XCTAssertTrue(delegate.notchResolves.isEmpty, "soft scrolling never resolves the conversation") - } - - func testNotchModeThreeFingerSwipeStillLatchesTheSwitcher() { - let delegate = MockDelegate() - let settings = makeSettings() - let r = makeRecognizer(settings, delegate: delegate) - r.notchConversationActive = true - - // A pure three-finger horizontal scrub — the notch mode must not swallow it. - r.feed(frame(3, x: 0.3, y: 0.5, t: 0)) - r.feed(frame(3, x: 0.3 + CGFloat(settings.activationThreshold) + 0.02, y: 0.5, t: 0.05)) - XCTAssertEqual(delegate.activateCount, 1, "the switcher latches exactly as with no conversation open") - XCTAssertTrue(delegate.notchResolves.isEmpty) - } - - func testNotchTwoToThreeMorphFallsThroughToTheSwitcher() { - let delegate = MockDelegate() - let settings = makeSettings() - let r = makeRecognizer(settings, delegate: delegate) - r.notchConversationActive = true - - r.feed(frame(2, x: 0.3, y: 0.5, t: 0)) // notch tracking begins - r.feed(frame(3, x: 0.31, y: 0.5, t: 0.03)) // third finger lands → falls through - r.feed(frame(3, x: 0.31 + CGFloat(settings.activationThreshold) + 0.02, y: 0.5, t: 0.08)) - r.feed(frame(0, x: 0, y: 0, t: 0.12)) - - XCTAssertEqual(delegate.activateCount, 1, "the growing gesture is handed to the switcher latch") - XCTAssertTrue(delegate.notchResolves.isEmpty, "the abandoned flick excursion never emits") - } - - func testCanvasModeTakesPrecedenceOverNotchMode() { - let delegate = MockDelegate() - let r = makeRecognizer(makeSettings(), delegate: delegate) - r.notchConversationActive = true - r.launcherCanvasResolutionActive = true // the foreground modal wins - - r.feed(frame(2, x: 0.5, y: 0.3, t: 0)) - r.feed(frame(2, x: 0.5, y: 0.48, vy: 2.0, t: 0.05)) - r.feed(frame(0, x: 0, y: 0, t: 0.08)) - - XCTAssertEqual(delegate.canvasResolves.count, 1, "the canvas grammar consumed the flick") - XCTAssertTrue(delegate.notchResolves.isEmpty) - } - - // MARK: - 3. Audit purge (the ledger's single removal operation) - - private func record(_ s: AgentSessionID, tool: String, t: Double) -> AuditRecord { - AuditRecord(sessionID: s, tool: tool, policy: .auto, argumentsSummary: "a", - outcome: .done, wasBackground: true, timestamp: Date(timeIntervalSince1970: t)) - } - - func testInMemoryPurgeRemovesOnlyTheTargetSession() { - let log = InMemoryAuditLog(cap: 100) - let a = AgentSessionID(); let b = AgentSessionID() - log.record(record(a, tool: "a1", t: 1)) - log.record(record(b, tool: "b1", t: 2)) - log.record(record(a, tool: "a2", t: 3)) - - log.purge(sessionID: a) - XCTAssertEqual(log.recent(limit: 10).map(\.tool), ["b1"], "only the purged session's records vanish") - } - - func testDiskPurgeRewritesTheFileAndSurvivesRelaunch() async { - let url = FileManager.default.temporaryDirectory - .appendingPathComponent("tfs-audit-purge-\(UUID().uuidString)", isDirectory: true) - .appendingPathComponent("audit.jsonl") - let a = AgentSessionID(); let b = AgentSessionID() - let log = DiskAuditLog(fileURL: url, cap: 100) - log.record(record(a, tool: "a1", t: 1)) - log.record(record(b, tool: "b1", t: 2)) - log.record(record(a, tool: "a2", t: 3)) - - log.purge(sessionID: a) - XCTAssertEqual(log.recent(limit: 10).map(\.tool), ["b1"], "the ring is purged synchronously") - - // The durable rewrite lands on the off-main writer queue — poll a fresh instance over the file. - let deadline = Date().addingTimeInterval(3) - var reopenedTools: [String] = [] - while Date() < deadline { - reopenedTools = DiskAuditLog(fileURL: url, cap: 100).recent(limit: 10).map(\.tool) - if reopenedTools == ["b1"] { break } - try? await Task.sleep(nanoseconds: 20_000_000) - } - XCTAssertEqual(reopenedTools, ["b1"], - "a relaunch (fresh instance over the same file) shows no purged-session record") - } - - func testFailablePurgeKeepsRingPurgedAndSurfacesPersistError() { - let log = FailableInMemoryAuditLog(cap: 100) - let a = AgentSessionID(); let b = AgentSessionID() - log.record(record(a, tool: "a1", t: 1)) - log.record(record(b, tool: "b1", t: 2)) - log.failPersist = true - - log.purge(sessionID: a) - XCTAssertEqual(log.recent(limit: 10).map(\.tool), ["b1"], "the ring is purged even when persist fails") - XCTAssertNotNil(log.lastPersistError, "the rewrite failure surfaces on the bounded channel") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/NotchTuningTests.swift b/Tests/ThreeFingerSwitcherTests/NotchTuningTests.swift deleted file mode 100644 index c56b1a7..0000000 --- a/Tests/ThreeFingerSwitcherTests/NotchTuningTests.swift +++ /dev/null @@ -1,36 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for the notch chat's tuning dial (`notch-timeline-and-tuning` D6): the four ordered stops' -/// (reasoning, context) semantics — reusing `AgentContextPreset`'s token resolution so a stop and a Hub -/// preset with the same name can never disagree — and the discrete slider ↔ stop mapping. -final class NotchTuningTests: XCTestCase { - - func testStopOrderAndReasoningSemantics() { - XCTAssertEqual(NotchTuning.allCases, [.quick, .balanced, .deep, .max], - "the slider scrubs the stops in this order") - XCTAssertFalse(NotchTuning.quick.reasoning, "Quick is the one no-thinking stop") - XCTAssertTrue(NotchTuning.balanced.reasoning) - XCTAssertTrue(NotchTuning.deep.reasoning) - XCTAssertTrue(NotchTuning.max.reasoning) - } - - func testContextTokensFollowThePresetResolutionAndClampToModelMax() { - XCTAssertEqual(NotchTuning.quick.contextTokens(modelMax: 131_072), 8_192) - XCTAssertEqual(NotchTuning.balanced.contextTokens(modelMax: 131_072), 8_192) - XCTAssertEqual(NotchTuning.deep.contextTokens(modelMax: 131_072), 32_768) - XCTAssertEqual(NotchTuning.max.contextTokens(modelMax: 131_072), 131_072, - "Max resolves to the model's architectural maximum") - XCTAssertEqual(NotchTuning.deep.contextTokens(modelMax: 16_000), 16_000, - "every stop clamps to the model max") - } - - func testSliderIndexRoundTripsAndClamps() { - for (index, stop) in NotchTuning.allCases.enumerated() { - XCTAssertEqual(stop.sliderIndex, index) - XCTAssertEqual(NotchTuning.fromSliderIndex(index), stop) - } - XCTAssertEqual(NotchTuning.fromSliderIndex(-1), .quick, "below-range clamps to the first stop") - XCTAssertEqual(NotchTuning.fromSliderIndex(99), .max, "above-range clamps to the last stop") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/OpenWithEntriesTests.swift b/Tests/ThreeFingerSwitcherTests/OpenWithEntriesTests.swift deleted file mode 100644 index e6b5dd7..0000000 --- a/Tests/ThreeFingerSwitcherTests/OpenWithEntriesTests.swift +++ /dev/null @@ -1,26 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for `OpenWithEntries.build` (spec files-band): the external apps that can open the file, wrapped -/// as `.external` rows in the system's order, with the default app flagged. -final class OpenWithEntriesTests: XCTestCase { - - private func external(_ name: String, isDefault: Bool = false) -> OpenWithCandidate { - OpenWithCandidate(app: AppCandidate(url: URL(fileURLWithPath: "/Applications/\(name).app")), - isDefault: isDefault) - } - - func testWrapsExternalsInOrderWithDefaultFlagged() { - let externals = [external("QuickTime", isDefault: true), external("VLC")] - let entries = OpenWithEntries.build(externalApps: externals) - XCTAssertEqual(entries.count, 2) - XCTAssertEqual(entries[0], .external(externals[0])) - XCTAssertEqual(entries[1], .external(externals[1])) - XCTAssertTrue(entries[0].isDefault, "the default external app is flagged") - XCTAssertFalse(entries[1].isDefault) - } - - func testEmptyWhenNoExternalApps() { - XCTAssertTrue(OpenWithEntries.build(externalApps: []).isEmpty) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/PTTArmingModelTests.swift b/Tests/ThreeFingerSwitcherTests/PTTArmingModelTests.swift deleted file mode 100644 index ae72a16..0000000 --- a/Tests/ThreeFingerSwitcherTests/PTTArmingModelTests.swift +++ /dev/null @@ -1,99 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// `voice-double-tap-dwell-trigger`: the double-tap-then-hold trigger grammar. Every spec scenario -/// is a transition path here: tap+tap-hold talks, long single holds are plain modifier use, bare -/// double-taps are no-ops, chords can never trigger voice, and the capture stack is untouched until -/// dwell-elapsed. -final class PTTArmingModelTests: XCTestCase { - - private var model = PTTArmingModel() - - func testDoubleTapAndHoldTalksThenReleaseSends() { - XCTAssertEqual(model.handle(.pttFlagDown), [.schedule(.tapMax)]) - XCTAssertEqual(model.phase, .firstDown) - XCTAssertEqual(model.handle(.pttFlagUp), [.cancelTimer, .schedule(.gap)]) - XCTAssertEqual(model.phase, .awaitingSecond) - XCTAssertEqual(model.handle(.pttFlagDown), [.cancelTimer, .schedule(.dwell)]) - XCTAssertEqual(model.phase, .dwelling) - XCTAssertEqual(model.handle(.timerFired), [.firePTTDown], "capture begins ONLY at dwell-elapsed") - XCTAssertEqual(model.phase, .held) - XCTAssertEqual(model.handle(.pttFlagUp), [.firePTTUp]) - XCTAssertEqual(model.phase, .idle) - } - - func testLongSingleHoldIsPlainModifierUse() { - _ = model.handle(.pttFlagDown) - XCTAssertEqual(model.handle(.timerFired), [], "tap-max elapsed: not a click — stand down") - XCTAssertEqual(model.phase, .inert) - XCTAssertEqual(model.handle(.pttFlagUp), []) - XCTAssertEqual(model.phase, .idle, "release restores idle with no pending state") - } - - func testBareDoubleTapIsANoOp() { - _ = model.handle(.pttFlagDown) - _ = model.handle(.pttFlagUp) - _ = model.handle(.pttFlagDown) - XCTAssertEqual(model.phase, .dwelling) - XCTAssertEqual(model.handle(.pttFlagUp), [.cancelTimer], "released before the dwell: nothing fires") - XCTAssertEqual(model.phase, .idle) - XCTAssertEqual(model.handle(.timerFired), [], "a late dwell timer is inert") - } - - func testGapExpiryReturnsToIdle() { - _ = model.handle(.pttFlagDown) - _ = model.handle(.pttFlagUp) - XCTAssertEqual(model.phase, .awaitingSecond) - XCTAssertEqual(model.handle(.timerFired), [], "no second tap came") - XCTAssertEqual(model.phase, .idle) - } - - func testOptionDeleteChordNeverTriggersAtAnyStage() { - // Chord during the FIRST press (⌥⌫ delete-word — the original regression). - _ = model.handle(.pttFlagDown) - XCTAssertEqual(model.handle(.otherKeyDown), [.cancelTimer]) - XCTAssertEqual(model.phase, .inert) - _ = model.handle(.pttFlagUp) - - // Typing between the taps. - _ = model.handle(.pttFlagDown); _ = model.handle(.pttFlagUp) - XCTAssertEqual(model.handle(.otherKeyDown), [.cancelTimer]) - XCTAssertEqual(model.phase, .idle) - - // Chord during the dwell. - _ = model.handle(.pttFlagDown); _ = model.handle(.pttFlagUp); _ = model.handle(.pttFlagDown) - XCTAssertEqual(model.handle(.otherKeyDown), [.cancelTimer]) - XCTAssertEqual(model.phase, .inert) - _ = model.handle(.pttFlagUp) - XCTAssertEqual(model.phase, .idle) - } - - func testRepeatedChordsStayInertForever() { - for _ in 0..<5 { - _ = model.handle(.pttFlagDown) - _ = model.handle(.otherKeyDown) - _ = model.handle(.pttFlagUp) - } - XCTAssertEqual(model.phase, .idle) - } - - func testKeysWhileGenuinelyTalkingDoNotCancel() { - _ = model.handle(.pttFlagDown); _ = model.handle(.pttFlagUp) - _ = model.handle(.pttFlagDown); _ = model.handle(.timerFired) - XCTAssertEqual(model.phase, .held) - XCTAssertEqual(model.handle(.otherKeyDown), [], "the chord rule exists only pre-capture") - XCTAssertEqual(model.phase, .held) - XCTAssertEqual(model.handle(.pttFlagUp), [.firePTTUp]) - } - - func testExactlyOneTimerPendingPerPhase() { - // Every schedule is preceded (same batch or earlier) by the previous timer's cancellation, - // so the driver's single one-shot slot is always unambiguous. - var actions = model.handle(.pttFlagDown) - XCTAssertEqual(actions.filter { if case .schedule = $0 { return true }; return false }.count, 1) - actions = model.handle(.pttFlagUp) - XCTAssertEqual(actions, [.cancelTimer, .schedule(.gap)]) - actions = model.handle(.pttFlagDown) - XCTAssertEqual(actions, [.cancelTimer, .schedule(.dwell)]) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/ParkedSessionsTests.swift b/Tests/ThreeFingerSwitcherTests/ParkedSessionsTests.swift deleted file mode 100644 index ff77ae6..0000000 --- a/Tests/ThreeFingerSwitcherTests/ParkedSessionsTests.swift +++ /dev/null @@ -1,1295 +0,0 @@ -import XCTest -import SwiftUI -@testable import ThreeFingerSwitcherCore - -/// Tests for the `ai-parked-sessions` slice (tasks §1–§5): the park types (§1), the `ParkScheduler` -/// one-active-now + K-ready seam (§2), the durable store round-trip + `ParkError` mapping (§3), the -/// lifecycle eviction/auto-dismiss/discard (§4 / D1), and the pure overscroll-park + anchor + reveal models (§5). -/// Everything is MLX-free Core, driven against temp dirs + stubs — no model. `@MainActor` because the -/// reveal/anchor models and `AppSettings` keys exercised here are main-actor-friendly. -@MainActor -final class ParkedSessionsTests: XCTestCase { - - private func sid() -> AgentSessionID { AgentSessionID() } - private func tempDir() -> URL { - FileManager.default.temporaryDirectory - .appendingPathComponent("tfs-parked-test-\(UUID().uuidString)", isDirectory: true) - } - private func conversation(_ id: AgentSessionID, _ title: String = "T") -> AgentConversation { - AgentConversation(id: id, title: title, messages: [AgentMessage(role: .user, text: "hi")]) - } - - // MARK: - 1. Park types - - func testParkedSessionCodableRoundTripPreservesIdentity() throws { - let id = sid() - let session = ParkedSession(id: id, title: "Draft", state: .parked, badgeCount: 3, - nextRunAt: Date(timeIntervalSince1970: 1000), - updatedAt: Date(timeIntervalSince1970: 2000)) - let data = try JSONEncoder().encode(session) - let back = try JSONDecoder().decode(ParkedSession.self, from: data) - XCTAssertEqual(session, back) - XCTAssertEqual(back.id, id) // identity stable across encode/decode - } - - func testParkStateProtectionFlags() { - XCTAssertTrue(ParkState.active.isProtectedFromAging) - XCTAssertTrue(ParkState.needsYou.isProtectedFromAging) - XCTAssertFalse(ParkState.parked.isProtectedFromAging) - XCTAssertFalse(ParkState.idle.isProtectedFromAging) - } - - /// `refactor-park-and-background-agents`: the terminal state was RETIRED — a row persisted by an - /// older build under "completed" (or any unknown raw value) decodes as `.idle`, never dropped. - func testRetiredCompletedStateDecodesAsIdle() throws { - let session = ParkedSession(id: sid(), title: "old", state: .parked, badgeCount: 0) - let encoded = String(data: try JSONEncoder().encode(session), encoding: .utf8)! - for legacyRaw in ["completed", "someFutureState"] { - let mutated = encoded.replacingOccurrences(of: "\"parked\"", with: "\"\(legacyRaw)\"") - let back = try JSONDecoder().decode(ParkedSession.self, from: Data(mutated.utf8)) - XCTAssertEqual(back.state, .idle, "a '\(legacyRaw)' row migrates to idle, never fails to load") - } - } - - func testParkErrorYieldsCleanHeadlineNeverRawText() { - for err in [ParkError.storeUnavailable(detail: "ENOSPC raw os text"), - .persistFailed(detail: "raw coding dump"), - .resumeMissing] { - let presented = AIError.message(for: err) - XCTAssertFalse(presented.headline.isEmpty) - XCTAssertFalse(presented.headline.contains("raw")) - XCTAssertFalse(presented.headline.contains("ENOSPC")) - } - // The raw detail is carried in details, not the headline. - let p = AIError.message(for: ParkError.persistFailed(detail: "raw coding dump")) - XCTAssertEqual(p.details, "raw coding dump") - } - - // MARK: - 2. Scheduler (one-active-now + K-ready) - - func testSerialSchedulerReturnsAtMostOneRegardlessOfSlots() { - let now = Date(timeIntervalSince1970: 10_000) - let a = ParkedSession(id: sid(), title: "a", state: .parked, - nextRunAt: now.addingTimeInterval(-30), updatedAt: now.addingTimeInterval(-30)) - let b = ParkedSession(id: sid(), title: "b", state: .parked, - nextRunAt: now.addingTimeInterval(-10), updatedAt: now.addingTimeInterval(-10)) - let sched = SerialParkScheduler(sessions: [b, a]) - // Oldest-waiting first → a, regardless of maxSlots. - XCTAssertEqual(sched.runnableSessions(now: now, maxSlots: 1), [a.id]) - XCTAssertEqual(sched.runnableSessions(now: now, maxSlots: 8), [a.id]) - XCTAssertEqual(sched.runnableSessions(now: now, maxSlots: 0), []) - } - - func testSchedulerExcludesNeedsYouFutureNextRunAtAndDormant() { - let now = Date(timeIntervalSince1970: 10_000) - let needs = ParkedSession(id: sid(), title: "n", state: .needsYou, updatedAt: now) - let future = ParkedSession(id: sid(), title: "f", state: .parked, - nextRunAt: now.addingTimeInterval(60), updatedAt: now) - let active = ParkedSession(id: sid(), title: "act", state: .active, updatedAt: now) - // DORMANT (`refactor-park-and-background-agents`): parked with NO next-run time = blocked on - // the user (a confirm-tier pause) — never runnable. - let dormant = ParkedSession(id: sid(), title: "d", state: .parked, - nextRunAt: nil, updatedAt: now) - let runnable = ParkedSession(id: sid(), title: "r", state: .parked, - nextRunAt: now.addingTimeInterval(-1), updatedAt: now) - let sched = SerialParkScheduler(sessions: [needs, future, active, dormant, runnable]) - XCTAssertEqual(sched.runnableSessions(now: now, maxSlots: 4), [runnable.id]) - } - - func testDidAdvanceTransitions() { - let id = sid() - let sched = SerialParkScheduler(sessions: [ParkedSession(id: id, title: "x", state: .parked)]) - sched.didAdvance(id, result: ToolStepResult(tool: "t", status: .done, summary: "ok")) - let afterDone = sched.snapshot().first { $0.id == id }! - XCTAssertEqual(afterDone.state, .idle, "a settled result idles with an unseen badge — NEVER a terminal state") - XCTAssertEqual(afterDone.badgeCount, 1) - - sched.didAdvance(id, result: ToolStepResult(tool: "t", status: .failed(headline: "Clean headline"), - summary: "x")) - let afterFailure = sched.snapshot().first { $0.id == id }! - XCTAssertEqual(afterFailure.state, .parked) - XCTAssertNotNil(afterFailure.nextRunAt, "a failed advance re-parks with a SCHEDULED retry, never silence") - - sched.didAdvance(id, result: ToolStepResult(tool: "t", status: .awaitingApproval, summary: "x")) - let afterPause = sched.snapshot().first { $0.id == id }! - XCTAssertEqual(afterPause.state, .parked, "a confirm-tier pause waits parked WITHOUT escalating") - XCTAssertNil(afterPause.nextRunAt, "…and goes dormant so the driver can't re-serve the paused turn") - - // An escalated (needs-you) session is never downgraded by a subsequent paused advance. - sched.escalate(id, reason: "dangerous") - sched.didAdvance(id, result: ToolStepResult(tool: "t", status: .awaitingApproval, summary: "x")) - XCTAssertEqual(sched.snapshot().first { $0.id == id }!.state, .needsYou) - } - - func testEscalateSetsNeedsYouAndBadge() { - let id = sid() - let sched = SerialParkScheduler(sessions: [ParkedSession(id: id, title: "x", state: .parked)]) - sched.escalate(id, reason: "dangerous write") - let row = sched.snapshot().first { $0.id == id }! - XCTAssertEqual(row.state, .needsYou) - XCTAssertGreaterThanOrEqual(row.badgeCount, 1) - // An escalated (needs-you) session is no longer runnable. - XCTAssertEqual(sched.runnableSessions(now: Date(), maxSlots: 4), []) - } - - /// The rail lists sessions **most-recently-used first** (`railSnapshot`) so the last-used session sits - /// right after the "+ New chat" card; `snapshot` stays oldest-first (the store/lifecycle order). - func testRailSnapshotOrdersMostRecentlyUsedFirst() { - let base = Date(timeIntervalSince1970: 1_000_000) - let old = ParkedSession(id: sid(), title: "old", state: .idle, updatedAt: base) - let mid = ParkedSession(id: sid(), title: "mid", state: .idle, updatedAt: base.addingTimeInterval(60)) - let recent = ParkedSession(id: sid(), title: "recent", state: .idle, updatedAt: base.addingTimeInterval(120)) - let sched = SerialParkScheduler(sessions: [mid, old, recent]) // insertion order irrelevant - - XCTAssertEqual(sched.railSnapshot().map(\.id), [recent.id, mid.id, old.id]) // newest → oldest - XCTAssertEqual(sched.snapshot().map(\.id), [old.id, mid.id, recent.id]) // lifecycle: oldest → newest - - // Touching a session (bumping updatedAt) floats it to the front of the rail. - sched.escalate(old.id, reason: "x") // escalate bumps updatedAt to now (> base+120) - XCTAssertEqual(sched.railSnapshot().first?.id, old.id) - } - - func testKReadyContractWithConcurrentStub() { - // A drop-in K scheduler reusing the SAME `ParkRunnable.ordered` filter returns up to K with NO - // protocol change — the batching seam contract. - final class ConcurrentParkScheduler: ParkScheduler, @unchecked Sendable { - let rows: [ParkedSession] - init(_ rows: [ParkedSession]) { self.rows = rows } - func runnableSessions(now: Date, maxSlots: Int) -> [AgentSessionID] { - Array(ParkRunnable.ordered(rows, now: now).prefix(max(maxSlots, 0))) - } - func didAdvance(_ id: AgentSessionID, result: ToolStepResult) {} - func escalate(_ id: AgentSessionID, reason: String) {} - } - let now = Date(timeIntervalSince1970: 100) - let rows = (0..<5).map { i in - ParkedSession(id: sid(), title: "\(i)", state: .parked, - nextRunAt: now.addingTimeInterval(Double(-i)), updatedAt: now.addingTimeInterval(Double(i))) - } - let k: ParkScheduler = ConcurrentParkScheduler(rows) - XCTAssertEqual(k.runnableSessions(now: now, maxSlots: 3).count, 3) - XCTAssertEqual(k.runnableSessions(now: now, maxSlots: 1).count, 1) - XCTAssertEqual(k.runnableSessions(now: now, maxSlots: 99).count, 5) - } - - // MARK: - 3. Durable store - - func testDiskStoreRoundTrip() throws { - let store = DiskParkedSessionStore(directory: tempDir()) - let id = sid() - let session = ParkedSession(id: id, title: "Round", state: .parked, badgeCount: 2) - try store.upsert(session, conversation: conversation(id, "Round")) - XCTAssertEqual(store.all().map(\.id), [id]) - XCTAssertEqual(store.conversation(id)?.title, "Round") - try store.remove(id) - XCTAssertTrue(store.all().isEmpty) - XCTAssertNil(store.conversation(id)) - } - - func testDiskStoreRebuildsFromDiskOnRelaunch() throws { - let dir = tempDir() - let id = sid() - do { - let store = DiskParkedSessionStore(directory: dir) - try store.upsert(ParkedSession(id: id, title: "Persist", state: .idle), - conversation: conversation(id, "Persist")) - } - // A fresh store over the SAME dir rebuilds the index (the relaunch path). - let reopened = DiskParkedSessionStore(directory: dir) - XCTAssertEqual(reopened.all().map(\.id), [id]) - XCTAssertEqual(reopened.conversation(id)?.title, "Persist") - } - - func testStoreResumeReadAndPersist() throws { - let store = DiskParkedSessionStore(directory: tempDir()) - let id = sid() - try store.upsert(ParkedSession(id: id, title: "R", state: .idle), conversation: conversation(id)) - XCTAssertNil(store.oneLineResume(id)) - try store.persistResume(id, resume: "Was drafting the email.") - XCTAssertEqual(store.oneLineResume(id), "Was drafting the email.") - } - - func testStoreFailureIsObservableNotSilent() { - let store = InMemoryParkedSessionStore() - store.failWrites = true - let id = sid() - XCTAssertThrowsError(try store.upsert(ParkedSession(id: id, title: "x", state: .parked), - conversation: conversation(id))) { error in - guard let park = error as? ParkError else { return XCTFail("not a ParkError") } - // It maps to the taxonomy and yields a clean headline (never raw / never silent). - XCTAssertEqual(AIError.message(for: park).headline, "That parked session couldn't be saved.") - } - } - - func testResumeMissingThrows() { - let store = InMemoryParkedSessionStore() - XCTAssertThrowsError(try store.persistResume(sid(), resume: "x")) { error in - XCTAssertEqual(error as? ParkError, .resumeMissing) - } - } - - // MARK: - 4. Lifecycle - - func testEvictionPicksLeastRecentlyUpdatedIdleNeverProtected() { - let now = Date(timeIntervalSince1970: 100_000) - let oldIdle = ParkedSession(id: sid(), title: "oldIdle", state: .idle, - updatedAt: now.addingTimeInterval(-100)) - let newIdle = ParkedSession(id: sid(), title: "newIdle", state: .idle, - updatedAt: now.addingTimeInterval(-10)) - let active = ParkedSession(id: sid(), title: "active", state: .active, - updatedAt: now.addingTimeInterval(-1000)) - let needs = ParkedSession(id: sid(), title: "needs", state: .needsYou, - updatedAt: now.addingTimeInterval(-2000)) - let life = ParkLifecycle(maxParked: 3, autoDismissCountdown: 60) - // 4 > 3 → evict the oldest IDLE one (never the older active/needsYou). - XCTAssertEqual(life.evictable([oldIdle, newIdle, active, needs], now: now), oldIdle.id) - } - - func testEvictionReturnsNilWhenUnderCapOrAllProtected() { - let now = Date() - let life = ParkLifecycle(maxParked: 2, autoDismissCountdown: 60) - let two = [ParkedSession(id: sid(), title: "a", state: .idle), - ParkedSession(id: sid(), title: "b", state: .idle)] - XCTAssertNil(life.evictable(two, now: now)) // under cap - // Over cap but ALL non-idle → nothing evicted (a conversation is never force-lost). - let allProtected = [ParkedSession(id: sid(), title: "a", state: .active), - ParkedSession(id: sid(), title: "b", state: .needsYou), - ParkedSession(id: sid(), title: "c", state: .parked)] - XCTAssertNil(life.evictable(allProtected, now: now)) - } - - func testDismissableOnlySeenStaleIdleNeverProtectedOrUnseenOrParked() { - let now = Date(timeIntervalSince1970: 100_000) - // Idle, fully seen, past the countdown → dismissed. - let staleIdle = ParkedSession(id: sid(), title: "stale", state: .idle, - updatedAt: now.addingTimeInterval(-301)) - // Idle under the countdown → kept. - let freshIdle = ParkedSession(id: sid(), title: "fresh", state: .idle, - updatedAt: now.addingTimeInterval(-10)) - // Idle with UNSEEN results → protected indefinitely (`refactor-park-and-background-agents`: - // a docked chat's answer must never be reaped before the user sees it). - let unseenIdle = ParkedSession(id: sid(), title: "unseen", state: .idle, badgeCount: 2, - updatedAt: now.addingTimeInterval(-9999)) - // Parked = pending work (streaming, dormant on an approval, or scheduled) → never expired. - let staleParked = ParkedSession(id: sid(), title: "parked", state: .parked, - updatedAt: now.addingTimeInterval(-9999)) - // Protected: never dismissed even when ancient. - let active = ParkedSession(id: sid(), title: "act", state: .active, - updatedAt: now.addingTimeInterval(-9999)) - let needs = ParkedSession(id: sid(), title: "needs", state: .needsYou, - updatedAt: now.addingTimeInterval(-9999)) - let life = ParkLifecycle(maxParked: 10, autoDismissCountdown: 300) - let dismissable = Set(life.dismissable([staleIdle, freshIdle, unseenIdle, staleParked, active, needs], - now: now)) - XCTAssertEqual(dismissable, Set([staleIdle.id])) - } - - func testZeroCountdownDisablesExpiryEntirely() { - let now = Date(timeIntervalSince1970: 100_000) - let ancient = ParkedSession(id: sid(), title: "ancient", state: .idle, - updatedAt: now.addingTimeInterval(-1_000_000)) - // The DEFAULT: countdown 0 = never — sessions persist until the user deletes them. - let life = ParkLifecycle(maxParked: 10, autoDismissCountdown: 0) - XCTAssertTrue(life.dismissable([ancient], now: now).isEmpty, - "the default countdown of 0 dismisses nothing, ever") - XCTAssertEqual(AppSettings.Defaults.agentParkAutoDismissCountdown, 0, - "expiry ships OFF — deletion is the user's (or the opt-in countdown / eviction)") - } - - func testAutoDismissCountdownRemovesStaleParked() throws { - // 301s old removed, 299s kept (the 300s default boundary). - let now = Date(timeIntervalSince1970: 100_000) - let stale = ParkedSession(id: sid(), title: "stale", state: .idle, - updatedAt: now.addingTimeInterval(-301)) - let fresh = ParkedSession(id: sid(), title: "fresh", state: .idle, - updatedAt: now.addingTimeInterval(-299)) - let life = ParkLifecycle(maxParked: 10, autoDismissCountdown: 300) - XCTAssertEqual(life.dismissable([stale, fresh], now: now), [stale.id]) - } - - func testAutoDismissNeverTouchesProtected() { - let now = Date(timeIntervalSince1970: 100_000) - let active = ParkedSession(id: sid(), title: "act", state: .active, - updatedAt: now.addingTimeInterval(-9999)) - let needs = ParkedSession(id: sid(), title: "needs", state: .needsYou, - updatedAt: now.addingTimeInterval(-9999)) - let life = ParkLifecycle(maxParked: 10, autoDismissCountdown: 300) - XCTAssertTrue(life.dismissable([active, needs], now: now).isEmpty) - } - - func testRunAutoDismissPassUsesDiscardPath() throws { - // The pass routes each dismissable id through the authoritative discard (remove; the CONTROLLER - // owns generation cancellation via the engine — the coordinator's old task table was dead code). - let store = InMemoryParkedSessionStore() - let now = Date(timeIntervalSince1970: 100_000) - let id = sid() - try store.upsert(ParkedSession(id: id, title: "Stale", state: .idle, - updatedAt: now.addingTimeInterval(-301)), - conversation: conversation(id)) - let coord = ParkLifecycleCoordinator(store: store, - lifecycle: ParkLifecycle(maxParked: 10, autoDismissCountdown: 300)) - let dismissed = coord.runAutoDismissPass(now: now) - XCTAssertEqual(dismissed, [id]) - XCTAssertNil(store.conversation(id)) // removed via discard path - XCTAssertTrue(store.all().isEmpty) - } - - func testDiscardRemovesDurablyAndIsNotAFailure() throws { - let store = InMemoryParkedSessionStore() - let id = sid() - try store.upsert(ParkedSession(id: id, title: "D", state: .parked), conversation: conversation(id)) - let coord = ParkLifecycleCoordinator(store: store, - lifecycle: ParkLifecycle(maxParked: 10, autoDismissCountdown: 300)) - try coord.discard(id) - XCTAssertNil(store.conversation(id)) // removed - XCTAssertTrue(store.all().isEmpty) - // No `failed` row was created — discard removed it cleanly (cancellation is not a failure; the - // live engine's cancellation is asserted at the controller level, the single cancellation owner). - } - - // MARK: - 5. Anchor + reveal models (overscroll-park was removed by `notch-native-conversations`: - // sessions are born at the notch; the launcher canvas is one-shot-only and never parks) - - func testAnchorNotchVsTab() { - let visible = CGRect(x: 0, y: 0, width: 1440, height: 900) - let size = CGSize(width: 120, height: 10) - // Notch (safeAreaTop > 0): tucked further below the top than a tab. - let notch = NotchHomeZoneAnchor.zoneRect(size: size, visibleFrame: visible, safeAreaTop: 38) - let tab = NotchHomeZoneAnchor.zoneRect(size: size, visibleFrame: visible, safeAreaTop: 0) - XCTAssertEqual(notch.midX, visible.midX, accuracy: 0.5) // top-center both - XCTAssertEqual(tab.midX, visible.midX, accuracy: 0.5) - XCTAssertLessThan(notch.maxY, tab.maxY) // notch sits lower (more top gap) - XCTAssertLessThanOrEqual(tab.maxY, visible.maxY) // never off the top - } - - func testRailClampsWithinScreenAndNeverResizes() { - let visible = CGRect(x: 0, y: 0, width: 400, height: 900) - let zone = NotchHomeZoneAnchor.zoneRect(size: CGSize(width: 120, height: 10), - visibleFrame: visible, safeAreaTop: 0) - let railSize = CGSize(width: 1200, height: 120) // wider than the screen - let rail = NotchHomeZoneAnchor.railRect(zone: zone, size: railSize, visibleFrame: visible) - // Clamp shifts the origin only — size is preserved (the DockHoverModel.clamp idiom). - XCTAssertEqual(rail.size, railSize) - XCTAssertGreaterThanOrEqual(rail.minX, visible.minX - 0.001) - XCTAssertLessThan(rail.minY, zone.minY) // grows below the zone - } - - /// D5: the rail EMERGES FROM the notch — its TOP edge is FLUSH at the resting zone's TOP edge - /// (`zone.maxY`, the notch / menu-bar lower edge), ZERO gap, growing downward. Regression guard for - /// the old free-floating rect that left a full `notchMargin` (8pt) gap below the zone. - func testRailEmergesFlushFromNotchEdge() { - let visible = CGRect(x: 0, y: 0, width: 1440, height: 900) - let zone = NotchHomeZoneAnchor.zoneRect(size: CGSize(width: 120, height: 10), - visibleFrame: visible, safeAreaTop: 38) - let size = CGSize(width: 600, height: NotchHomeZoneLayout.railHeight) - let rail = NotchHomeZoneAnchor.railRect(zone: zone, size: size, visibleFrame: visible) - // Flush: the rail TOP meets the zone TOP exactly — no 8pt gap. - XCTAssertEqual(rail.maxY, zone.maxY, accuracy: 0.001) - XCTAssertEqual(rail.size, size) // never resized - XCTAssertEqual(rail.midX, zone.midX, accuracy: 0.5) // top-centered on the zone - XCTAssertLessThan(rail.minY, zone.minY) // grows downward - XCTAssertGreaterThanOrEqual(rail.minY, visible.minY - 0.001) // on-screen - } - - /// D5 content-fit: the rail HUGS N session cards + chrome in width (mirroring SwitcherLayout's - /// contentSize), clamps to the screen fraction (overflow → horizontal scroll), and a single/empty - /// rail hugs to the one-card floor. Height fits one band, flagging overflow → vertical scroll only - /// when clamped below the band. - func testRailContentFitHugsSessionsAndClamps() { - let visible = CGRect(x: 0, y: 0, width: 1440, height: 900) - - // Single card → exactly the one-card floor, no overflow. - let one = NotchHomeZoneLayout.solve(count: 1, visibleFrame: visible) - XCTAssertEqual(one.contentSize.width, NotchHomeZoneLayout.oneCardWidth, accuracy: 0.001) - XCTAssertFalse(one.overflowsHorizontally) - - // Three cards → chrome + 3 cards + 2 gaps (hugs, well under the clamp). - let three = NotchHomeZoneLayout.solve(count: 3, visibleFrame: visible) - XCTAssertEqual(three.contentSize.width, NotchHomeZoneLayout.naturalWidth(count: 3), accuracy: 0.001) - XCTAssertGreaterThan(three.contentSize.width, one.contentSize.width) // wider with more cards - XCTAssertFalse(three.overflowsHorizontally) - - // Empty → still the one-card floor (never a zero-width panel). - let empty = NotchHomeZoneLayout.solve(count: 0, visibleFrame: visible) - XCTAssertEqual(empty.contentSize.width, NotchHomeZoneLayout.oneCardWidth, accuracy: 0.001) - - // Many cards exceeding the width fraction → clamped to the fraction + flagged to scroll. - let many = NotchHomeZoneLayout.solve(count: 40, visibleFrame: visible) - let maxW = visible.width * NotchHomeZoneLayout.maxWidthFraction - XCTAssertEqual(many.contentSize.width, maxW, accuracy: 0.001) - XCTAssertTrue(many.overflowsHorizontally) - - // Height hugs one band within a tall screen (no vertical overflow). - XCTAssertEqual(three.contentSize.height, NotchHomeZoneLayout.railHeight, accuracy: 0.001) - XCTAssertFalse(three.overflowsVertically) - - // A very short screen clamps the height below the band → vertical scroll. - let shortScreen = CGRect(x: 0, y: 0, width: 1440, height: 100) - let clampedH = NotchHomeZoneLayout.solve(count: 1, visibleFrame: shortScreen) - XCTAssertLessThan(clampedH.contentSize.height, NotchHomeZoneLayout.railHeight) - XCTAssertTrue(clampedH.overflowsVertically) - } - - /// The dock is NON-scrollable and sizes to HUG every rendered card — the persistent "+ New chat" card - /// PLUS one per session (the `sessions.count + 1` the controller passes) — so it EXPANDS as sessions are - /// added and never scrolls within the parked-session cap. - func testRailHugsNewChatCardAndExpandsWithoutScrolling() { - let visible = CGRect(x: 0, y: 0, width: 1512, height: 949) // this Mac's default visible frame - - // 2 sessions render 3 cards (new-chat + 2): the panel must hug all 3, no scroll. - let two = NotchHomeZoneLayout.solve(count: 2 + 1, visibleFrame: visible) - XCTAssertEqual(two.contentSize.width, NotchHomeZoneLayout.naturalWidth(count: 3), accuracy: 0.001) - XCTAssertFalse(two.overflowsHorizontally) - - // The full parked cap (6 sessions → 7 cards) still hugs without overflowing on a 14". - let full = NotchHomeZoneLayout.solve(count: 6 + 1, visibleFrame: visible) - XCTAssertFalse(full.overflowsHorizontally) - XCTAssertGreaterThan(full.contentSize.width, two.contentSize.width) // expands with more sessions - } - - func testRevealModelLifecycle() { - let model = NotchRevealModel(graceInterval: 0.25) - let zone = CGRect(x: 100, y: 800, width: 120, height: 10) - let rail = CGRect(x: 60, y: 660, width: 200, height: 120) - // In the zone → reveal. - XCTAssertEqual(model.feed(cursor: CGPoint(x: 160, y: 805), zoneRect: zone, railFrame: nil, now: 0), .reveal) - // Traveling onto the rail → keep. - XCTAssertEqual(model.feed(cursor: CGPoint(x: 160, y: 700), zoneRect: zone, railFrame: rail, now: 0.1), .reveal) - // Leave both, within grace → still revealed. - XCTAssertEqual(model.feed(cursor: CGPoint(x: 5, y: 5), zoneRect: zone, railFrame: rail, now: 0.2), .reveal) - // Past grace → dismiss. - XCTAssertEqual(model.feed(cursor: CGPoint(x: 5, y: 5), zoneRect: zone, railFrame: rail, now: 1.0), .dismiss) - // Then idle. - XCTAssertEqual(model.feed(cursor: CGPoint(x: 5, y: 5), zoneRect: zone, railFrame: nil, now: 1.1), .idle) - } - - /// D5 regression guard for "move into the notch dismisses": a cursor in the FORMER GAP between the - /// zone and the container, AND a cursor in the NOTCH BAND ABOVE the zone, both KEEP the rail (`.reveal`, - /// never `.dismiss`) because they fall inside the ONE contiguous live region. The reveal model uses the - /// supplied `liveZone`, not just the zone+rail union with its dead gap. - func testRevealModelContiguousLiveZoneDocksIntoNotch() { - let visible = CGRect(x: 0, y: 0, width: 1440, height: 900) - let zone = NotchHomeZoneAnchor.zoneRect(size: CGSize(width: 120, height: 10), - visibleFrame: visible, safeAreaTop: 38) - let rail = NotchHomeZoneAnchor.railRect( - zone: zone, - size: CGSize(width: 600, height: NotchHomeZoneLayout.railHeight), - visibleFrame: visible) - let live = NotchHomeZoneAnchor.liveZoneRect(zone: zone, rail: rail, visibleFrame: visible) - - // The live region spans contiguously from below the container up past the zone into the notch. - XCTAssertLessThanOrEqual(live.minY, rail.minY + 0.001) // reaches the container bottom - XCTAssertGreaterThan(live.maxY, zone.maxY) // extends UP into the notch pixels - XCTAssertLessThanOrEqual(live.maxY, visible.maxY + 0.001) // but not off the top - - let model = NotchRevealModel(graceInterval: 0.25) - // Reveal by entering the zone. - XCTAssertEqual(model.feed(cursor: CGPoint(x: zone.midX, y: zone.midY), - zoneRect: zone, railFrame: rail, liveZone: live, now: 0), .reveal) - - // A point in the FORMER GAP between zone and container (just below the zone, above old rail top) — - // formerly fell in NEITHER rect and grace-armed → dismiss. Now it's inside the live zone → keep. - let gapPoint = CGPoint(x: zone.midX, y: zone.minY - 4) - XCTAssertFalse(zone.contains(gapPoint)) - XCTAssertTrue(live.contains(gapPoint)) - XCTAssertEqual(model.feed(cursor: gapPoint, zoneRect: zone, railFrame: rail, - liveZone: live, now: 0.1), .reveal) - - // A point in the NOTCH BAND ABOVE the zone — moving UP into the notch must dock, not dismiss. - let notchPoint = CGPoint(x: zone.midX, y: zone.maxY + 4) - XCTAssertFalse(zone.contains(notchPoint)) - XCTAssertTrue(live.contains(notchPoint)) - XCTAssertEqual(model.feed(cursor: notchPoint, zoneRect: zone, railFrame: rail, - liveZone: live, now: 5.0), .reveal) // even well past the grace interval - - // Truly outside the live region, past grace → dismiss (the lifecycle still ends). - let outside = CGPoint(x: 5, y: 5) - XCTAssertFalse(live.contains(outside)) - _ = model.feed(cursor: outside, zoneRect: zone, railFrame: rail, liveZone: live, now: 5.1) - XCTAssertEqual(model.feed(cursor: outside, zoneRect: zone, railFrame: rail, - liveZone: live, now: 6.0), .dismiss) - } - - // MARK: - Attached (notch-merged) mode — change `notch-attached-park-dock` - - /// The notch box is the gap BETWEEN the two aux menu-bar strips, its height `safeAreaTop`, its top edge - /// the physical top (`screenFrame.maxY`). Nil on a notchless/external display (no aux areas / no safe - /// area) so the controller degrades to the honest top-center tab. - func testNotchRectDerivationAndTabDegradation() throws { - let screen = CGRect(x: 0, y: 0, width: 1440, height: 900) - let auxL = CGRect(x: 0, y: 862, width: 620, height: 38) - let auxR = CGRect(x: 820, y: 862, width: 620, height: 38) - let notch = try XCTUnwrap(NotchHomeZoneAnchor.notchRect( - screenFrame: screen, safeAreaTop: 38, auxLeft: auxL, auxRight: auxR)) - XCTAssertEqual(notch, CGRect(x: 620, y: 862, width: 200, height: 38)) - XCTAssertEqual(notch.maxY, screen.maxY) // top edge is the physical top - XCTAssertEqual(notch.midX, screen.midX, accuracy: 0.5) // notch is screen-centered - - // Degradation: no safe area, or missing aux areas, or degenerate overlap → nil (tab mode). - XCTAssertNil(NotchHomeZoneAnchor.notchRect(screenFrame: screen, safeAreaTop: 0, auxLeft: auxL, auxRight: auxR)) - XCTAssertNil(NotchHomeZoneAnchor.notchRect(screenFrame: screen, safeAreaTop: 38, auxLeft: nil, auxRight: auxR)) - XCTAssertNil(NotchHomeZoneAnchor.notchRect(screenFrame: screen, safeAreaTop: 38, - auxLeft: CGRect(x: 0, y: 862, width: 900, height: 38), - auxRight: CGRect(x: 800, y: 862, width: 640, height: 38))) - } - - /// The resting nub hugs the notch: centered on the cutout, its TOP edge FLUSH at the notch's bottom - /// (zero gap), growing downward — the "attached" fix for the old free-floating rect. - func testAttachedNubHugsNotchFlush() { - let screen = CGRect(x: 0, y: 0, width: 1440, height: 900) - let notch = CGRect(x: 620, y: 862, width: 200, height: 38) - let nub = NotchHomeZoneAnchor.attachedNubRect(size: CGSize(width: 120, height: 10), - notch: notch, screenFrame: screen) - XCTAssertEqual(nub.maxY, notch.minY, accuracy: 0.001) // top flush at the notch bottom - XCTAssertEqual(nub.midX, notch.midX, accuracy: 0.5) // centered on the notch - XCTAssertLessThan(nub.minY, notch.minY) // grows downward - } - - /// The merged panel reaches the PHYSICAL top (its black spans the notch band), is centered on the notch, - /// floors its width at the notch + flanks, and stacks the notch band ON TOP of the content height. - func testAttachedPanelMergesIntoNotch() { - let screen = CGRect(x: 0, y: 0, width: 1440, height: 900) - let notch = CGRect(x: 620, y: 862, width: 200, height: 38) - - // Narrow content (one-card floor) → width floored at notch + 2 flanks; panel top at the physical top. - let narrow = NotchHomeZoneAnchor.attachedPanelRect( - contentSize: CGSize(width: NotchHomeZoneLayout.oneCardWidth, height: NotchHomeZoneLayout.railHeight), - notch: notch, screenFrame: screen) - XCTAssertEqual(narrow.maxY, notch.maxY, accuracy: 0.001) // reaches the physical top - XCTAssertEqual(narrow.midX, notch.midX, accuracy: 0.5) // centered on the notch - XCTAssertEqual(narrow.width, notch.width + 2 * NotchHomeZoneAnchor.minNotchFlank, accuracy: 0.001) - XCTAssertEqual(narrow.height, NotchHomeZoneLayout.railHeight + notch.height, accuracy: 0.001) - - // Wide content → the panel hugs the content width (above the notch floor). - let wide = NotchHomeZoneAnchor.attachedPanelRect( - contentSize: CGSize(width: 900, height: NotchHomeZoneLayout.railHeight), - notch: notch, screenFrame: screen) - XCTAssertEqual(wide.width, 900, accuracy: 0.001) - XCTAssertEqual(wide.maxY, notch.maxY, accuracy: 0.001) - } - - /// The attached rail panel is sized so that centering the card row yields SYMMETRIC vertical padding of - /// `notch.height + railNotchClearance` on top and bottom — the cards clear the notch by the small - /// clearance and sit balanced (not shoved to the bottom). - func testAttachedRailCentersCardsWithNotchClearance() { - let screen = CGRect(x: 0, y: 0, width: 1440, height: 900) - let notch = CGRect(x: 620, y: 862, width: 200, height: 38) - let contentH = NotchHomeZoneLayout.attachedRailContentHeight(notchHeight: notch.height) - let panel = NotchHomeZoneAnchor.attachedPanelRect( - contentSize: CGSize(width: NotchHomeZoneLayout.oneCardWidth, height: contentH), - notch: notch, screenFrame: screen) - - // Total panel = card + 2*(notch + clearance). - let expectedTotal = NotchHomeZoneLayout.cardHeight + 2 * (notch.height + NotchHomeZoneLayout.railNotchClearance) - XCTAssertEqual(panel.height, expectedTotal, accuracy: 0.001) - // Centering the card row gives equal top/bottom padding = notch + clearance (symmetric, balanced). - let vPad = (panel.height - NotchHomeZoneLayout.cardHeight) / 2 - XCTAssertEqual(vPad, notch.height + NotchHomeZoneLayout.railNotchClearance, accuracy: 0.001) - // …so the visible gap below the notch is exactly the small clearance (cards clear the notch). - XCTAssertEqual(vPad - notch.height, NotchHomeZoneLayout.railNotchClearance, accuracy: 0.001) - XCTAssertEqual(panel.maxY, notch.maxY, accuracy: 0.001) // still welded to the physical top - } - - /// The attached live zone is one contiguous region spanning the panel, the nub, AND the notch band, so a - /// cursor moving UP into the notch docks (never grace-dismisses) — the attached analogue of `liveZoneRect`. - func testAttachedLiveZoneUnionsNotchAndDocks() { - let screen = CGRect(x: 0, y: 0, width: 1440, height: 900) - let notch = CGRect(x: 620, y: 862, width: 200, height: 38) - let nub = NotchHomeZoneAnchor.attachedNubRect(size: CGSize(width: 120, height: 10), - notch: notch, screenFrame: screen) - let panel = NotchHomeZoneAnchor.attachedPanelRect( - contentSize: CGSize(width: 600, height: NotchHomeZoneLayout.railHeight), - notch: notch, screenFrame: screen) - let live = NotchHomeZoneAnchor.attachedLiveZone(nub: nub, panel: panel, notch: notch) - - XCTAssertGreaterThanOrEqual(live.maxY, notch.maxY - 0.001) // extends up to the physical top - XCTAssertTrue(live.contains(CGPoint(x: notch.midX, y: notch.midY))) // the notch band is inside - XCTAssertTrue(live.contains(CGPoint(x: panel.midX, y: panel.midY))) // the panel is inside - - // The reveal model keeps the panel open for a cursor anywhere in the notch band, past grace. - let model = NotchRevealModel(graceInterval: 0.25) - XCTAssertEqual(model.feed(cursor: CGPoint(x: nub.midX, y: nub.midY), - zoneRect: nub, railFrame: panel, liveZone: live, now: 0), .reveal) - XCTAssertEqual(model.feed(cursor: CGPoint(x: notch.midX, y: notch.midY), - zoneRect: nub, railFrame: panel, liveZone: live, now: 5.0), .reveal) - } - - /// The reveal TRIGGER in attached mode is the physical notch cutout (extended down by the small - /// cross-tolerance), welded to the physical top — so the rail reveals only when the cursor crosses UP - /// behind the notch, never on the resting nub below it. - func testNotchTriggerRectIsTheCutoutBehindTheNotch() { - let notch = CGRect(x: 620, y: 862, width: 200, height: 38) - let trigger = NotchHomeZoneAnchor.notchTriggerRect(notch: notch) - XCTAssertEqual(trigger.maxY, notch.maxY, accuracy: 0.001) // top welded to the physical top - XCTAssertEqual(trigger.midX, notch.midX, accuracy: 0.5) // centered on the notch - XCTAssertEqual(trigger.width, notch.width, accuracy: 0.001) // spans the cutout width - XCTAssertEqual(trigger.minY, notch.minY - NotchHomeZoneAnchor.notchCrossTolerance, accuracy: 0.001) - XCTAssertTrue(trigger.contains(CGPoint(x: notch.midX, y: notch.midY))) // behind the notch → triggers - XCTAssertTrue(trigger.contains(CGPoint(x: notch.midX, y: notch.minY))) // right at the notch edge → triggers - } - - /// The notchless/external reveal trigger is a thin band hugging the physical top edge at top-center, - /// spanning from just under the menu bar up to the physical top — the "slam to the top edge" mimic. - func testTopEdgeTriggerRectHugsPhysicalTop() { - let screen = CGRect(x: 0, y: 0, width: 1440, height: 900) - let visible = CGRect(x: 0, y: 0, width: 1440, height: 876) // 24pt menu bar - let trigger = NotchHomeZoneAnchor.topEdgeTriggerRect(width: 120, visibleFrame: visible, screenFrame: screen) - XCTAssertEqual(trigger.minY, visible.maxY, accuracy: 0.001) // bottom just under the menu bar - XCTAssertEqual(trigger.maxY, screen.maxY, accuracy: 0.001) // top at the physical top edge - XCTAssertEqual(trigger.midX, visible.midX, accuracy: 0.5) // centered - XCTAssertEqual(trigger.width, 120, accuracy: 0.001) - } - - /// The behavioral heart of the change: fed the notch cutout as its trigger, the reveal model reveals - /// ONLY when the cursor is behind the notch — a cursor on the resting nub *below* the notch (while - /// hidden) does NOT reveal (`.idle`); crossing UP into the notch does (`.reveal`). - func testRevealFiresOnlyBehindNotchNotOnNub() { - let screen = CGRect(x: 0, y: 0, width: 1440, height: 900) - let notch = CGRect(x: 620, y: 862, width: 200, height: 38) - let nub = NotchHomeZoneAnchor.attachedNubRect(size: CGSize(width: 120, height: 10), - notch: notch, screenFrame: screen) - let trigger = NotchHomeZoneAnchor.notchTriggerRect(notch: notch) - let live = NotchHomeZoneAnchor.attachedLiveZone(nub: nub, panel: nil, notch: notch) - let model = NotchRevealModel(graceInterval: 0.25) - - // Cursor grazing the nub BELOW the notch while hidden → no reveal (the whole point of the change). - XCTAssertEqual(model.feed(cursor: CGPoint(x: nub.midX, y: nub.midY), - zoneRect: trigger, railFrame: nil, liveZone: live, now: 0), .idle) - // Cursor crosses UP behind the notch → reveal. - XCTAssertEqual(model.feed(cursor: CGPoint(x: notch.midX, y: notch.midY), - zoneRect: trigger, railFrame: nil, liveZone: live, now: 0.1), .reveal) - // Now shown, dropping back down onto the nub keeps it open (the nub is inside the live zone). - XCTAssertEqual(model.feed(cursor: CGPoint(x: nub.midX, y: nub.midY), - zoneRect: trigger, railFrame: nil, liveZone: live, now: 0.2), .reveal) - } - - /// The reveal DWELL (Hub-configurable): with a non-zero dwell the cursor must stay in the trigger - /// CONTINUOUSLY for the dwell before the rail reveals; leaving the trigger cancels an in-progress dwell - /// (a re-entry restarts it); once shown, keep-open is instant (no dwell). - func testRevealDwellDelaysRevealAndCancelsOnLeave() { - let trigger = CGRect(x: 100, y: 800, width: 120, height: 32) - let inside = CGPoint(x: trigger.midX, y: trigger.midY) - let outside = CGPoint(x: 5, y: 5) - - let model = NotchRevealModel(graceInterval: 0.25, dwellInterval: 0.3) - // Enter the trigger → dwelling, nothing shown yet. - XCTAssertEqual(model.feed(cursor: inside, zoneRect: trigger, railFrame: nil, now: 0), .idle) - XCTAssertTrue(model.isDwelling) - XCTAssertEqual(model.state, .hidden) - // Before the dwell elapses → still idle. - XCTAssertEqual(model.feed(cursor: inside, zoneRect: trigger, railFrame: nil, now: 0.2), .idle) - XCTAssertEqual(model.state, .hidden) - // Past the dwell → reveals, dwell cleared. - XCTAssertEqual(model.feed(cursor: inside, zoneRect: trigger, railFrame: nil, now: 0.31), .reveal) - XCTAssertEqual(model.state, .shown) - XCTAssertFalse(model.isDwelling) - - // Leaving the trigger mid-dwell cancels it; a re-entry restarts the countdown from that moment. - let m2 = NotchRevealModel(graceInterval: 0.25, dwellInterval: 0.3) - XCTAssertEqual(m2.feed(cursor: inside, zoneRect: trigger, railFrame: nil, now: 0), .idle) - XCTAssertTrue(m2.isDwelling) - XCTAssertEqual(m2.feed(cursor: outside, zoneRect: trigger, railFrame: nil, now: 0.1), .idle) // left → cancel - XCTAssertFalse(m2.isDwelling) - XCTAssertEqual(m2.feed(cursor: inside, zoneRect: trigger, railFrame: nil, now: 0.2), .idle) // re-enter → restart - XCTAssertEqual(m2.feed(cursor: inside, zoneRect: trigger, railFrame: nil, now: 0.45), .idle) // 0.25 since restart < 0.3 - XCTAssertEqual(m2.feed(cursor: inside, zoneRect: trigger, railFrame: nil, now: 0.51), .reveal) // 0.31 since restart - - // A zero dwell reveals immediately (the model default, preserving pre-dwell behavior). - let instant = NotchRevealModel(graceInterval: 0.25) // dwellInterval defaults to 0 - XCTAssertEqual(instant.feed(cursor: inside, zoneRect: trigger, railFrame: nil, now: 0), .reveal) - } - - /// `reset()` (the swipe-up straight-close hook): after a direct close the model is forced hidden, so the - /// SAME cursor position that was keeping it open no longer reveals — a fresh cross-behind + dwell is - /// required, rather than an immediate re-reveal. - func testResetForcesHiddenSoDirectCloseNeedsAFreshCross() { - let trigger = CGRect(x: 100, y: 800, width: 120, height: 32) - let inside = CGPoint(x: trigger.midX, y: trigger.midY) - let live = trigger.insetBy(dx: -60, dy: -60) // a generous live zone around the trigger - - let model = NotchRevealModel(graceInterval: 0.25, dwellInterval: 0.3) - // Reveal it (cross + dwell elapses). - _ = model.feed(cursor: inside, zoneRect: trigger, railFrame: nil, liveZone: live, now: 0) - XCTAssertEqual(model.feed(cursor: inside, zoneRect: trigger, railFrame: nil, liveZone: live, now: 0.31), .reveal) - XCTAssertEqual(model.state, .shown) - - // Straight close → reset. The model is hidden and holds no dwell/grace. - model.reset() - XCTAssertEqual(model.state, .hidden) - XCTAssertFalse(model.isDwelling) - - // The very same cursor (still inside the trigger) now only STARTS a fresh dwell — no instant reveal. - XCTAssertEqual(model.feed(cursor: inside, zoneRect: trigger, railFrame: nil, liveZone: live, now: 5.0), .idle) - XCTAssertTrue(model.isDwelling) - XCTAssertEqual(model.feed(cursor: inside, zoneRect: trigger, railFrame: nil, liveZone: live, now: 5.31), .reveal) - } - - // MARK: - 5.4 AppSettings keys - - func testAgentParkSettingsDefaultsAndReset() { - let defaults = UserDefaults(suiteName: "tfs-parked-settings-\(UUID().uuidString)")! - let settings = AppSettings(defaults: defaults) - XCTAssertEqual(settings.agentMaxParkedSessions, 6) - XCTAssertEqual(settings.agentParkIdleTimeout, 30 * 60, accuracy: 0.5) - XCTAssertEqual(settings.agentParkAutoDismissCountdown, 0, accuracy: 0.5) // 0 = never (opt-in expiry) - XCTAssertEqual(settings.agentOverscrollParkThreshold, 0.22, accuracy: 0.0001) - XCTAssertEqual(settings.agentNotchRevealDwell, 0.3, accuracy: 0.0001) - // Flick scroll-vs-flick tuning (D4): defaults + reset round-trip. - XCTAssertEqual(settings.flickVelocityThreshold, 0.8, accuracy: 0.0001) - XCTAssertEqual(settings.flickLiftWindow, 0.12, accuracy: 0.0001) - settings.agentMaxParkedSessions = 99 - settings.agentParkIdleTimeout = 5 - settings.agentParkAutoDismissCountdown = 42 - settings.agentOverscrollParkThreshold = 0.5 - settings.agentNotchRevealDwell = 0.75 - settings.flickVelocityThreshold = 1.5 - settings.flickLiftWindow = 0.3 - settings.resetToDefaults() - XCTAssertEqual(settings.agentMaxParkedSessions, 6) - XCTAssertEqual(settings.agentParkIdleTimeout, 30 * 60, accuracy: 0.5) - XCTAssertEqual(settings.agentParkAutoDismissCountdown, 0, accuracy: 0.5) // 0 = never (opt-in expiry) - XCTAssertEqual(settings.agentOverscrollParkThreshold, 0.22, accuracy: 0.0001) - XCTAssertEqual(settings.agentNotchRevealDwell, 0.3, accuracy: 0.0001) - XCTAssertEqual(settings.flickVelocityThreshold, 0.8, accuracy: 0.0001) - XCTAssertEqual(settings.flickLiftWindow, 0.12, accuracy: 0.0001) - } - - // MARK: - 7/8. ParkController glue (`notch-native-conversations`: sessions born at the notch, - // expanded in place, collapsed back to background — no canvas park/restore bridge) - - private final class NullSelection: SelectionProviding { - func readSelectedText() async -> String? { nil } - func readClipboardText() -> String? { nil } - func readClipboardImage() -> Data? { nil } - @discardableResult func replaceSelection(_ text: String) async -> Bool { true } - @discardableResult func pasteAtCursor(_ text: String) async -> Bool { true } - } - private final class FakeDownloader: ModelDownloading, @unchecked Sendable { - let payload: Data; init(payload: Data) { self.payload = payload } - func download(_ d: ModelDescriptor, to dest: URL, progress: @Sendable (Double) -> Void) async throws -> Data { progress(1); return payload } - } - /// A loaded manager whose runtime factory returns `runtime` (for glue tests that RUN turns). - private func loadedManager(_ runtime: LLMRuntime) async throws -> ModelManager { - let payload = Data("w".utf8) - let registry = ModelCatalog(models: [ModelDescriptor(id: "m", displayName: "M", sizeBytes: 1, - integritySHA: ModelManager.sha256Hex(payload), downloadURL: URL(string: "https://x.invalid/m")!, - capabilities: [.text, .vision], quantization: .qat4bit)], defaultModelID: "m") - let m = ModelManager(registry: registry, downloader: FakeDownloader(payload: payload), optedIn: true, - storageRoot: tempDir(), runtimeFactory: { _ in runtime }) - try await m.downloadAndVerify(registry.models[0]); return m - } - /// A cold manager for glue tests that never run a turn (session verbs only). - private func coldManager() -> ModelManager { - let payload = Data("w".utf8) - let registry = ModelCatalog(models: [ModelDescriptor(id: "m", displayName: "M", sizeBytes: 1, - integritySHA: ModelManager.sha256Hex(payload), downloadURL: URL(string: "https://x.invalid/m")!, - capabilities: [.text], quantization: .qat4bit)], defaultModelID: "m") - return ModelManager(registry: registry, downloader: FakeDownloader(payload: payload), optedIn: false, - storageRoot: tempDir(), runtimeFactory: { _ in StubLLMRuntime(interTokenDelayNanos: 0) }) - } - private func makeCtrl(store: ParkedSessionStore, manager: ModelManager? = nil, - audit: AuditLog? = nil) -> ParkController { - let m = manager ?? coldManager() - return ParkController(store: store, maxParked: 6, autoDismissCountdown: 60, - engineFactory: { NotchSessionEngine(modelManager: m, selection: NullSelection()) }, - auditLog: audit) - } - private func waitUntil(_ p: @MainActor () -> Bool, _ timeout: TimeInterval = 3, - file: StaticString = #filePath, line: UInt = #line) async { - let deadline = Date().addingTimeInterval(timeout) - while !p() && Date() < deadline { try? await Task.sleep(nanoseconds: 2_000_000) } - XCTAssertTrue(p(), "condition not met", file: file, line: line) - } - /// Seed a stored session directly (the pre-existing-sessions path a relaunch would produce). - private func seedStored(_ store: ParkedSessionStore, _ id: AgentSessionID, _ title: String = "T", - state: ParkState = .idle, badge: Int = 0) { - try? store.upsert(ParkedSession(id: id, title: title, state: state, badgeCount: badge, - nextRunAt: nil, updatedAt: Date()), - conversation: conversation(id, title)) - } - - func testNewSessionIsUnsavedUntilFirstMessage() { - let store = InMemoryParkedSessionStore() - let ctrl = makeCtrl(store: store) - let id = ctrl.newSession() - // NOT durable at birth: an empty new chat writes nothing to the store and adds no rail row. - XCTAssertNil(store.conversation(id), "an empty new chat is not saved to the dock") - XCTAssertTrue(store.all().isEmpty) - XCTAssertEqual(ctrl.expandedID, id) - XCTAssertNotNil(ctrl.engineForTest(id), "an engine is bound to the newborn (unsaved) session") - XCTAssertEqual(ctrl.engineForTest(id)?.conversation?.id, id) - - // The first message makes it durable + docks it as the foreground .active row (persists even with a - // cold model — the message is saved before the availability gate, then the turn shows unavailable). - ctrl.engineForTest(id)?.send("hello there") - XCTAssertNotNil(store.conversation(id), "the first message persists the session") - XCTAssertEqual(store.all().first?.id, id) - XCTAssertEqual(store.all().first?.state, .active, "the expanded (foreground) row is .active") - XCTAssertEqual(store.all().first?.badgeCount, 0) - } - - func testEmptyNewChatIsDiscardedOnCollapseNotDocked() { - let store = InMemoryParkedSessionStore() - let ctrl = makeCtrl(store: store) - let id = ctrl.newSession() // opened, never messaged - ctrl.collapse() // closed empty - XCTAssertNil(ctrl.expandedID) - XCTAssertNil(store.conversation(id), "an empty new chat is discarded, never saved to the dock") - XCTAssertTrue(store.all().isEmpty, "no rail row is created for an empty chat") - XCTAssertNil(ctrl.engineForTest(id), "its engine is dropped") - } - - func testExpandBindsStoredConversationAndClearsBadge() { - let store = InMemoryParkedSessionStore() - let id = sid() - seedStored(store, id, "Draft", state: .idle, badge: 3) - let ctrl = makeCtrl(store: store) - - ctrl.expand(id) - XCTAssertEqual(ctrl.expandedID, id) - XCTAssertEqual(store.all().first?.state, .active, "an expanded session's row is foreground-active") - XCTAssertEqual(store.all().first?.badgeCount, 0, "expanding clears the unseen-result badge") - XCTAssertEqual(ctrl.engineForTest(id)?.conversation?.id, id, "the engine bound the stored conversation") - XCTAssertNotNil(store.conversation(id), "expand NEVER removes the durable row (sessions never leave the notch)") - } - - func testExpandNeedsYouClearsGlowViaRepublish() { - let store = InMemoryParkedSessionStore() - let id = sid() - seedStored(store, id, "Esc", state: .needsYou, badge: 1) - let ctrl = makeCtrl(store: store) - XCTAssertTrue(ctrl.notch.hasNeedsYouForTest, "seeded needs-you lights the glow") - - ctrl.expand(id) - XCTAssertFalse(ctrl.notch.hasNeedsYouForTest, - "expanding the last needs-you session clears the ambient glow (the addressing begins)") - } - - func testCollapseIdlePersistsAndDropsEngine() { - let store = InMemoryParkedSessionStore() - let ctrl = makeCtrl(store: store) - let id = ctrl.newSession() - ctrl.engineForTest(id)?.send("hi") // a real (non-empty) session — an empty one would be discarded - - ctrl.collapse() - XCTAssertNil(ctrl.expandedID) - XCTAssertEqual(store.all().first?.state, .idle, "a collapsed idle session returns to the background set") - XCTAssertNotNil(store.conversation(id), "collapse persists the conversation") - XCTAssertNil(ctrl.engineForTest(id), "an idle collapsed session holds no engine (recreated on expand)") - } - - func testCollapseMidTurnKeepsTurnRunningThenBadges() async throws { - let stub = StubLLMRuntime(scriptedTokens: Array(repeating: "x", count: 40), - interTokenDelayNanos: 5_000_000) - let manager = try await loadedManager(stub) - let store = InMemoryParkedSessionStore() - let ctrl = makeCtrl(store: store, manager: manager) - let id = ctrl.newSession() - let engine = try XCTUnwrap(ctrl.engineForTest(id)) - - engine.send("U1") - await waitUntil { if case .conversing = engine.state { return true }; return false } - - ctrl.collapse() // the load-bearing contract: does NOT cancel the in-flight turn - XCTAssertNil(ctrl.expandedID) - XCTAssertEqual(store.all().first?.state, .active, - "a collapsed-mid-turn session stays .active (never handed to the background scheduler)") - XCTAssertNotNil(ctrl.engineForTest(id), "the engine stays alive detached until the turn settles") - - // The detached turn settles: conversation persisted with the assistant turn, row idles + badges. - await waitUntil({ store.conversation(id)?.messages.count == 2 }, 5) - XCTAssertEqual(store.all().first?.state, .idle) - XCTAssertEqual(store.all().first?.badgeCount, 1, "the settled background turn is an unseen result") - XCTAssertNil(ctrl.engineForTest(id), "the detached engine is dropped once its turn settled") - XCTAssertFalse(stub.observedCancellation, "collapse never cancelled the turn") - } - - func testExpandAnotherCollapsesCurrentFirst() { - let store = InMemoryParkedSessionStore() - let a = sid(); let b = sid() - seedStored(store, a, "A"); seedStored(store, b, "B") - let ctrl = makeCtrl(store: store) - - ctrl.expand(a) - XCTAssertEqual(ctrl.expandedID, a) - ctrl.expand(b) - XCTAssertEqual(ctrl.expandedID, b, "exactly one session is expanded at a time") - XCTAssertEqual(store.all().first { $0.id == a }?.state, .idle, "the previous session collapsed to idle") - XCTAssertEqual(store.all().first { $0.id == b }?.state, .active) - XCTAssertNil(ctrl.engineForTest(a), "the previous session's idle engine was dropped") - } - - func testDiscardRemovesEverywhereIncludingExpanded() { - let store = InMemoryParkedSessionStore() - let ctrl = makeCtrl(store: store) - let id = ctrl.newSession() - - ctrl.discard(id) - XCTAssertNil(store.conversation(id)) - XCTAssertTrue(store.all().isEmpty) - XCTAssertNil(ctrl.expandedID, "discarding the expanded session collapses the panel") - XCTAssertNil(ctrl.engineForTest(id)) - XCTAssertTrue(ctrl.notch.overlayModelSessionsForTest.isEmpty) // rail view-model cleared - } - - /// The relaunch half of durable-at-first-message: a session that SENT a message (no completed turn) - /// rebuilds from disk; an empty new chat that was closed — or a discarded one — does not. - func testMessagedSessionSurvivesRelaunchButEmptyAndDiscardedDoNot() { - let dir = tempDir() - let born: AgentSessionID - let discarded: AgentSessionID - let neverMessaged: AgentSessionID - do { - let store = DiskParkedSessionStore(directory: dir) - let ctrl = makeCtrl(store: store) - born = ctrl.newSession() - ctrl.engineForTest(born)?.send("keep me") // the first message makes it durable - ctrl.collapse() - neverMessaged = ctrl.newSession() // opened but never sent → discarded on collapse - ctrl.collapse() - discarded = ctrl.newSession() - ctrl.engineForTest(discarded)?.send("bye") - ctrl.discard(discarded) - } - let reopened = DiskParkedSessionStore(directory: dir) - XCTAssertNotNil(reopened.conversation(born), "a session that sent a message survives relaunch") - XCTAssertEqual(reopened.all().map(\.id), [born]) - XCTAssertNil(reopened.conversation(neverMessaged), "an empty new chat is never saved to the dock") - XCTAssertNil(reopened.conversation(discarded), "a deleted session leaves nothing behind") - } - - /// The quick-action firewall (`ai-command-band` delta): a one-shot fire has NO session machinery at - /// all — nothing it does can reach the notch store/scheduler (they are not even connected; the - /// executor lost its conversation/park seams in the revert). The store stays empty across a full fire. - func testQuickActionsNeverTouchTheNotchStore() async throws { - let store = InMemoryParkedSessionStore() - let ctrl = makeCtrl(store: store) - _ = ctrl // the live rail exists; a quick action must never populate it - - let stub = StubLLMRuntime(scriptedTokens: ["fixed"], interTokenDelayNanos: 0) - let manager = try await loadedManager(stub) - final class Sel: SelectionProviding { - func readSelectedText() async -> String? { "teh txt" } - func readClipboardText() -> String? { nil } - func readClipboardImage() -> Data? { nil } - @discardableResult func replaceSelection(_ text: String) async -> Bool { true } - @discardableResult func pasteAtCursor(_ text: String) async -> Bool { true } - } - final class Disp: TaskDispatching { - func prepare(_ kind: TaskKind, resolvedPrompt: String, source: TaskSource, reasoning: Bool) async -> TaskReview { .unavailable(reason: "unused") } - func execute(_ review: TaskReview) async throws {} - } - let executor = AICommandExecutor(modelManager: manager, selection: Sel(), dispatcher: Disp()) - let command = AICommand(name: "Fix", icon: .emoji("✍️"), input: .selection, - promptTemplate: "Fix: {input}", output: .previewOnly) - executor.fire(command) - await waitUntil { if case .ready = executor.state { return true }; return false } - try await executor.commit() - - XCTAssertTrue(store.all().isEmpty, "a full quick-action fire+commit left no notch dock row") - XCTAssertTrue(ctrl.notch.overlayModelSessionsForTest.isEmpty) - } - - /// `refactor-park-and-background-agents`: there is NO terminal advance — a settled `.done` idles - /// with an unseen badge and the session ALWAYS stays (the old `taskComplete: true` flow deleted it - /// on the spot, which is how docked chats vanished the moment their answer landed). - func testDoneAdvanceNeverDismissesTheSession() { - let store = InMemoryParkedSessionStore() - let id = sid() - seedStored(store, id, "Step", state: .parked) - let ctrl = makeCtrl(store: store) - ctrl.didAdvance(id, result: ToolStepResult(tool: "t", status: .done, summary: "ok")) - XCTAssertNotNil(store.conversation(id), "a settled result never removes the session") - XCTAssertEqual(store.all().first?.state, .idle) - XCTAssertGreaterThan(store.all().first?.badgeCount ?? 0, 0, "the result is an unseen badge") - } - - func testParkControllerEscalateSetsNeedsYouForGlow() { - let store = InMemoryParkedSessionStore() - let id = sid() - seedStored(store, id, state: .parked) - let ctrl = makeCtrl(store: store) - ctrl.escalate(id, reason: "dangerous write") - XCTAssertEqual(store.all().first?.state, .needsYou) - // The rail view-model reflects needs-you → the controller lights the ambient glow. - XCTAssertTrue(ctrl.notch.hasNeedsYouForTest) - } - - func testParkControllerSchedulerSeamIsKReady() throws { - let store = InMemoryParkedSessionStore() - let now = Date() - for i in 0..<2 { - let id = sid() - try store.upsert(ParkedSession(id: id, title: "\(i)", state: .parked, - nextRunAt: now.addingTimeInterval(-1), updatedAt: now), - conversation: conversation(id)) - } - let ctrl = makeCtrl(store: store) - // The exposed seam is the SerialParkScheduler: one active now regardless of slots, but the protocol - // shape serves K (the batched runtime fills more) with no change. - XCTAssertEqual(ctrl.parkScheduler.runnableSessions(now: now, maxSlots: 1).count, 1) - XCTAssertEqual(ctrl.parkScheduler.runnableSessions(now: now, maxSlots: 8).count, 1) - } - - // MARK: - 10. `refactor-park-and-background-agents`: dock-mid-response survival, docked approvals, - // relaunch recovery, and the background driver (the ROUTED production path, previously uncovered) - - /// A scripted routing runtime: `structured()` dequeues route JSON; `generate()` streams the answer. - private final class ScriptedRoutingRuntime: LLMRuntime, @unchecked Sendable { - let capabilities: Set = [.text] - private var routes: [String] - private let answerTokens: [String] - private let delayNanos: UInt64 - private let lock = NSLock() - init(routes: [String], answer: [String], delayNanos: UInt64 = 0) { - self.routes = routes; self.answerTokens = answer; self.delayNanos = delayNanos - } - func generate(_ request: LLMRequest) -> AsyncThrowingStream { - let toks = answerTokens, delay = delayNanos - return AsyncThrowingStream { c in - let task = Task { - do { - for (i, t) in toks.enumerated() { - try Task.checkCancellation() - if delay > 0 { try await Task.sleep(nanoseconds: delay) } - c.yield(Token(t, isFinal: i == toks.count - 1)) - } - c.finish() - } catch { - c.finish(throwing: RuntimeError.cancelled) - } - } - c.onTermination = { _ in task.cancel() } - } - } - func structured(_ r: LLMRequest, schema: StructuredSchema, - as type: T.Type) async throws -> StructuredOutcome { - let json: String = { - lock.lock(); defer { lock.unlock() } - return routes.isEmpty ? "{\"tool\":\"\"}" : routes.removeFirst() - }() - return .value(try JSONDecoder().decode(T.self, from: Data(json.utf8))) - } - } - - /// A confirm-tier tool whose run awaits the injected gate (the foreground approval path). - private final class GatedContributor: ToolContributor, @unchecked Sendable { - let descriptor: ToolDescriptor - private(set) var ran = false - init(name: String) { - descriptor = ToolDescriptor(name: name, summary: "s", - argsSchema: StructuredSchema(name: name, json: "{\"type\":\"object\"}"), - writePolicy: .confirm) - } - func descriptors() -> [ToolDescriptor] { [descriptor] } - func canHandle(_ tool: String) -> Bool { tool == descriptor.name } - func run(_ call: RoutedCall, gate: ApprovalGate) async -> ToolStepResult { - switch await gate.awaitDecision(for: .unavailable(reason: "approve me")) { - case .approve: - ran = true - return ToolStepResult(tool: descriptor.name, status: .done, summary: "did it") - case .skip: - return ToolStepResult(tool: descriptor.name, status: .declined(reason: "skipped"), summary: "skipped") - case .cancel: - return ToolStepResult(tool: descriptor.name, - status: .declined(reason: TaskKindToolContributor.cancelledReason), - summary: "cancelled") - } - } - } - - /// A controller whose engines run the ROUTED turn path (registry-wired) — the production shape. - private func makeRoutedCtrl(store: ParkedSessionStore, manager: ModelManager, - registry: ToolRegistry = ToolRegistry([])) -> ParkController { - ParkController(store: store, maxParked: 6, autoDismissCountdown: 60, - engineFactory: { - NotchSessionEngine(modelManager: manager, selection: NullSelection(), - registry: registry, - candidateSource: KeywordToolCandidateSource( - all: { registry.allDescriptors() })) - }) - } - - /// THE regression: docking mid-response of a ROUTED turn (production always routes; every answer - /// used to report terminal → `.completed` → instant auto-dismiss) must leave the chat alive as an - /// unseen result, and no later dismiss pass may reap it while the result is unseen. - func testDockMidRoutedTurnSurvivesSettleAndDismissPass() async throws { - let rt = ScriptedRoutingRuntime(routes: ["{\"tool\":\"\"}"], - answer: Array(repeating: "x", count: 40), - delayNanos: 5_000_000) - let manager = try await loadedManager(rt) - let store = InMemoryParkedSessionStore() - let ctrl = makeRoutedCtrl(store: store, manager: manager) - let id = ctrl.newSession() - let engine = try XCTUnwrap(ctrl.engineForTest(id)) - - engine.send("U1") - await waitUntil { if case .conversing = engine.state { return true }; return false } - - ctrl.collapse() // dock mid-response - await waitUntil({ store.conversation(id)?.messages.count == 2 }, 5) - - XCTAssertNotNil(store.conversation(id), "the docked chat SURVIVES its answer landing") - XCTAssertEqual(store.all().first?.state, .idle) - XCTAssertEqual(store.all().first?.badgeCount, 1, "the answer is an unseen result") - XCTAssertNil(ctrl.engineForTest(id), "the detached engine is dropped once settled") - - _ = ctrl.runAutoDismissPass(now: Date().addingTimeInterval(10_000)) - XCTAssertNotNil(store.conversation(id), - "no dismiss pass ever reaps a chat with an unseen result") - } - - /// Docking while the routing loop is PAUSED at an approval keeps the suspended step alive as - /// needs-you; re-expanding re-presents the SAME step and approve resumes it (the old behavior - /// dropped the engine — orphaning the continuation — and let the idle countdown delete the chat). - func testDockAtApprovalSurfacesNeedsYouAndApproveAfterExpandResumes() async throws { - let tool = GatedContributor(name: "confirm_tool") - let rt = ScriptedRoutingRuntime(routes: ["{\"tool\":\"confirm_tool\"}", "{\"tool\":\"\"}"], - answer: ["done"]) - let manager = try await loadedManager(rt) - let store = InMemoryParkedSessionStore() - let ctrl = makeRoutedCtrl(store: store, manager: manager, registry: ToolRegistry([tool])) - let id = ctrl.newSession() - let engine = try XCTUnwrap(ctrl.engineForTest(id)) - - engine.send("go") - await waitUntil { engine.isPausedAtApproval } - - ctrl.collapse() // dock while paused at the gate - XCTAssertEqual(store.all().first?.state, .needsYou, - "a docked pending approval surfaces needs-you (protected, badged)") - XCTAssertNotNil(ctrl.engineForTest(id), "the engine — and its suspended gate — survives the dock") - - _ = ctrl.runAutoDismissPass(now: Date().addingTimeInterval(10_000)) - XCTAssertNotNil(store.conversation(id), "a needs-you session is never expired") - - ctrl.expand(id) - let reused = try XCTUnwrap(ctrl.engineForTest(id)) - XCTAssertTrue(reused === engine, "expand reuses the paused engine — the turn is never restarted") - XCTAssertTrue(reused.isPausedAtApproval, "the SAME approval re-presents") - XCTAssertTrue(reused.approve()) - await waitUntil({ store.conversation(id)?.messages.count == 2 }, 5) - XCTAssertTrue(tool.ran, "the approved step fired after the dock/expand round-trip") - } - - /// Relaunch normalization + the driver: a row persisted `.active` (quit mid-turn) whose - /// conversation awaits a reply becomes parked + scheduled, and the advance pass re-runs the - /// interrupted turn to an unseen result (the old behavior stranded it as a protected zombie). - func testRelaunchNormalizesStaleActiveAndDriverRecoversTheTurn() async throws { - let store = InMemoryParkedSessionStore() - let id = sid() - try store.upsert(ParkedSession(id: id, title: "Interrupted", state: .active, updatedAt: Date()), - conversation: conversation(id, "Interrupted")) // ends on a user message - let rt = ScriptedRoutingRuntime(routes: ["{\"tool\":\"\"}"], answer: ["recovered"]) - let manager = try await loadedManager(rt) - let ctrl = makeRoutedCtrl(store: store, manager: manager) - - let normalized = try XCTUnwrap(store.all().first { $0.id == id }) - XCTAssertEqual(normalized.state, .parked, "a stale .active row normalizes to parked at launch") - XCTAssertNotNil(normalized.nextRunAt, "…scheduled to run now, so the driver re-runs the turn") - - XCTAssertEqual(ctrl.runAdvancePass(now: Date().addingTimeInterval(1)), [id]) - await waitUntil({ store.conversation(id)?.messages.count == 2 }, 5) - XCTAssertEqual(store.conversation(id)?.messages.last?.text, "recovered") - XCTAssertEqual(store.all().first { $0.id == id }?.state, .idle) - XCTAssertEqual(store.all().first { $0.id == id }?.badgeCount, 1, - "the recovered answer lands as an unseen result") - } - - func testRelaunchNormalizesStaleActiveWithNothingPendingToIdle() throws { - let store = InMemoryParkedSessionStore() - let id = sid() - let now = Date() - let answered = AgentConversation( - id: id, title: "Done", - messages: [AgentMessage(role: .user, text: "U", createdAt: now), - AgentMessage(role: .assistant, text: "A", createdAt: now)]) - try store.upsert(ParkedSession(id: id, title: "Done", state: .active, updatedAt: now), - conversation: answered) - _ = makeCtrl(store: store) - XCTAssertEqual(store.all().first?.state, .idle, "nothing pends → plain idle, not a zombie") - XCTAssertNil(store.all().first?.nextRunAt) - } - - /// The driver serves ONE session and never double-serves: a tick while a turn is generating - /// serves nothing (the in-flight engine owns the single slot). - func testDriverServesOneAndNeverDoubleServes() async throws { - let store = InMemoryParkedSessionStore() - let now = Date() - for (i, id) in [sid(), sid()].enumerated() { - try store.upsert(ParkedSession(id: id, title: "\(i)", state: .parked, - nextRunAt: now.addingTimeInterval(Double(-10 + i)), - updatedAt: now.addingTimeInterval(Double(-20 + i))), - conversation: conversation(id)) - } - let rt = ScriptedRoutingRuntime(routes: ["{\"tool\":\"\"}", "{\"tool\":\"\"}"], - answer: Array(repeating: "x", count: 60), - delayNanos: 5_000_000) - let manager = try await loadedManager(rt) - let ctrl = makeRoutedCtrl(store: store, manager: manager) - - XCTAssertEqual(ctrl.runAdvancePass(now: now).count, 1, "one slot — one session") - XCTAssertTrue(ctrl.runAdvancePass(now: now).isEmpty, - "a tick while a turn is generating serves nothing (no double-serve)") - } - - // MARK: - 9. Purge-delete + the expanded-state choke point (`notch-conversation-gestures`) - - private func auditRecord(_ s: AgentSessionID, tool: String) -> AuditRecord { - AuditRecord(sessionID: s, tool: tool, policy: .auto, argumentsSummary: "a", - outcome: .done, wasBackground: true, timestamp: Date(timeIntervalSince1970: 1)) - } - - func testPurgeRemovesStoreEngineAndAuditWhilePlainDiscardKeepsLedger() { - let store = InMemoryParkedSessionStore() - let audit = InMemoryAuditLog(cap: 100) - let ctrl = makeCtrl(store: store, audit: audit) - - let purged = ctrl.newSession() - ctrl.collapse() - let kept = ctrl.newSession() - ctrl.collapse() - audit.record(auditRecord(purged, tool: "p1")) - audit.record(auditRecord(kept, tool: "k1")) - audit.record(auditRecord(purged, tool: "p2")) - - ctrl.expand(purged) - ctrl.purge(purged) - // Everything about the purged session is gone: row, conversation, engine, audit records. - XCTAssertNil(store.conversation(purged)) - XCTAssertNil(ctrl.engineForTest(purged)) - XCTAssertNil(ctrl.expandedID, "purging the expanded session collapses the panel") - XCTAssertEqual(audit.recent(limit: 10).map(\.tool), ["k1"], - "only the purged session's ledger entries vanish") - - // A PLAIN discard on the sibling keeps its ledger entries (the purge is the single carve-out). - ctrl.discard(kept) - XCTAssertNil(store.conversation(kept)) - XCTAssertEqual(audit.recent(limit: 10).map(\.tool), ["k1"], - "a plain delete leaves the audit ledger intact") - } - - func testOnExpandedChangedFiresOnTransitionsWithoutBlips() { - let store = InMemoryParkedSessionStore() - let a = sid(); let b = sid() - seedStored(store, a, "A"); seedStored(store, b, "B") - let ctrl = makeCtrl(store: store) - var published: [Bool] = [] - ctrl.onExpandedChanged = { published.append($0) } - - ctrl.expand(a) - XCTAssertEqual(published, [true]) - ctrl.expand(b) - XCTAssertEqual(published, [true], "expand-another stays expanded — no false blip") - ctrl.collapse() - XCTAssertEqual(published, [true, false]) - _ = ctrl.newSession() - XCTAssertEqual(published, [true, false, true], "a new session expands") - ctrl.discard(ctrl.expandedID!) - XCTAssertEqual(published, [true, false, true, false], "discarding the expanded session publishes false") - ctrl.expand(a) - XCTAssertEqual(published, [true, false, true, false, true]) - ctrl.setEnabled(false) - XCTAssertEqual(published, [true, false, true, false, true, false], "feature-off collapses and publishes") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/PromptTemplateTests.swift b/Tests/ThreeFingerSwitcherTests/PromptTemplateTests.swift deleted file mode 100644 index 32226b3..0000000 --- a/Tests/ThreeFingerSwitcherTests/PromptTemplateTests.swift +++ /dev/null @@ -1,87 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for prompt template resolution (spec: "Prompt template token resolution"): {input}/{app}/ -/// {url} substitution, missing-token degradation to empty (never fail), and unknown-token passthrough. -final class PromptTemplateTests: XCTestCase { - - func testInputTokenIsSubstituted() { - let context = FireContext(inputText: "teh quick brown fox") - let resolved = PromptTemplate.resolve("Fix the grammar:\n{input}", with: context) - XCTAssertEqual(resolved, "Fix the grammar:\nteh quick brown fox") - } - - func testAppAndUrlTokensSubstitute() { - let context = FireContext(capturedAppName: "Safari", - inputText: "x", - url: URL(string: "https://example.com/page")) - let resolved = PromptTemplate.resolve("In {app} at {url}: {input}", with: context) - XCTAssertEqual(resolved, "In Safari at https://example.com/page: x") - } - - func testMissingUrlAndAppDegradeToEmptyString() { - // No app name, no URL exposed → both resolve to "" and the command still runs (spec). - let context = FireContext(capturedAppName: nil, inputText: "body", url: nil) - let resolved = PromptTemplate.resolve("[{app}|{url}] {input}", with: context) - XCTAssertEqual(resolved, "[|] body", "missing {app}/{url} degrade to empty, never fail") - } - - func testEmptyInputDegradesToEmptyString() { - let context = FireContext(inputText: nil) - let resolved = PromptTemplate.resolve("Q: {input}", with: context) - XCTAssertEqual(resolved, "Q: ") - } - - func testUnknownTokenIsLeftUntouched() { - let context = FireContext(inputText: "hi") - let resolved = PromptTemplate.resolve("{input} {unknown} {foo}", with: context) - XCTAssertEqual(resolved, "hi {unknown} {foo}", - "unknown tokens are passed through verbatim, not dropped") - } - - func testUnknownTokenBeforeKnownTokenStillResolvesKnown() { - let context = FireContext(inputText: "BODY") - let resolved = PromptTemplate.resolve("{mystery} then {input}", with: context) - XCTAssertEqual(resolved, "{mystery} then BODY") - } - - func testDateTokenIsSubstituted() { - // Pin a known date and a fixed format so the assertion is deterministic. - let date = Date(timeIntervalSince1970: 1_700_000_000) // 2023-11-14 - let context = FireContext(date: date) - let resolved = PromptTemplate.resolve("Today: {date}", with: context) - - let expected = DateFormatter.localizedString(from: date, dateStyle: .medium, timeStyle: .short) - XCTAssertEqual(resolved, "Today: \(expected)") - } - - func testSubstitutedValueIsNotReinterpretedAsToken() { - // If {input} contains text that looks like a token, it must NOT be re-substituted. - let context = FireContext(capturedAppName: "Mail", inputText: "see {app}") - let resolved = PromptTemplate.resolve("{input}", with: context) - XCTAssertEqual(resolved, "see {app}", - "a substituted value is emitted literally, not re-scanned for tokens") - } - - func testLoneBraceIsPreserved() { - let context = FireContext(inputText: "x") - let resolved = PromptTemplate.resolve("a { b {input}", with: context) - XCTAssertEqual(resolved, "a { b x") - } - - // MARK: - {lang} runtime-parameter token (spec: "Language token resolves to the active language") - - func testLangTokenSubstitutesActiveLanguage() { - let context = FireContext(inputText: "hello") - let resolved = PromptTemplate.resolve("Translate to {lang}:\n{input}", with: context, - activeLanguage: "Hebrew") - XCTAssertEqual(resolved, "Translate to Hebrew:\nhello") - } - - func testLangTokenWithoutActiveLanguageDegradesToEmpty() { - // A command with no language parameter passes activeLanguage == nil → {lang} resolves empty. - let context = FireContext(inputText: "hello") - let resolved = PromptTemplate.resolve("[{lang}] {input}", with: context) - XCTAssertEqual(resolved, "[] hello", "{lang} with no active language degrades to empty, never fails") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/RegionPickerModelTests.swift b/Tests/ThreeFingerSwitcherTests/RegionPickerModelTests.swift deleted file mode 100644 index 21824bd..0000000 --- a/Tests/ThreeFingerSwitcherTests/RegionPickerModelTests.swift +++ /dev/null @@ -1,68 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// The pure picker brain (spec `screen-region-picker`): the drag → rectangle geometry and the -/// click-without-drag → cancel verdict, verified headless (no overlay, no ScreenCaptureKit). -final class RegionPickerModelTests: XCTestCase { - - func testDragCommitsNormalizedRegion() { - var model = RegionPickerModel() - model.begin(at: CGPoint(x: 100, y: 100)) - model.drag(to: CGPoint(x: 300, y: 250)) - let resolution = model.end(at: CGPoint(x: 300, y: 250)) - XCTAssertEqual(resolution, .region(CGRect(x: 100, y: 100, width: 200, height: 150))) - } - - func testDragInAnyDirectionNormalizesToPositiveRect() { - // Drag up-and-left: origin is the bottom-right of the resulting rect. - var model = RegionPickerModel() - model.begin(at: CGPoint(x: 300, y: 250)) - let resolution = model.end(at: CGPoint(x: 100, y: 100)) - XCTAssertEqual(resolution, .region(CGRect(x: 100, y: 100, width: 200, height: 150))) - } - - func testClickWithoutDraggingCancels() { - var model = RegionPickerModel() - model.begin(at: CGPoint(x: 200, y: 200)) - // Release essentially where we pressed (a click) → cancel, capture nothing. - let resolution = model.end(at: CGPoint(x: 201, y: 201)) - XCTAssertEqual(resolution, .cancel) - } - - func testJustBelowThresholdCancelsJustAboveCaptures() { - // Just below the click/drag threshold → cancel. - var below = RegionPickerModel() - below.begin(at: .zero) - XCTAssertEqual(below.end(at: CGPoint(x: RegionPickerModel.minDragDistance - 1, y: 0)), .cancel) - - // Just above the threshold → a (tiny) region, not a cancel. - var above = RegionPickerModel() - above.begin(at: .zero) - if case .region = above.end(at: CGPoint(x: RegionPickerModel.minDragDistance + 1, y: 0)) { - // expected - } else { - XCTFail("a drag past the threshold captures a region") - } - } - - func testReleaseWithoutBeginIsCancel() { - var model = RegionPickerModel() - XCTAssertEqual(model.end(at: CGPoint(x: 10, y: 10)), .cancel, "a release with no drag in progress cancels") - } - - func testLiveRectTracksDragAndClearsAfterResolve() { - var model = RegionPickerModel() - XCTAssertNil(model.liveRect, "no rect before a drag") - model.begin(at: CGPoint(x: 10, y: 10)) - model.drag(to: CGPoint(x: 40, y: 60)) - XCTAssertEqual(model.liveRect, CGRect(x: 10, y: 10, width: 30, height: 50)) - _ = model.end(at: CGPoint(x: 40, y: 60)) - XCTAssertNil(model.liveRect, "the live rect clears once the pick resolves") - } - - func testDragIsIgnoredWithoutABegin() { - var model = RegionPickerModel() - model.drag(to: CGPoint(x: 99, y: 99)) // no begin → ignored - XCTAssertNil(model.liveRect) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/ResidencyPlannerTests.swift b/Tests/ThreeFingerSwitcherTests/ResidencyPlannerTests.swift deleted file mode 100644 index 035fca8..0000000 --- a/Tests/ThreeFingerSwitcherTests/ResidencyPlannerTests.swift +++ /dev/null @@ -1,126 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// The pure residency / eviction MATH (tasks 3.1–3.4, design D3). No Metal, fixed injected free-memory. -final class ResidencyPlannerTests: XCTestCase { - - private let GB: UInt64 = 1024 * 1024 * 1024 - - private func descriptor(_ id: String, role: ModelRole, lane: ComputeLane?, gb: UInt64, - provider: ModelProvider = .onDevice) -> ModelDescriptor { - ModelDescriptor(id: id, displayName: id, sizeBytes: Int64(gb * GB), integritySHA: "x", - downloadURL: URL(string: "https://example.com/\(id)")!, - capabilities: [.text], quantization: .qat4bit, - role: role, lane: lane, provider: provider, residencyBytes: gb * GB) - } - - /// The standard fleet shapes used across cases. - private var fleet: [ModelDescriptor] { - [ - descriptor("chat", role: .chat, lane: .gpu, gb: 17), - descriptor("ternary", role: .ternaryChat, lane: .cpuTernary, gb: 1), - descriptor("imageQ4", role: .image, lane: .gpu, gb: 7), - descriptor("imageFP16", role: .image, lane: .gpu, gb: 24), - descriptor("video", role: .video, lane: .gpu, gb: 24), - descriptor("cloud", role: .cloudEscalation, lane: nil, gb: 0, provider: .cloud) - ] - } - - private let budget: UInt64 = 48 * 1024 * 1024 * 1024 - - // MARK: - 3.2 Co-residency - - /// Q4 image co-resides with chat + ternary + KV under 48 GB (no eviction). - func testQ4ImageCoResidesWithChatTernaryKV() { - let planner = ResidencyPlanner(kvReserveBytes: 4 * GB, fp16ImageThresholdBytes: 16 * GB) - let plan = planner.plan(target: "imageQ4", descriptors: fleet, - budgetBytes: budget, freeBytes: budget, - currentlyResident: ["chat", "ternary"]) - XCTAssertEqual(plan.admit, ["imageQ4"]) - XCTAssertTrue(plan.evict.isEmpty, "Q4 image must co-reside, not evict") - XCTAssertFalse(plan.infeasible) - XCTAssertEqual(Set(plan.coResident), ["chat", "ternary", "imageQ4"]) - // 17 + 1 + 7 + 4(KV) = 29 GB ≤ 48 GB. - } - - // MARK: - 3.3 Eviction trigger - - func testVideoEvictsChatTernarySurvives() { - let planner = ResidencyPlanner(kvReserveBytes: 4 * GB) - let plan = planner.plan(target: "video", descriptors: fleet, - budgetBytes: budget, freeBytes: budget, - currentlyResident: ["chat", "ternary"]) - XCTAssertEqual(plan.admit, ["video"]) - XCTAssertTrue(plan.evict.contains("chat"), "video must evict the GPU-lane chat") - XCTAssertFalse(plan.evict.contains("ternary"), "the CPU-lane ternary survives a GPU gen") - XCTAssertTrue(plan.coResident.contains("ternary")) - XCTAssertTrue(plan.coResident.contains("video")) - XCTAssertFalse(plan.coResident.contains("chat")) - XCTAssertFalse(plan.infeasible) - // 17(chat) + 1 + 24(video) + 4 = 46 ≤ 48, but the planner still evicts chat because chat + video - // + KV (17 + 24 + 4 = 45) already needs chat gone for headroom past KV... assert the documented - // behavior: chat is evicted so the heavy gen owns the GPU budget. - } - - func testFP16ImageEvictsChat() { - let planner = ResidencyPlanner(kvReserveBytes: 4 * GB) - let plan = planner.plan(target: "imageFP16", descriptors: fleet, - budgetBytes: budget, freeBytes: budget, - currentlyResident: ["chat", "ternary"]) - XCTAssertEqual(plan.admit, ["imageFP16"]) - XCTAssertTrue(plan.evict.contains("chat")) - XCTAssertFalse(plan.evict.contains("ternary")) - XCTAssertFalse(plan.infeasible) - } - - // MARK: - 3.4 Cloud + infeasible - - func testCloudTargetEmptyPlan() { - let planner = ResidencyPlanner() - let plan = planner.plan(target: "cloud", descriptors: fleet, - budgetBytes: budget, freeBytes: budget, - currentlyResident: ["chat", "ternary"]) - XCTAssertTrue(plan.admit.isEmpty) - XCTAssertTrue(plan.evict.isEmpty) - XCTAssertFalse(plan.infeasible) - XCTAssertEqual(Set(plan.coResident), ["chat", "ternary"]) - } - - /// A target larger than the whole budget cannot fit even after evicting everything → infeasible. - func testOverBudgetReportsInfeasible() { - let huge = descriptor("huge", role: .video, lane: .gpu, gb: 60) // > 48 GB on its own - let planner = ResidencyPlanner(kvReserveBytes: 4 * GB) - let plan = planner.plan(target: "huge", - descriptors: fleet + [huge], - budgetBytes: budget, freeBytes: budget, - currentlyResident: ["chat", "ternary"]) - XCTAssertTrue(plan.infeasible) - XCTAssertTrue(plan.admit.isEmpty, "an infeasible target is never admitted") - } - - /// Already-resident target → warm, nothing to do. - func testAlreadyResidentWarm() { - let planner = ResidencyPlanner() - let plan = planner.plan(target: "chat", descriptors: fleet, - budgetBytes: budget, freeBytes: budget, - currentlyResident: ["chat"]) - XCTAssertTrue(plan.admit.isEmpty) - XCTAssertTrue(plan.evict.isEmpty) - } - - // MARK: - 7.2 admissionEvictsChat (plan → warning) - - func testAdmissionEvictsChatMapping() { - let planner = ResidencyPlanner(kvReserveBytes: 4 * GB) - XCTAssertTrue(planner.admissionEvictsChat(targetID: "video", descriptors: fleet, - budgetBytes: budget, freeBytes: budget, - currentlyResident: ["chat", "ternary"])) - XCTAssertTrue(planner.admissionEvictsChat(targetID: "imageFP16", descriptors: fleet, - budgetBytes: budget, freeBytes: budget, - currentlyResident: ["chat", "ternary"])) - XCTAssertFalse(planner.admissionEvictsChat(targetID: "imageQ4", descriptors: fleet, - budgetBytes: budget, freeBytes: budget, - currentlyResident: ["chat", "ternary"]), - "Q4 image co-resides → no evict-chat warning") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/ScrollConsumeRuleTests.swift b/Tests/ThreeFingerSwitcherTests/ScrollConsumeRuleTests.swift index b288b0a..d42e8c1 100644 --- a/Tests/ThreeFingerSwitcherTests/ScrollConsumeRuleTests.swift +++ b/Tests/ThreeFingerSwitcherTests/ScrollConsumeRuleTests.swift @@ -3,55 +3,31 @@ import XCTest /// The scroll tap's consume decision is a pure helper so it can be asserted without standing up a /// `CGEventTap`. The overlay-open clauses are what let two-finger navigation be captured; with both -/// overlays closed the rule must fall back to `≥3` fingers so normal two-finger scrolling passes. -/// While the AI canvas is active the launcher / switcher clauses relax for 1-2 finger scroll so it -/// reaches the canvas's ScrollView, but `≥3` fingers stays consumed (gesture territory). +/// overlays closed the rule must fall back to `>=3` fingers so normal two-finger scrolling passes. @MainActor final class ScrollConsumeRuleTests: XCTestCase { func test_consumes_atTwoFingers_whenLauncherOpen() { - XCTAssertTrue(AppCoordinator.shouldConsumeScroll(fingerCount: 2, launcherOpen: true, switcherOpen: false, canvasActive: false)) + XCTAssertTrue(AppCoordinator.shouldConsumeScroll(fingerCount: 2, launcherOpen: true, switcherOpen: false)) } func test_consumes_atTwoFingers_whenSwitcherOpen() { - XCTAssertTrue(AppCoordinator.shouldConsumeScroll(fingerCount: 2, launcherOpen: false, switcherOpen: true, canvasActive: false)) + XCTAssertTrue(AppCoordinator.shouldConsumeScroll(fingerCount: 2, launcherOpen: false, switcherOpen: true)) } func test_passesThrough_atTwoFingers_whenBothOverlaysClosed() { - XCTAssertFalse(AppCoordinator.shouldConsumeScroll(fingerCount: 2, launcherOpen: false, switcherOpen: false, canvasActive: false)) - XCTAssertFalse(AppCoordinator.shouldConsumeScroll(fingerCount: 1, launcherOpen: false, switcherOpen: false, canvasActive: false)) - XCTAssertFalse(AppCoordinator.shouldConsumeScroll(fingerCount: 0, launcherOpen: false, switcherOpen: false, canvasActive: false)) + XCTAssertFalse(AppCoordinator.shouldConsumeScroll(fingerCount: 2, launcherOpen: false, switcherOpen: false)) + XCTAssertFalse(AppCoordinator.shouldConsumeScroll(fingerCount: 1, launcherOpen: false, switcherOpen: false)) + XCTAssertFalse(AppCoordinator.shouldConsumeScroll(fingerCount: 0, launcherOpen: false, switcherOpen: false)) } func test_consumes_atThreeOrMore_regardlessOfOverlays() { - XCTAssertTrue(AppCoordinator.shouldConsumeScroll(fingerCount: 3, launcherOpen: false, switcherOpen: false, canvasActive: false)) - XCTAssertTrue(AppCoordinator.shouldConsumeScroll(fingerCount: 4, launcherOpen: false, switcherOpen: false, canvasActive: false)) - XCTAssertTrue(AppCoordinator.shouldConsumeScroll(fingerCount: 3, launcherOpen: true, switcherOpen: true, canvasActive: false)) - } - - // (a) Canvas active + 1-2 finger scroll passes through so it reaches the canvas's ScrollView. - func test_passesThrough_atTwoFingers_whenCanvasActive() { - XCTAssertFalse(AppCoordinator.shouldConsumeScroll(fingerCount: 2, launcherOpen: true, switcherOpen: false, canvasActive: true)) - XCTAssertFalse(AppCoordinator.shouldConsumeScroll(fingerCount: 1, launcherOpen: true, switcherOpen: false, canvasActive: true)) - } - - // (b) Canvas active still consumes 3+ fingers (a 4-finger resolve swipe must not leak to the front app). - func test_consumes_atFourFingers_whenCanvasActive() { - XCTAssertTrue(AppCoordinator.shouldConsumeScroll(fingerCount: 4, launcherOpen: true, switcherOpen: false, canvasActive: true)) - XCTAssertTrue(AppCoordinator.shouldConsumeScroll(fingerCount: 3, launcherOpen: true, switcherOpen: false, canvasActive: true)) - } - - // (c) Normal launcher nav (canvas not active) still consumes 1-2 finger scroll. - func test_consumes_atTwoFingers_whenLauncherOpen_canvasInactive() { - XCTAssertTrue(AppCoordinator.shouldConsumeScroll(fingerCount: 2, launcherOpen: true, switcherOpen: false, canvasActive: false)) - } - - // (d) Switcher open consumes 1-2 finger scroll (canvas is a launcher-only concept). - func test_consumes_atTwoFingers_whenSwitcherOpen_explicit() { - XCTAssertTrue(AppCoordinator.shouldConsumeScroll(fingerCount: 2, launcherOpen: false, switcherOpen: true, canvasActive: false)) + XCTAssertTrue(AppCoordinator.shouldConsumeScroll(fingerCount: 3, launcherOpen: false, switcherOpen: false)) + XCTAssertTrue(AppCoordinator.shouldConsumeScroll(fingerCount: 4, launcherOpen: false, switcherOpen: false)) + XCTAssertTrue(AppCoordinator.shouldConsumeScroll(fingerCount: 3, launcherOpen: true, switcherOpen: true)) } func test_passesThrough_whenNoOverlay() { - XCTAssertFalse(AppCoordinator.shouldConsumeScroll(fingerCount: 2, launcherOpen: false, switcherOpen: false, canvasActive: false)) + XCTAssertFalse(AppCoordinator.shouldConsumeScroll(fingerCount: 2, launcherOpen: false, switcherOpen: false)) } } diff --git a/Tests/ThreeFingerSwitcherTests/SelectionServiceTests.swift b/Tests/ThreeFingerSwitcherTests/SelectionServiceTests.swift deleted file mode 100644 index 1429b82..0000000 --- a/Tests/ThreeFingerSwitcherTests/SelectionServiceTests.swift +++ /dev/null @@ -1,405 +0,0 @@ -import XCTest -import AppKit -import CoreGraphics -@testable import ThreeFingerSwitcherCore - -/// Tests for `SelectionService` (spec: `selection-io`; tasks phase 6) covering the parts that run -/// headless: the pasteboard save→mutate→restore round-trip (clipboard left as it was after a paste), -/// the "empty/whitespace = no selection" normalization, the settable→AX-else-paste branch decision, -/// the clipboard read, and PNG encoding. The AX read/replace, ⌘C/⌘V synthesis, and ScreenCaptureKit -/// paths can't run deterministically headless — they're in the MANUAL-TEST CHECKLIST for the signed -/// build, not faked here. -@MainActor -final class SelectionServiceTests: XCTestCase { - - // MARK: - Fake pasteboard - - /// An in-memory `PasteboardAccess` mimicking `NSPasteboard` semantics: `changeCount` advances on - /// every write (set/restore), so the ⌘C poll and the save/restore round-trip are testable without - /// the real system pasteboard. `setStringExternally` simulates a copy landing on the board between - /// snapshot and restore (what a synthesized ⌘C would do). - private final class FakePasteboard: PasteboardAccess { - private(set) var changeCount = 0 - private var current: PasteboardSnapshot = PasteboardSnapshot(items: []) - - private(set) var setStrings: [String] = [] - private(set) var restoreCount = 0 - - var stringValue: String? { - current.items.first?[NSPasteboard.PasteboardType.string.rawValue] - .flatMap { String(data: $0, encoding: .utf8) } - } - - func string() -> String? { stringValue } - - /// The current board's best image representation (PNG before TIFF), mirroring `SystemPasteboard`. - func imageData() -> Data? { - current.items.first?[NSPasteboard.PasteboardType.png.rawValue] - ?? current.items.first?[NSPasteboard.PasteboardType.tiff.rawValue] - } - - /// Seed the board with a single image representation under `type` (e.g. `.png` / `.tiff`). - func seedImage(_ data: Data, type: NSPasteboard.PasteboardType) { - seed(PasteboardSnapshot(items: [[type.rawValue: data]])) - } - - func setString(_ text: String) { - setStrings.append(text) - current = PasteboardSnapshot(items: [[NSPasteboard.PasteboardType.string.rawValue: Data(text.utf8)]]) - changeCount += 1 - } - - private(set) var fileDeliveries: [(url: URL, path: String)] = [] - func setFileDelivery(url: URL, path: String) { - fileDeliveries.append((url, path)) - current = PasteboardSnapshot(items: [[ - NSPasteboard.PasteboardType.string.rawValue: Data(path.utf8), - NSPasteboard.PasteboardType.fileURL.rawValue: Data(url.absoluteString.utf8) - ]]) - changeCount += 1 - } - - func snapshot() -> PasteboardSnapshot { current } - - func restore(_ snapshot: PasteboardSnapshot) { - restoreCount += 1 - current = snapshot - changeCount += 1 - } - - /// Seed the board with a raw snapshot (e.g. a "password" or image item) for restore tests. - /// Distinct from `setString` so `setStrings` records ONLY service-driven writes. - func seed(_ snapshot: PasteboardSnapshot) { - current = snapshot - changeCount += 1 - } - - /// Seed a plain-text string without polluting `setStrings`. - func seedString(_ text: String) { - seed(PasteboardSnapshot(items: [[NSPasteboard.PasteboardType.string.rawValue: Data(text.utf8)]])) - } - } - - /// A front app that is never our own pid, so `frontApp()` resolves it. - private func realFrontApp() -> NSRunningApplication? { - // Any running app whose pid isn't ours. The current process's frontmost may BE us under test, - // so synthesize from another running app when possible; nil is fine for the pure-logic tests. - NSWorkspace.shared.runningApplications.first { $0.processIdentifier != getpid() } - } - - /// A `SelectionService` whose paste keystroke is intercepted (no real app activation / no real - /// ⌘V) so the paste pasteboard round-trip is exercised headless and side-effect-free. `fired` - /// records which app the paste targeted. - private func makeService(pasteboard: FakePasteboard, - frontApp: NSRunningApplication?, - fired: @escaping (NSRunningApplication) -> Void = { _ in }) -> SelectionService { - SelectionService(frontAppProvider: { frontApp }, - pasteboard: pasteboard, - pasteKeystroke: fired) - } - - // MARK: - deliverFile() — dual-representation write + clipboard restore - - /// Delivering a file writes BOTH representations (path string + file URL) in one go and restores the - /// user's prior clipboard, mirroring the text-paste round-trip (`files-contextual-delivery`). - func testDeliverFileWritesBothRepsAndRestoresClipboard() async { - guard let app = realFrontApp() else { return } // headless guard: no other running app - let pb = FakePasteboard() - pb.seedString("user's prior clipboard") // what the user had copied - var firedPid: pid_t? - let svc = makeService(pasteboard: pb, frontApp: app, fired: { firedPid = $0.processIdentifier }) - - let attempted = await svc.deliverFile(url: URL(fileURLWithPath: "/tmp/a.txt"), path: "/tmp/a.txt") - - XCTAssertTrue(attempted, "a paste was attempted into the front app") - XCTAssertEqual(pb.fileDeliveries.count, 1) - XCTAssertEqual(pb.fileDeliveries.first?.path, "/tmp/a.txt", "the path rep is written for text targets") - XCTAssertEqual(pb.fileDeliveries.first?.url, URL(fileURLWithPath: "/tmp/a.txt"), "the file ref for Finder") - XCTAssertEqual(firedPid, app.processIdentifier, "the keystroke targeted the captured front app") - XCTAssertEqual(pb.string(), "user's prior clipboard", "the user's clipboard is restored after delivery") - XCTAssertEqual(pb.restoreCount, 1) - } - - /// With no front app, delivery does nothing and reports not-attempted — the caller surfaces a bounded - /// failure, never a false success, never a clipboard touch. - func testDeliverFileWithNoFrontAppReportsFalse() async { - let pb = FakePasteboard() - let svc = makeService(pasteboard: pb, frontApp: nil) - let attempted = await svc.deliverFile(url: URL(fileURLWithPath: "/tmp/a.txt"), path: "/tmp/a.txt") - XCTAssertFalse(attempted, "no front app → not attempted") - XCTAssertTrue(pb.fileDeliveries.isEmpty, "no pasteboard write") - XCTAssertEqual(pb.restoreCount, 0, "and no clipboard touch") - } - - // MARK: - normalized() — empty/whitespace = no selection - - func testNormalizedTreatsNilAsNoSelection() { - XCTAssertNil(SelectionService.normalized(nil)) - } - - func testNormalizedTreatsEmptyAsNoSelection() { - XCTAssertNil(SelectionService.normalized("")) - } - - func testNormalizedTreatsWhitespaceOnlyAsNoSelection() { - XCTAssertNil(SelectionService.normalized(" \n\t ")) - } - - func testNormalizedKeepsRealTextVerbatim() { - // Non-whitespace content is kept UNtrimmed so deliberate surrounding spacing reaches the model. - XCTAssertEqual(SelectionService.normalized(" hello world "), " hello world ") - } - - // MARK: - shouldUseAX() — settable→AX else paste branch - - func testShouldUseAXWhenSettable() { - XCTAssertTrue(SelectionService.shouldUseAX(focusedElementSettable: true)) - } - - func testShouldFallBackToPasteWhenNotSettable() { - XCTAssertFalse(SelectionService.shouldUseAX(focusedElementSettable: false)) - } - - // MARK: - readClipboardText() — current clipboard, normalized - - func testReadClipboardReturnsCurrentString() { - let pb = FakePasteboard() - pb.seedString("on the board") - let svc = SelectionService(frontAppProvider: { nil }, pasteboard: pb) - XCTAssertEqual(svc.readClipboardText(), "on the board") - } - - func testReadClipboardTreatsWhitespaceAsEmpty() { - let pb = FakePasteboard() - pb.seedString(" \n ") - let svc = SelectionService(frontAppProvider: { nil }, pasteboard: pb) - XCTAssertNil(svc.readClipboardText(), "whitespace-only clipboard is not input") - } - - func testReadClipboardNilWhenEmpty() { - let pb = FakePasteboard() - let svc = SelectionService(frontAppProvider: { nil }, pasteboard: pb) - XCTAssertNil(svc.readClipboardText()) - } - - // MARK: - readClipboardImage() / normalizedPNG() — clipboard image, normalized to PNG - - private static let pngSignature: [UInt8] = [0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A] - - /// Real PNG and TIFF bytes for the same tiny opaque image (built like `testPNGDataEncodesImage`). - private func makeImageBytes() -> (png: Data, tiff: Data) { - let width = 2, height = 2 - let cs = CGColorSpaceCreateDeviceRGB() - let ctx = CGContext(data: nil, width: width, height: height, bitsPerComponent: 8, - bytesPerRow: 0, space: cs, - bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue)! - ctx.setFillColor(CGColor(red: 0, green: 1, blue: 0, alpha: 1)) - ctx.fill(CGRect(x: 0, y: 0, width: width, height: height)) - let rep = NSBitmapImageRep(cgImage: ctx.makeImage()!) - return (rep.representation(using: .png, properties: [:])!, - rep.representation(using: .tiff, properties: [:])!) - } - - func testReadClipboardImageReturnsPNGUnchanged() { - let (png, _) = makeImageBytes() - let pb = FakePasteboard() - pb.seedImage(png, type: .png) - let svc = SelectionService(frontAppProvider: { nil }, pasteboard: pb) - XCTAssertEqual(svc.readClipboardImage(), png, "a PNG on the clipboard is handed back without a re-encode") - } - - func testReadClipboardImageNormalizesTIFFToPNG() { - let (_, tiff) = makeImageBytes() - let pb = FakePasteboard() - pb.seedImage(tiff, type: .tiff) // TIFF-only clipboard (no PNG rep) - let svc = SelectionService(frontAppProvider: { nil }, pasteboard: pb) - let out = svc.readClipboardImage() - XCTAssertNotNil(out, "a TIFF-only clipboard image is normalized, not dropped") - XCTAssertEqual(Array(out!.prefix(8)), Self.pngSignature, "it is normalized to PNG") - } - - func testReadClipboardImageNilWhenNoImage() { - let pb = FakePasteboard() - pb.seedString("just text, no image") - let svc = SelectionService(frontAppProvider: { nil }, pasteboard: pb) - XCTAssertNil(svc.readClipboardImage(), "text on the clipboard is not an image") - } - - func testReadClipboardImageNilWhenEmpty() { - let pb = FakePasteboard() - let svc = SelectionService(frontAppProvider: { nil }, pasteboard: pb) - XCTAssertNil(svc.readClipboardImage()) - } - - func testNormalizedPNGHandlesNilEmptyAndGarbage() { - XCTAssertNil(SelectionService.normalizedPNG(from: nil), "no data → nil") - XCTAssertNil(SelectionService.normalizedPNG(from: Data()), "empty data → nil") - XCTAssertNil(SelectionService.normalizedPNG(from: Data([0x00, 0x01, 0x02, 0x03])), - "undecodable bytes → nil (never garbage to the model)") - } - - func testNormalizedPNGPassesThroughPNGAndConvertsTIFF() { - let (png, tiff) = makeImageBytes() - XCTAssertEqual(SelectionService.normalizedPNG(from: png), png, "PNG passes through byte-for-byte") - let converted = SelectionService.normalizedPNG(from: tiff) - XCTAssertNotNil(converted) - XCTAssertEqual(Array(converted!.prefix(8)), Self.pngSignature, "TIFF is re-encoded to PNG") - } - - // MARK: - Paste round-trip — clipboard restored after a paste - - func testPasteAtCursorRestoresPriorClipboard() async throws { - let front = realFrontApp() - try XCTSkipIf(front == nil, "needs another running process to resolve a front app to paste into") - let pb = FakePasteboard() - pb.seedString("user's original clipboard") - let priorSnapshot = pb.snapshot() - var firedInto: NSRunningApplication? - let svc = makeService(pasteboard: pb, frontApp: front) { firedInto = $0 } - - await svc.pasteAtCursor("pasted result") - - XCTAssertEqual(pb.setStrings, ["pasted result"], "the result is put on the board for ⌘V") - XCTAssertNotNil(firedInto, "the ⌘V keystroke is fired into the captured front app") - XCTAssertEqual(pb.restoreCount, 1, "the prior clipboard is restored exactly once") - XCTAssertEqual(pb.snapshot(), priorSnapshot, "clipboard ends as the user's original contents") - XCTAssertEqual(pb.stringValue, "user's original clipboard") - } - - func testPasteRestoresNonTextClipboardUnchanged() async throws { - let front = realFrontApp() - try XCTSkipIf(front == nil, "needs another running process to resolve a front app to paste into") - // A non-text clipboard (e.g. a copied password stored under a sensitive type, or image bytes) - // must survive the paste fallback byte-for-byte (spec: "does not clobber a password"). - let secret = PasteboardSnapshot(items: [[ - "org.nspasteboard.ConcealedType": Data("hunter2".utf8), - NSPasteboard.PasteboardType.string.rawValue: Data("hunter2".utf8) - ]]) - let pb = FakePasteboard() - pb.seed(secret) - let svc = makeService(pasteboard: pb, frontApp: front) - - await svc.pasteAtCursor("some output") - - XCTAssertEqual(pb.snapshot(), secret, "the sensitive clipboard is restored unchanged") - XCTAssertEqual(pb.restoreCount, 1) - } - - func testPasteOfEmptyTextDoesNothing() async { - let pb = FakePasteboard() - pb.seedString("keep me") - var fired = false - let svc = makeService(pasteboard: pb, frontApp: realFrontApp()) { _ in fired = true } - - await svc.pasteAtCursor("") - - XCTAssertTrue(pb.setStrings.isEmpty, "empty text is never written to the board") - XCTAssertFalse(fired, "empty text never fires a paste keystroke") - XCTAssertEqual(pb.restoreCount, 0, "nothing to restore when nothing was pasted") - XCTAssertEqual(pb.stringValue, "keep me") - } - - func testPasteNoOpWithoutFrontApp() async { - let pb = FakePasteboard() - pb.seedString("untouched") - var fired = false - let svc = makeService(pasteboard: pb, frontApp: nil) { _ in fired = true } - - await svc.pasteAtCursor("would-be result") - - XCTAssertTrue(pb.setStrings.isEmpty, "no front app ⇒ no write/paste at all") - XCTAssertFalse(fired, "no front app ⇒ no keystroke") - XCTAssertEqual(pb.stringValue, "untouched") - } - - // MARK: - replaceSelection() — no-front-app guard - - func testReplaceSelectionFailsWithoutFrontApp() async { - let pb = FakePasteboard() - let svc = SelectionService(frontAppProvider: { nil }, pasteboard: pb) - let applied = await svc.replaceSelection("new text") - XCTAssertFalse(applied, "no front app ⇒ replace reports not applied") - } - - // MARK: - captureScreenRegion(_:) — Screen Recording gate + degenerate rect - - func testScreenCaptureReportsPermissionDeniedWhenNotGranted() async { - let pb = FakePasteboard() - let svc = SelectionService(frontAppProvider: { self.realFrontApp() }, - pasteboard: pb, - screenRecordingGranted: { false }) - let outcome = await svc.captureScreenRegion(CGRect(x: 0, y: 0, width: 100, height: 80)) - XCTAssertEqual(outcome, .permissionDenied, - "no Screen Recording ⇒ a NAMED permission gap (not silent no-input)") - } - - func testScreenCaptureReportsUnavailableForDegenerateRect() async { - let pb = FakePasteboard() - let svc = SelectionService(frontAppProvider: { self.realFrontApp() }, - pasteboard: pb, - screenRecordingGranted: { true }) // granted, but the rect is empty - let outcome = await svc.captureScreenRegion(CGRect(x: 10, y: 10, width: 0, height: 0)) - XCTAssertEqual(outcome, .unavailable, "an empty rectangle is unavailable, never a full-screen fallback") - } - - // MARK: - displayLocalRect() — Cocoa global (bottom-left) → display-local (top-left), flipped - - func testDisplayLocalRectFlipsToTopLeftWithinDisplay() { - // A 1440×900 main display at the origin; a selection 100 from the left, whose top is 200 below - // the display's top edge (so its Cocoa maxY = 900 - 200 = 700). - let displayFrame = CGRect(x: 0, y: 0, width: 1440, height: 900) - let selection = CGRect(x: 100, y: 600, width: 300, height: 100) // Cocoa: maxY = 700 - let local = SelectionService.displayLocalRect(forGlobalCocoa: selection, displayFrameCocoa: displayFrame) - XCTAssertEqual(local, CGRect(x: 100, y: 200, width: 300, height: 100), - "x offset within the display; y flipped to distance from the display's top") - } - - func testDisplayLocalRectOffsetsBySecondaryDisplayOrigin() { - // A secondary display to the right of and below the main one: its Cocoa origin offsets the local rect. - let displayFrame = CGRect(x: 1440, y: -200, width: 1280, height: 800) // maxY = 600 - let selection = CGRect(x: 1540, y: 400, width: 200, height: 100) // Cocoa: maxY = 500 - let local = SelectionService.displayLocalRect(forGlobalCocoa: selection, displayFrameCocoa: displayFrame) - XCTAssertEqual(local, CGRect(x: 100, y: 100, width: 200, height: 100), - "local x is rect.minX - frame.minX; local y is frame.maxY - rect.maxY") - } - - // MARK: - pngData() — encoding step - - func testPNGDataEncodesImage() { - // A 2×2 opaque image → non-empty PNG bytes with the PNG signature. - let width = 2, height = 2 - let cs = CGColorSpaceCreateDeviceRGB() - let ctx = CGContext(data: nil, width: width, height: height, bitsPerComponent: 8, - bytesPerRow: 0, space: cs, - bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue)! - ctx.setFillColor(CGColor(red: 1, green: 0, blue: 0, alpha: 1)) - ctx.fill(CGRect(x: 0, y: 0, width: width, height: height)) - let cgImage = ctx.makeImage()! - - let png = SelectionService.pngData(from: cgImage) - XCTAssertNotNil(png) - // PNG magic number: 89 50 4E 47. - XCTAssertEqual(Array(png!.prefix(4)), [0x89, 0x50, 0x4E, 0x47]) - } - - // MARK: - SystemPasteboard snapshot/restore against a real (non-general) NSPasteboard - - func testSystemPasteboardRoundTripsItems() { - // Use a uniquely-named pasteboard (not .general) so the test never disturbs the user's - // clipboard, while still exercising the real NSPasteboard snapshot/restore code path. - let raw = NSPasteboard(name: NSPasteboard.Name("tfs-selection-test-\(UUID().uuidString)")) - defer { raw.releaseGlobally() } - raw.clearContents() - raw.setString("original", forType: .string) - - let board = SystemPasteboard(raw) - let snapshot = board.snapshot() - XCTAssertEqual(board.string(), "original") - - board.setString("temporary") - XCTAssertEqual(board.string(), "temporary") - - board.restore(snapshot) - XCTAssertEqual(board.string(), "original", "restore returns the captured contents") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/SkillsTests.swift b/Tests/ThreeFingerSwitcherTests/SkillsTests.swift deleted file mode 100644 index 6201ee7..0000000 --- a/Tests/ThreeFingerSwitcherTests/SkillsTests.swift +++ /dev/null @@ -1,513 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for the `ai-skills-as-files` slice (tasks §1–§7): the pure `SkillManifest`/`SkillFile` -/// parse↔serialize round-trip + sink encoding (1.x/2.x), the shared `InMemoryDocIndex` retrieval -/// (3.x), the `SkillStore` built-in projection + shadowing + problem collection (4.x), the -/// `SkillToolProvider` descriptor projection + `run()` outcome mapping (5.x), and the `SkillError` -/// taxonomy + the one translator (6.x). Everything here is MLX-free Core, driven by scripted fakes — -/// no model, no live filesystem events. -@MainActor -final class SkillsTests: XCTestCase { - - // MARK: - 1/2. SkillFile parse → serialize → parse round-trip - - /// A fixture exercising hex tint + a `language` runtimeParameter is a fixed point: parse, - /// serialize, re-parse, and the manifest is stable. - func testSkillFileRoundTripIsAFixedPoint() { - let fixture = """ - --- - id: fix-grammar - title: Fix Grammar - summary: Correct spelling and grammar in the selected text, returning only the fixed text. - keywords: [grammar, spelling, proofread] - category: Writing - icon: text.badge.checkmark - tint: #40B866 - input: selection - output: replaceSelection - confirmBeforeRun: false - runtimeParameter: language:English - --- - Fix the spelling and grammar of the following text in {lang}. Return only the corrected text: - - {input} - """ - guard case let .success(first) = SkillFile.parse(fixture) else { - return XCTFail("the fixture should parse") - } - // The hex tint decoded to the right ItemColor. - let tint = first.command.tint - XCTAssertNotNil(tint) - XCTAssertEqual(tint.map { Int(($0.red * 255).rounded()) }, 0x40) - XCTAssertEqual(tint.map { Int(($0.green * 255).rounded()) }, 0xB8) - XCTAssertEqual(tint.map { Int(($0.blue * 255).rounded()) }, 0x66) - // The language runtimeParameter survived. - XCTAssertEqual(first.command.runtimeParameter?.languageDefault, "English") - XCTAssertEqual(first.command.runtimeParameter?.options, AILanguages.all) - - // serialize → re-parse is a fixed point. - let serialized = SkillFile.serialize(first) - guard case let .success(second) = SkillFile.parse(serialized) else { - return XCTFail("the serialized form should re-parse") - } - XCTAssertEqual(first.id, second.id) - XCTAssertEqual(first.title, second.title) - XCTAssertEqual(first.summary, second.summary) - XCTAssertEqual(first.keywords, second.keywords) - XCTAssertEqual(first.category, second.category) - // The embedded AICommand is a fixed point in every field EXCEPT its per-instance `id` (minted - // fresh on each parse — the file `id` string is the skill identity, the AICommand.id is the - // per-band-instance identity, exactly the copy(of:) stencil rule). - var aligned = second.command - aligned.id = first.command.id - XCTAssertEqual(first.command, aligned, "the embedded AICommand round-trips in every field but its fresh id") - XCTAssertNotEqual(first.command.id, second.command.id, "each parse mints a fresh AICommand.id") - - // Idempotent: serializing the re-parsed value yields the same text. - XCTAssertEqual(serialized, SkillFile.serialize(second)) - } - - /// A `.skill.md` saved with the OLD single `input:`/`output:` lines still parses, migrated into the - /// capability sets (change `ai-action-context-resolution`). - func testLegacySkillInputOutputMigrateToSets() { - let legacy = """ - --- - id: legacy - title: Legacy - summary: A skill saved with the old single input/output fields. - icon: sparkles - input: selection - output: replaceSelection - --- - {input} - """ - guard case let .success(m) = SkillFile.parse(legacy) else { return XCTFail("legacy skill should parse") } - XCTAssertEqual(m.command.inputs, [.selection, .clipboard], - "legacy input: selection migrates to the ambient text set") - XCTAssertEqual(m.command.outputs, [.replaceSelection, .pasteAtCursor], - "legacy output: replaceSelection migrates to the write-back pair") - } - - /// A `.skill.md` using the new comma-joined `inputs:`/`outputs:` sets parses exactly and round-trips. - func testNewSkillInputsOutputsSetsParseAndRoundTrip() { - let modern = """ - --- - id: modern - title: Modern - summary: A skill using capability sets. - icon: sparkles - inputs: selection, clipboard, clipboardImage - outputs: pasteAtCursor, previewOnly - --- - {input} - """ - guard case let .success(m) = SkillFile.parse(modern) else { return XCTFail("modern skill should parse") } - XCTAssertEqual(m.command.inputs, [.selection, .clipboard, .clipboardImage]) - XCTAssertEqual(m.command.outputs, [.pasteAtCursor, .previewOnly]) - guard case let .success(reparsed) = SkillFile.parse(SkillFile.serialize(m)) else { - return XCTFail("the serialized set form should re-parse") - } - XCTAssertEqual(reparsed.command.inputs, m.command.inputs, "inputs set round-trips") - XCTAssertEqual(reparsed.command.outputs, m.command.outputs, "outputs set round-trips") - } - - /// Every `OutputTarget` case round-trips through the flat colon encoding. - func testEveryOutputTargetRoundTripsThroughTheColonEncoding() { - let cases: [OutputTarget] = [ - .replaceSelection, - .pasteAtCursor, - .previewOnly, - .runTask(.addToCalendar), - .runTask(.addToReminder), - .runTask(.newContact), - .runTask(.saveToProject(project: "Inbox")), - .runTask(.openToolWithPayload(tool: "com.example.tool")), - .sendTo(.shortcut(name: "My Shortcut")), - .sendTo(.urlScheme("x-callback://run")), - .sendTo(.shell(command: "pbcopy")), - ] - for output in cases { - let encoded = SkillFile.serializeOutput(output) - let decoded = SkillFile.parseOutput(encoded) - XCTAssertEqual(decoded, output, "OutputTarget \(output) should round-trip via \(encoded)") - } - } - - /// A malformed fixture yields exactly one `SkillProblem` (no crash) — exercised through the store's - /// per-file mapping (missing front-matter, missing required field, unknown enum value). - func testMalformedFixturesEachYieldExactlyOneProblem() throws { - let folder = try makeTempFolder() - defer { try? FileManager.default.removeItem(at: folder) } - - write("no-frontmatter.skill.md", "just a body, no delimiters", to: folder) - write("missing-id.skill.md", """ - --- - title: No Id - summary: missing the id field - input: selection - output: previewOnly - --- - body - """, to: folder) - write("bad-input.skill.md", """ - --- - id: bad-input - title: Bad Input - summary: an unknown input enum value - input: telepathy - output: previewOnly - --- - body - """, to: folder) - - let (skills, problems) = SkillStore.loadUserFolder(folder) - XCTAssertTrue(skills.isEmpty, "no malformed file parses into a skill") - XCTAssertEqual(problems.count, 3, "exactly one problem per malformed file, no crash, no silent drop") - for problem in problems { - XCTAssertFalse(problem.headline.isEmpty, "every problem carries a clean headline") - XCTAssertFalse(problem.headline.contains("SkillError"), "the headline is never a raw enum dump") - } - } - - // MARK: - 3. InMemoryDocIndex - - private func doc(_ id: String, summary: String, keywords: [String]) -> IndexedDoc { - IndexedDoc(id: id, title: id, summary: summary, keywords: keywords, kind: .skill, - bodyPath: URL(fileURLWithPath: "/tmp/\(id).skill.md")) - } - - func testInMemoryDocIndexAllSummariesAndDeterministicRetrieve() { - let docs = [ - doc("fix-grammar", summary: "Correct spelling and grammar in text.", keywords: ["grammar", "spelling"]), - doc("add-calendar", summary: "Create a meeting from the text.", keywords: ["calendar", "meeting"]), - doc("summarize", summary: "Summarize the text into a short paragraph.", keywords: ["summary"]), - ] - let bodies = ["fix-grammar": "FIX BODY", "add-calendar": "CAL BODY", "summarize": "SUM BODY"] - let index = InMemoryDocIndex(docs: docs, bodies: bodies) - - XCTAssertEqual(index.allSummaries().map(\.id), ["fix-grammar", "add-calendar", "summarize"], - "allSummaries returns the full TOC in snapshot order") - - // Keyword match ranks first. - let hits = index.retrieve(query: "schedule a meeting on my calendar", limit: 5) - XCTAssertEqual(hits.first?.id, "add-calendar", "the keyword-matching doc ranks first") - - // No-hit returns [] (the router falls back to the full TOC; never throws). - XCTAssertTrue(index.retrieve(query: "zzz totally unrelated qqq", limit: 5).isEmpty, - "a no-hit query returns an empty list") - - // body(of:) returns the right body. - XCTAssertEqual(try index.body(of: "summarize"), "SUM BODY") - XCTAssertThrowsError(try index.body(of: "ghost"), "an unknown id throws, never returns junk") - } - - func testIndexedDocCodableRoundTrip() throws { - let original = doc("fix-grammar", summary: "Fix grammar.", keywords: ["grammar"]) - let data = try JSONEncoder().encode(original) - let decoded = try JSONDecoder().decode(IndexedDoc.self, from: data) - XCTAssertEqual(decoded, original) - } - - // MARK: - 4. SkillStore - - func testBuiltInProjectionMatchesCatalog() async throws { - let folder = try makeTempFolder() - defer { try? FileManager.default.removeItem(at: folder) } - let store = SkillStore(userFolder: folder) - let result = await store.loadAll() - - XCTAssertTrue(result.problems.isEmpty, "an empty user folder yields no problems") - XCTAssertEqual(result.skills.count, AICommandCatalog.entries.count, - "one built-in skill per catalog preset") - - // Categories are preserved on the projected skills. - let projectedCategories = Set(result.skills.compactMap(\.category)) - let catalogCategories = Set(AICommandCatalog.Category.allCases.map(\.rawValue)) - XCTAssertEqual(projectedCategories, catalogCategories, "every catalog category survives the projection") - - // Each built-in skill carries an AICommand equal to its catalog command (reused verbatim). - let firstEntry = AICommandCatalog.entries[0] - let firstSkill = result.skills[0] - XCTAssertEqual(firstSkill.command, firstEntry.command) - XCTAssertEqual(firstSkill.origin, .builtIn) - } - - func testUserFileShadowsBuiltInByID() async throws { - let folder = try makeTempFolder() - defer { try? FileManager.default.removeItem(at: folder) } - - // The built-in "Fix Grammar" projects to id `fix-grammar`. Shadow it with a user file. - XCTAssertTrue(SkillStore.builtInManifests().contains { $0.id == "fix-grammar" }, - "the catalog projects a fix-grammar built-in") - write("fix-grammar.skill.md", """ - --- - id: fix-grammar - title: My Fixed Grammar - summary: A user-authored override of the built-in. - input: selection - output: previewOnly - --- - USER OVERRIDE BODY - """, to: folder) - - let store = SkillStore(userFolder: folder) - let result = await store.loadAll() - XCTAssertEqual(result.skills.count, AICommandCatalog.entries.count, - "a shadow replaces in place — not a duplicate, count unchanged") - guard let shadowed = result.skills.first(where: { $0.id == "fix-grammar" }) else { - return XCTFail("the shadowed id is still present") - } - XCTAssertEqual(shadowed.origin, .user, "origin flips to .user") - XCTAssertEqual(shadowed.title, "My Fixed Grammar") - // The body served by the index is the user's body. - XCTAssertEqual(try store.index().body(of: "fix-grammar"), "USER OVERRIDE BODY") - } - - func testDuplicateUserIDIsOneProblemAndRestSucceed() async throws { - let folder = try makeTempFolder() - defer { try? FileManager.default.removeItem(at: folder) } - - // Two user files share an id (and a distinct valid user skill rides alongside). - write("a-dup.skill.md", userSkill(id: "dup", title: "First", body: "A"), to: folder) - write("b-dup.skill.md", userSkill(id: "dup", title: "Second", body: "B"), to: folder) - write("solo.skill.md", userSkill(id: "solo-skill", title: "Solo", body: "S"), to: folder) - - let (skills, problems) = SkillStore.loadUserFolder(folder) - XCTAssertEqual(problems.count, 1, "the duplicate is exactly one problem") - XCTAssertEqual(Set(skills.map(\.id)), ["dup", "solo-skill"], - "the first-by-filename dup wins and the unrelated skill still loads") - // Deterministic winner: a-dup.skill.md sorts before b-dup.skill.md. - XCTAssertEqual(skills.first(where: { $0.id == "dup" })?.title, "First") - } - - func testEmptyOrAbsentFolderIsNotAnError() async throws { - let absent = URL(fileURLWithPath: "/tmp/three-finger-skills-\(UUID().uuidString)") - let (skills, problems) = SkillStore.loadUserFolder(absent) - XCTAssertTrue(skills.isEmpty) - XCTAssertTrue(problems.isEmpty, "an absent folder is idle, not a failure") - } - - // MARK: - 4.4 Watch + coalesced reload - - /// The pure coalescer collapses a burst: a check scheduled for an early event declines once a - /// newer event has pushed `lastEvent` forward, and only fires once the window has truly settled. - func testReloadCoalescerCollapsesABurst() { - let coalescer = ReloadCoalescer(interval: 0.25) - let t0 = Date(timeIntervalSinceReferenceDate: 1000) - - // A single event: the check one window later is settled. - XCTAssertTrue(coalescer.isSettled(lastEvent: t0, now: t0.addingTimeInterval(0.25))) - - // A burst: the check scheduled for t0 fires at t0+0.25, but a newer event landed at t0+0.20, - // so `lastEvent` is t0+0.20 and only 0.05s has elapsed → not settled (the burst keeps collapsing). - let newer = t0.addingTimeInterval(0.20) - XCTAssertFalse(coalescer.isSettled(lastEvent: newer, now: t0.addingTimeInterval(0.25))) - - // The window after the LAST event of the burst is settled → exactly one reload fires. - XCTAssertTrue(coalescer.isSettled(lastEvent: newer, now: newer.addingTimeInterval(0.25))) - - // Just-before the window is not yet settled. - XCTAssertFalse(coalescer.isSettled(lastEvent: t0, now: t0.addingTimeInterval(0.10))) - } - - /// The watcher's coalesced-reload path re-runs `loadAll()` off-main and republishes the fresh - /// result on the main actor (driven by the manual trigger — the live FS event is user-run-verify). - /// A file dropped after the watcher started shows up in the republished snapshot, proving - /// `loadAll()` is idempotent (re-call picks up the new file). - func testWatcherReloadPicksUpANewFileAndRepublishes() async throws { - let folder = try makeTempFolder() - defer { try? FileManager.default.removeItem(at: folder) } - - let store = SkillStore(userFolder: folder) - _ = await store.loadAll() // initial: built-ins only, no user skill - - let exp = expectation(description: "republished") - let republished = UncheckedBox(nil) - let watcher = SkillFolderWatcher(store: store) { result in - republished.value = result - exp.fulfill() - } - watcher.start() - defer { watcher.stop() } - - // Drop a new user skill, then drive the coalesced reload manually. - write("dropped-in.skill.md", userSkill(id: "dropped-in", title: "Dropped In", body: "B"), to: folder) - watcher.triggerReloadForTesting() - - await fulfillment(of: [exp], timeout: 5) - let result = republished.value - XCTAssertNotNil(result, "the reload republished a fresh snapshot") - XCTAssertTrue(result?.skills.contains { $0.id == "dropped-in" } ?? false, - "loadAll() is idempotent — the re-run picked up the new file") - // The store's own snapshot also reflects the reload (manifest(id:) resolves it). - XCTAssertNotNil(store.manifest(id: "dropped-in")) - } - - // MARK: - 5. SkillToolProvider descriptor projection - - func testCalendarSkillProjectsParsedCalendarSchemaAndConfirm() { - let cmd = AICommand(name: "Add to Calendar", icon: .sfSymbol("calendar"), - input: .selection, promptTemplate: "{input}", output: .runTask(.addToCalendar)) - let manifest = SkillManifest(id: "add-to-calendar", origin: .builtIn, title: "Add to Calendar", - summary: "Create a meeting from the selected text.", command: cmd) - let descriptor = SkillToolProvider.descriptor(for: manifest) - XCTAssertEqual(descriptor.name, "add-to-calendar") - XCTAssertEqual(descriptor.summary, "Create a meeting from the selected text.") - XCTAssertEqual(descriptor.argsSchema, ParsedCalendarEvent.schema, - "a calendar skill projects the ParsedCalendarEvent schema") - XCTAssertEqual(descriptor.writePolicy, .confirm, "a side-effecting sink defaults to .confirm") - } - - func testGrammarSkillProjectsTextSchemaAndAuto() { - let cmd = AICommand(name: "Fix Grammar", icon: .sfSymbol("textformat"), - input: .selection, promptTemplate: "{input}", output: .replaceSelection) - let manifest = SkillManifest(id: "fix-grammar", origin: .builtIn, title: "Fix Grammar", - summary: "Correct spelling and grammar.", command: cmd) - let descriptor = SkillToolProvider.descriptor(for: manifest) - XCTAssertEqual(descriptor.argsSchema.name, "text_result", "an in-place sink projects the minimal text schema") - XCTAssertEqual(descriptor.writePolicy, .auto, "an in-place sink is .auto") - } - - // MARK: - 5.2 SkillToolProvider.run() outcome mapping - - func testRunInPlaceSkillReturnsDone() async { - let cmd = AICommand(name: "Fix Grammar", icon: .sfSymbol("textformat"), - input: .selection, promptTemplate: "Fix: {input}", output: .replaceSelection) - let manifest = SkillManifest(id: "fix-grammar", origin: .builtIn, title: "Fix Grammar", - summary: "Correct grammar.", command: cmd) - let runtime = StubLLMRuntime(scriptedTokens: ["Fixed ", "text."]) - let provider = SkillToolProvider(manifests: [manifest], runtime: runtime, - dispatcher: FakeDispatcher()) - let result = await provider.run(call(for: manifest), gate: ScriptedApprovalGate()) - XCTAssertEqual(result.status, .done, "an in-place skill returns .done") - XCTAssertEqual(result.summary, "Fixed text.", "the generated text is the step outcome") - } - - func testRunSideEffectingDeclineReturnsDeclined() async { - let cmd = AICommand(name: "Add to Calendar", icon: .sfSymbol("calendar"), - input: .selection, promptTemplate: "{input}", output: .runTask(.addToCalendar)) - let manifest = SkillManifest(id: "add-to-calendar", origin: .builtIn, title: "Add to Calendar", - summary: "Create a meeting.", command: cmd) - let dispatcher = FakeDispatcher() - dispatcher.reviewToReturn = .declined(reason: "not a meeting") - let provider = SkillToolProvider(manifests: [manifest], runtime: StubLLMRuntime(), - dispatcher: dispatcher) - let result = await provider.run(call(for: manifest), gate: ScriptedApprovalGate([.approve])) - XCTAssertEqual(result.status, .declined(reason: "not a meeting")) - XCTAssertEqual(dispatcher.executed, 0, "a declined review fires no side effect") - } - - func testRunSideEffectingSinkFailureReturnsFailedNotFalseDone() async { - let cmd = AICommand(name: "Add to Calendar", icon: .sfSymbol("calendar"), - input: .selection, promptTemplate: "{input}", output: .runTask(.addToCalendar)) - let manifest = SkillManifest(id: "add-to-calendar", origin: .builtIn, title: "Add to Calendar", - summary: "Create a meeting.", command: cmd) - let dispatcher = FakeDispatcher() - dispatcher.executeError = TaskError.calendarPermissionDenied - let provider = SkillToolProvider(manifests: [manifest], runtime: StubLLMRuntime(), - dispatcher: dispatcher) - let result = await provider.run(call(for: manifest), gate: ScriptedApprovalGate([.approve])) - guard case let .failed(headline) = result.status else { - return XCTFail("a sink that didn't land is .failed, never a false Done") - } - XCTAssertTrue(headline.contains("Calendar"), "the clean TaskError headline, not a raw dump") - } - - // MARK: - 6. SkillError + the one translator - - func testSkillErrorHeadlinesAreCleanAndBounded() { - let errors: [SkillError] = [ - .malformedFrontMatter(detail: "raw parser dump xyz"), - .missingRequiredField(name: "id"), - .unknownEnumValue(field: "input", value: "telepathy"), - .duplicateID(id: "dup"), - .unreadable(detail: "/private/var/secret/path"), - ] - for error in errors { - let presented = AIError.message(for: error) - XCTAssertFalse(presented.headline.isEmpty) - XCTAssertFalse(presented.headline.contains("SkillError"), "no reflected enum dump in the headline") - XCTAssertFalse(presented.headline.contains("detail:"), "no raw associated-value name leaks") - } - // The raw detail rides on `details`, never the headline. - let unreadable = AIError.message(for: SkillError.unreadable(detail: "/private/var/secret/path")) - XCTAssertFalse(unreadable.headline.contains("/private/var/secret"), - "the raw path stays out of the headline") - } - - // MARK: - Catalog unchanged sanity - - func testCatalogProjectionPreservesSeededNamesAndOrder() { - // The migration keeps AICommandCatalog as the source of truth; confirm it is unchanged. - XCTAssertEqual(AICommandCatalog.seeded().map(\.name), - ["Fix Grammar", "Make Concise", "Improve Writing", "Translate", - "Explain", "Summarize", "Draft a Reply", "Add to Calendar"]) - } - - // MARK: - Fakes (mirroring ToolRoutingTests) - - private func call(for m: SkillManifest) -> RoutedCall { - RoutedCall(descriptor: SkillToolProvider.descriptor(for: m), - route: ToolRoute(tool: m.id, argumentsJSON: "{}"), - userText: "the input text", source: TaskSource()) - } - - private final class FakeDispatcher: TaskDispatching { - var reviewToReturn: TaskReview = .action( - title: "Event", - fields: [ReviewField("Title", "Lunch")], - payload: .openTool(tool: "x", action: ParsedOpenTool(applicable: true, reason: nil, payload: "p"))) - var executeError: Error? - private(set) var executed = 0 - func prepare(_ kind: TaskKind, resolvedPrompt: String, source: TaskSource, reasoning: Bool) async -> TaskReview { - reviewToReturn - } - func execute(_ review: TaskReview) async throws { - if let executeError { throw executeError } - executed += 1 - } - } - - private final class ScriptedApprovalGate: ApprovalGate, @unchecked Sendable { - private var decisions: [ApprovalDecision] - private let lock = NSLock() - init(_ decisions: [ApprovalDecision] = []) { self.decisions = decisions } - func awaitDecision(for review: TaskReview) async -> ApprovalDecision { - lock.lock(); defer { lock.unlock() } - return decisions.isEmpty ? .approve : decisions.removeFirst() - } - } - - /// A tiny mutable box for capturing a value out of a `@Sendable` republish closure in a test. - private final class UncheckedBox: @unchecked Sendable { - var value: T - init(_ value: T) { self.value = value } - } - - // MARK: - Temp-folder helpers - - private func makeTempFolder() throws -> URL { - let url = FileManager.default.temporaryDirectory - .appendingPathComponent("three-finger-skills-\(UUID().uuidString)") - try FileManager.default.createDirectory(at: url, withIntermediateDirectories: true) - return url - } - - private func write(_ name: String, _ contents: String, to folder: URL) { - try? contents.write(to: folder.appendingPathComponent(name), atomically: true, encoding: .utf8) - } - - private func userSkill(id: String, title: String, body: String) -> String { - """ - --- - id: \(id) - title: \(title) - summary: A user-authored skill named \(title). - input: selection - output: previewOnly - --- - \(body) - """ - } -} diff --git a/Tests/ThreeFingerSwitcherTests/StubTernaryRuntimeTests.swift b/Tests/ThreeFingerSwitcherTests/StubTernaryRuntimeTests.swift deleted file mode 100644 index bf73b07..0000000 --- a/Tests/ThreeFingerSwitcherTests/StubTernaryRuntimeTests.swift +++ /dev/null @@ -1,152 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for the deterministic CPU-lane stub (`StubTernaryRuntime`, task §2.3) and the CPU-lane error -/// mapping into `RuntimeError` → `AIError.message(for:)` (tasks §4.1 / §4.2). These exercise the -/// EXISTING `LLMRuntime` seam (design D2 — a second conformer, not a new protocol) without real weights. -final class StubTernaryRuntimeTests: XCTestCase { - - private struct RouteDecision: Codable, Equatable { let action: String } - - // MARK: - §2.3 structured route turn + short generate, de-mux to the right session - - func testStructuredRouteTurnProducesValue() async throws { - let json = "{\"action\":\"open\"}" - let runtime = StubTernaryRuntime( - structuredScript: .valid(json: json) - ) - let schema = StructuredSchema(name: "route", json: "{\"required\":[\"action\"]}") - let outcome = try await runtime.structured( - LLMRequest(prompt: "route this"), - schema: schema, - as: RouteDecision.self - ) - XCTAssertEqual(outcome.value, RouteDecision(action: "open")) - } - - func testStructuredCanDecline() async throws { - let runtime = StubTernaryRuntime(structuredScript: .decline(reason: "not a tool task")) - let schema = StructuredSchema(name: "route", json: "{\"required\":[\"action\"]}") - let outcome = try await runtime.structured( - LLMRequest(prompt: "chat"), - schema: schema, - as: RouteDecision.self - ) - XCTAssertTrue(outcome.isDeclined) - XCTAssertEqual(outcome.declineReason, "not a tool task") - } - - func testShortGenerateStreamsScriptedTokens() async throws { - let runtime = StubTernaryRuntime(scriptedTokens: ["yes", " ", "park"], interTokenDelayNanos: 0) - var out = "" - for try await token in runtime.generate(LLMRequest(prompt: "should park?")) { - out += token.text - } - XCTAssertEqual(out, "yes park") - } - - func testTokensDeMuxToTheRightSession() async throws { - // Two independent sessions, each its own stub burst → each session's text stays its own. - let sessionA = AgentSessionID() - let sessionB = AgentSessionID() - let rtA = StubTernaryRuntime(scriptedTokens: ["A1", "A2"], interTokenDelayNanos: 0) - let rtB = StubTernaryRuntime(scriptedTokens: ["B1", "B2"], interTokenDelayNanos: 0) - - var collected: [AgentSessionID: String] = [:] - for try await t in rtA.generate(LLMRequest(prompt: "a")) { - collected[sessionA, default: ""] += t.text - } - for try await t in rtB.generate(LLMRequest(prompt: "b")) { - collected[sessionB, default: ""] += t.text - } - XCTAssertEqual(collected[sessionA], "A1A2") - XCTAssertEqual(collected[sessionB], "B1B2") - } - - // MARK: - §5.3 capabilities = text only; vision is a hard error (never routed to CPU) - - func testCapabilitiesAdvertiseTextOnly() { - let runtime = StubTernaryRuntime() - XCTAssertEqual(runtime.capabilities, [.text]) - XCTAssertFalse(runtime.capabilities.contains(.vision)) - } - - func testVisionRequestIsHardError() async { - let runtime = StubTernaryRuntime() - let request = LLMRequest(prompt: "describe", image: Data([0x1])) - do { - for try await _ in runtime.generate(request) {} - XCTFail("a vision request against the text-only CPU lane must error") - } catch let RuntimeError.unsupportedModality(modality) { - XCTAssertEqual(modality, .vision) - } catch { - XCTFail("expected unsupportedModality, got \(error)") - } - } - - // MARK: - §4.1 a simulated CPU-lane failure → clean AIPresentedError headline (no raw text) - - func testSimulatedCPULaneFailureMapsToCleanHeadline() async { - let runtime = StubTernaryRuntime(scriptedError: .modelLoadFailed(detail: "bitnet ctx init -12")) - do { - for try await _ in runtime.generate(LLMRequest(prompt: "route")) {} - XCTFail("the scripted failure must throw") - } catch { - let presented = AIError.message(for: error) - XCTAssertEqual(presented.headline, "The model could not be loaded.") - // The raw vendor text rides on details ONLY — never the headline. - XCTAssertFalse(presented.headline.contains("bitnet")) - XCTAssertEqual(presented.details, "bitnet ctx init -12") - } - } - - func testStructuredFailureMapsToCleanHeadline() async { - let runtime = StubTernaryRuntime(scriptedError: .unavailable(reason: "ternary lane unavailable")) - let schema = StructuredSchema(name: "route", json: "{\"required\":[\"action\"]}") - do { - _ = try await runtime.structured(LLMRequest(prompt: "x"), schema: schema, as: RouteDecision.self) - XCTFail("the scripted failure must throw") - } catch { - let presented = AIError.message(for: error) - XCTAssertEqual(presented.headline, "ternary lane unavailable") - } - } - - // MARK: - §4.2 a cancelled CPU-lane turn is NOT a failure - - func testCancelledCPULaneTurnIsNotAFailure() async throws { - // A slow burst we cancel mid-stream. Dropping the iterator after the first token fires the - // stream's `onTermination`, which cancels the producer — whose next checkpoint surfaces a benign - // cancellation (RuntimeError.cancelled / CancellationError), NEVER a real `.failed` for the turn. - let runtime = StubTernaryRuntime(scriptedTokens: Array(repeating: "x", count: 30), - interTokenDelayNanos: 20_000_000) // 20 ms/token - enum Outcome: Equatable { case benignCancelled, completed, otherFailure } - var consumed = 0 - let stream = runtime.generate(LLMRequest(prompt: "long")) - let task = Task { () -> Outcome in - do { - for try await _ in stream { - consumed += 1 - if consumed >= 1 { break } // stop consuming → onTermination cancels the producer - } - return .completed - } catch RuntimeError.cancelled { - return .benignCancelled - } catch is CancellationError { - return .benignCancelled - } catch { - return .otherFailure - } - } - try await Task.sleep(nanoseconds: 50_000_000) - task.cancel() - let result = await task.value - // The decisive property: the burst did NOT run to completion, and the outcome is benign (a - // discarded turn), not a failure. - XCTAssertLessThan(consumed, 30, "cancellation stops the CPU-lane burst before all tokens emit") - XCTAssertNotEqual(result, .otherFailure, "a cancelled CPU-lane turn is NOT a real failure") - // AIError gives a clean benign "Cancelled." headline — NOT a generic failure or a false "done." - XCTAssertEqual(AIError.message(for: CancellationError()).headline, "Cancelled.") - XCTAssertEqual(AIError.message(for: RuntimeError.cancelled).headline, "Cancelled.") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/SubagentToolContributorTests.swift b/Tests/ThreeFingerSwitcherTests/SubagentToolContributorTests.swift deleted file mode 100644 index b47bcc9..0000000 --- a/Tests/ThreeFingerSwitcherTests/SubagentToolContributorTests.swift +++ /dev/null @@ -1,151 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for subagents as routable tools (`refactor-park-and-background-agents` / `ai-subagents`): -/// registered templates are invocable as `subagent:` steps, the sub-task runs in a FRESH -/// conversation (none of the orchestrator's history), ONLY the summary re-enters the orchestrator -/// thread, cancellation rides the turn, and a failure is a clean failed step. MLX-free, stub-driven. -final class SubagentToolContributorTests: XCTestCase { - - /// A chat runtime that records every request it serves (so isolation is assertable) and streams a - /// scripted response — or throws a scripted error. - private final class RecordingRuntime: LLMRuntime, @unchecked Sendable { - let capabilities: Set = [.text] - private(set) var servedPrompts: [String] = [] - private let response: [String] - private let error: RuntimeError? - private let lock = NSLock() - init(response: [String], error: RuntimeError? = nil) { - self.response = response - self.error = error - } - func generate(_ request: LLMRequest) -> AsyncThrowingStream { - lock.lock(); servedPrompts.append(request.prompt); lock.unlock() - let toks = response, err = error - return AsyncThrowingStream { c in - if let err { c.finish(throwing: err); return } - for (i, t) in toks.enumerated() { c.yield(Token(t, isFinal: i == toks.count - 1)) } - c.finish() - } - } - func structured(_ r: LLMRequest, schema: StructuredSchema, - as type: T.Type) async throws -> StructuredOutcome { - throw RuntimeError.serverUnavailable // subagents are prompt-only; never routed internally - } - } - - private func summarizeCall(input: String? = "long text to condense", - userText: String = "please summarize") -> RoutedCall { - let descriptor = Subagent.builtIns[0].toolDescriptor - let args = input.map { "{\"input\":\"\($0)\"}" } ?? "{}" - return RoutedCall(descriptor: descriptor, - route: ToolRoute(tool: descriptor.name, argumentsJSON: args), - userText: userText, source: TaskSource()) - } - - func testBuiltInsAreRegisteredAsRoutableTools() { - let contributor = SubagentToolContributor(templates: Subagent.builtIns, - runtimeProvider: { RecordingRuntime(response: ["x"]) }) - let names = contributor.descriptors().map(\.name) - XCTAssertEqual(names, ["subagent:summarize", "subagent:draft"]) - XCTAssertTrue(contributor.canHandle("subagent:summarize")) - XCTAssertFalse(contributor.canHandle("subagent:invented"), - "only REGISTERED templates are invocable — no model-invented subagents") - XCTAssertTrue(contributor.descriptors().allSatisfy { $0.writePolicy == .auto }, - "a subagent is CONTAINED (read-only to the orchestrator's world)") - } - - func testRunsInFreshConversationAndReturnsOnlyTheSummary() async { - let runtime = RecordingRuntime(response: ["the ", "gist"]) - let contributor = SubagentToolContributor(templates: Subagent.builtIns, - runtimeProvider: { runtime }) - let result = await contributor.run(summarizeCall(), gate: AutoApproveGate()) - - XCTAssertEqual(result.status, .done) - XCTAssertEqual(result.summary, "the gist", "ONLY the final text re-enters the orchestrator") - - let served = runtime.servedPrompts.joined() - XCTAssertTrue(served.contains("long text to condense"), "the routed input seeds the sub-task") - XCTAssertFalse(served.contains("please summarize"), - "the orchestrator's history is NOT visible to the subagent (fresh context)") - } - - func testEmptyRoutedArgsFallBackToTheUserText() async { - let runtime = RecordingRuntime(response: ["ok"]) - let contributor = SubagentToolContributor(templates: Subagent.builtIns, - runtimeProvider: { runtime }) - _ = await contributor.run(summarizeCall(input: nil, userText: "the actual ask"), - gate: AutoApproveGate()) - XCTAssertTrue(runtimeServed(runtime, contains: "the actual ask")) - } - - func testFailureIsACleanFailedStepNeverAFabricatedSummary() async { - let contributor = SubagentToolContributor( - templates: Subagent.builtIns, - runtimeProvider: { RecordingRuntime(response: [], error: .serverUnavailable) }) - let result = await contributor.run(summarizeCall(), gate: AutoApproveGate()) - guard case let .failed(headline) = result.status else { - return XCTFail("expected a failed step, got \(result.status)") - } - XCTAssertEqual(headline, AIError.message(for: RuntimeError.serverUnavailable).headline, - "the failure carries the clean translated headline") - XCTAssertTrue(result.summary.isEmpty, "no fabricated summary rides a failure") - } - - func testCancellationResolvesAsTheLoopCancelSentinelNotAFailure() async { - let contributor = SubagentToolContributor( - templates: Subagent.builtIns, - runtimeProvider: { RecordingRuntime(response: [], error: .cancelled) }) - let result = await contributor.run(summarizeCall(), gate: AutoApproveGate()) - XCTAssertEqual(result.status, .declined(reason: TaskKindToolContributor.cancelledReason), - "a cancelled subagent stops the loop quietly (a discard, never a failure)") - } - - /// End-to-end: the loop routes a `subagent:summarize` step and the summary re-enters the - /// orchestrator context as ONE tool message; the loop then answers over it. - func testLoopRoutesSubagentStepAndAbsorbsOnlyTheSummary() async { - let subagentRuntime = RecordingRuntime(response: ["condensed"]) - let contributor = SubagentToolContributor(templates: Subagent.builtIns, - runtimeProvider: { subagentRuntime }) - let registry = ToolRegistry([contributor]) - let loopRuntime = ScriptedLoopRuntime( - routes: ["{\"tool\":\"subagent:summarize\",\"argumentsJSON\":\"{\\\"input\\\":\\\"stuff\\\"}\"}", - "{\"tool\":\"\"}"], - answer: ["final answer"]) - let loop = AgentLoop(runtime: loopRuntime, registry: registry, - candidateSource: KeywordToolCandidateSource(all: { registry.allDescriptors() }), - gate: AutoApproveGate()) - let result = await loop.run(context: RouteContext(messages: [AgentMessage(role: .user, text: "summarize stuff")])) - XCTAssertEqual(result.outcome, .answered(text: "final answer")) - XCTAssertEqual(result.steps.map(\.summary), ["condensed"], - "exactly one tool message — the summary — re-entered the orchestrator") - } - - private func runtimeServed(_ runtime: RecordingRuntime, contains needle: String) -> Bool { - runtime.servedPrompts.joined().contains(needle) - } - - /// The orchestrator-side runtime for the end-to-end test: scripted routes + a final answer. - private final class ScriptedLoopRuntime: LLMRuntime, @unchecked Sendable { - let capabilities: Set = [.text] - private var routes: [String] - private let answerTokens: [String] - private let lock = NSLock() - init(routes: [String], answer: [String]) { self.routes = routes; self.answerTokens = answer } - func generate(_ request: LLMRequest) -> AsyncThrowingStream { - let toks = answerTokens - return AsyncThrowingStream { c in - for (i, t) in toks.enumerated() { c.yield(Token(t, isFinal: i == toks.count - 1)) } - c.finish() - } - } - func structured(_ r: LLMRequest, schema: StructuredSchema, - as type: T.Type) async throws -> StructuredOutcome { - let json: String = { - lock.lock(); defer { lock.unlock() } - return routes.isEmpty ? "{\"tool\":\"\"}" : routes.removeFirst() - }() - return .value(try JSONDecoder().decode(T.self, from: Data(json.utf8))) - } - } -} diff --git a/Tests/ThreeFingerSwitcherTests/TaskDispatcherTests.swift b/Tests/ThreeFingerSwitcherTests/TaskDispatcherTests.swift deleted file mode 100644 index 7da12d2..0000000 --- a/Tests/ThreeFingerSwitcherTests/TaskDispatcherTests.swift +++ /dev/null @@ -1,479 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for the agentic task layer (tasks phase 13.7; spec ai-command-tasks) against `StubLLMRuntime` -/// + fake sinks: each kind produces a schema-valid action OR a clean decline; non-conforming output is -/// repaired/retried and, if still invalid, yields NO action (`.unavailable`) — never a malformed side -/// effect; `execute` fires the side effect ONLY for a confirmed `.action`; calendar / save / open / -/// send each route to their injected sink/store/opener/sender with the right payload; save-to-project -/// appends to a temp-dir note. (EventKit's real prompt + real event creation is in the MANUAL-TEST -/// checklist — the system EventKit prompt is never faked.) -@MainActor -final class TaskDispatcherTests: XCTestCase { - - // MARK: - Fakes - - private final class FakeCalendarSink: CalendarSink { - private(set) var created: [ParsedCalendarEvent] = [] - var errorToThrow: Error? - func create(_ event: ParsedCalendarEvent) async throws { - if let e = errorToThrow { throw e } - created.append(event) - } - } - - private final class FakeReminderSink: ReminderSink { - private(set) var created: [ParsedReminder] = [] - var errorToThrow: Error? - func create(_ reminder: ParsedReminder) async throws { - if let e = errorToThrow { throw e } - created.append(reminder) - } - } - - private final class FakeContactSink: ContactSink { - private(set) var created: [ParsedContact] = [] - var errorToThrow: Error? - func create(_ contact: ParsedContact) async throws { - if let e = errorToThrow { throw e } - created.append(contact) - } - } - - private final class FakeProjectStore: ProjectStore { - private(set) var appended: [(project: String, content: String, source: TaskSource)] = [] - func append(project: String, content: String, source: TaskSource) throws { - appended.append((project, content, source)) - } - } - - private final class FakeToolOpener: ToolOpener { - private(set) var opened: [(tool: String, payload: String)] = [] - func open(tool: String, payload: String) async throws { opened.append((tool, payload)) } - } - - private final class FakeDestinationSender: DestinationSender { - private(set) var sent: [(destination: Destination, content: String)] = [] - func send(_ destination: Destination, content: String) async throws { - sent.append((destination, content)) - } - } - - /// A runtime that records the `LLMRequest` handed to `structured(...)`, then runs the real validate - /// pipeline over `scriptedJSON`. Lets a test assert the dispatcher propagated `reasoning` onto the - /// structured request (the lock guards the capture so it's robust under the dispatcher's async work). - private final class CapturingStructuredRuntime: LLMRuntime, @unchecked Sendable { - let capabilities: Set = [.text] - private let scriptedJSON: String - private let lock = NSLock() - private var _lastRequest: LLMRequest? - var lastRequest: LLMRequest? { lock.lock(); defer { lock.unlock() }; return _lastRequest } - - init(scriptedJSON: String) { self.scriptedJSON = scriptedJSON } - - func generate(_ request: LLMRequest) -> AsyncThrowingStream { - AsyncThrowingStream { $0.finish() } - } - - func structured( - _ request: LLMRequest, schema: StructuredSchema, as type: T.Type - ) async throws -> StructuredOutcome { - lock.lock(); _lastRequest = request; lock.unlock() - let data = Data(scriptedJSON.utf8) - return .value(try JSONDecoder().decode(T.self, from: data)) - } - } - - /// Build a dispatcher over a scripted stub runtime + the four fakes (created here so the fakes' - /// main-actor inits aren't evaluated in a nonisolated default-argument context). - private func makeDispatcher(stub: StubLLMRuntime, - calendar: FakeCalendarSink? = nil, - reminders: FakeReminderSink? = nil, - contacts: FakeContactSink? = nil, - projects: FakeProjectStore? = nil, - tools: FakeToolOpener? = nil, - senders: FakeDestinationSender? = nil) -> TaskDispatcher { - TaskDispatcher(runtimeProvider: { stub }, - calendarSink: calendar ?? FakeCalendarSink(), - reminderSink: reminders ?? FakeReminderSink(), - contactSink: contacts ?? FakeContactSink(), - projectStore: projects ?? FakeProjectStore(), - toolOpener: tools ?? FakeToolOpener(), - destinationSender: senders ?? FakeDestinationSender()) - } - - // MARK: - Calendar: schema-valid action - - func testCalendarProducesSchemaValidAction() async throws { - let stub = StubLLMRuntime(structuredScript: .valid( - json: #"{"applicable":true,"title":"Sync","start":"2026-06-09T15:00","end":"2026-06-09T16:00","attendees":["sam"],"notes":"weekly"}"#)) - let dispatcher = makeDispatcher(stub: stub) - let review = await dispatcher.prepare(.addToCalendar, resolvedPrompt: "meet sam tue 3pm", - source: TaskSource(), reasoning: false) - guard case let .action(title, fields, _) = review else { - return XCTFail("a meeting text should yield an action, got \(review)") - } - XCTAssertEqual(title, "Add to Calendar") - XCTAssertEqual(fields.first(where: { $0.label == "Title" })?.value, "Sync") - XCTAssertEqual(fields.first(where: { $0.label == "Start" })?.value, "2026-06-09T15:00") - XCTAssertEqual(fields.first(where: { $0.label == "Attendees" })?.value, "sam") - } - - // MARK: - Reasoning flag propagation (structured path) - - /// The dispatcher must carry the `reasoning` value passed to `prepare(...)` onto the structured - /// `LLMRequest`: when `prepare(..., reasoning: true)`, the request it hands `runtime.structured(...)` - /// has `reasoning == true` (the executor now owns reasoning resolution and threads it through). - func testReasoningFlagPropagatesToStructuredRequest() async throws { - let runtime = CapturingStructuredRuntime( - scriptedJSON: #"{"applicable":true,"title":"Sync","start":"2026-06-09T15:00"}"#) - let dispatcher = TaskDispatcher(runtimeProvider: { runtime }, - calendarSink: FakeCalendarSink(), - reminderSink: FakeReminderSink(), - contactSink: FakeContactSink(), - projectStore: FakeProjectStore(), - toolOpener: FakeToolOpener(), - destinationSender: FakeDestinationSender()) - _ = await dispatcher.prepare(.addToCalendar, resolvedPrompt: "meet sam tue 3pm", - source: TaskSource(), reasoning: true) - XCTAssertEqual(runtime.lastRequest?.reasoning, true, - "the dispatcher sets request.reasoning from the value passed to prepare") - } - - // MARK: - Reminders: schema-valid action routes to the reminder sink on commit - - func testReminderProducesActionAndExecutesToSink() async throws { - let stub = StubLLMRuntime(structuredScript: .valid( - json: #"{"applicable":true,"title":"Email Sam","due":"2026-06-09T09:00","priority":1,"notes":"re: launch"}"#)) - let reminders = FakeReminderSink() - let dispatcher = makeDispatcher(stub: stub, reminders: reminders) - let review = await dispatcher.prepare(.addToReminder, resolvedPrompt: "remind me to email sam", - source: TaskSource(), reasoning: false) - guard case let .action(title, fields, _) = review else { - return XCTFail("a task text should yield an action, got \(review)") - } - XCTAssertEqual(title, "Add to Reminders") - XCTAssertEqual(fields.first(where: { $0.label == "Title" })?.value, "Email Sam") - XCTAssertEqual(fields.first(where: { $0.label == "Due" })?.value, "2026-06-09T09:00") - - try await dispatcher.execute(review) - XCTAssertEqual(reminders.created.count, 1, "the reminder is created exactly once on commit") - XCTAssertEqual(reminders.created.first?.title, "Email Sam") - } - - func testReminderDeclineYieldsNoAction() async throws { - let stub = StubLLMRuntime(structuredScript: .valid( - json: #"{"applicable":false,"reason":"No task described"}"#)) - let reminders = FakeReminderSink() - let dispatcher = makeDispatcher(stub: stub, reminders: reminders) - let review = await dispatcher.prepare(.addToReminder, resolvedPrompt: "nice weather", - source: TaskSource(), reasoning: false) - guard case .declined = review else { return XCTFail("no-task text should decline, got \(review)") } - try await dispatcher.execute(review) - XCTAssertTrue(reminders.created.isEmpty, "a declined reminder fires no side effect") - } - - // MARK: - Contacts: schema-valid action routes to the contact sink on commit - - func testContactProducesActionAndExecutesToSink() async throws { - let stub = StubLLMRuntime(structuredScript: .valid( - json: #"{"applicable":true,"name":"Sam Rivera","email":"sam@example.com","phone":"+1 555 0100","organization":"Acme"}"#)) - let contacts = FakeContactSink() - let dispatcher = makeDispatcher(stub: stub, contacts: contacts) - let review = await dispatcher.prepare(.newContact, resolvedPrompt: "Sam Rivera, Acme, sam@example.com", - source: TaskSource(), reasoning: false) - guard case let .action(title, fields, _) = review else { - return XCTFail("a signature should yield an action, got \(review)") - } - XCTAssertEqual(title, "New Contact") - XCTAssertEqual(fields.first(where: { $0.label == "Name" })?.value, "Sam Rivera") - XCTAssertEqual(fields.first(where: { $0.label == "Email" })?.value, "sam@example.com") - - try await dispatcher.execute(review) - XCTAssertEqual(contacts.created.count, 1, "the contact is created exactly once on commit") - XCTAssertEqual(contacts.created.first?.organization, "Acme") - } - - func testContactDeclineWhenNoDetails() async throws { - let stub = StubLLMRuntime(structuredScript: .valid( - json: #"{"applicable":false,"reason":"No contact details"}"#)) - let contacts = FakeContactSink() - let dispatcher = makeDispatcher(stub: stub, contacts: contacts) - let review = await dispatcher.prepare(.newContact, resolvedPrompt: "let's meet sometime", - source: TaskSource(), reasoning: false) - guard case .declined = review else { return XCTFail("no-detail text should decline, got \(review)") } - try await dispatcher.execute(review) - XCTAssertTrue(contacts.created.isEmpty, "a declined contact fires no side effect") - } - - // MARK: - Calendar: model declines (typed) → no action - - func testCalendarTypedDeclineYieldsNoAction() async throws { - let stub = StubLLMRuntime(structuredScript: .decline(reason: "This text is not a meeting")) - let dispatcher = makeDispatcher(stub: stub) - let review = await dispatcher.prepare(.addToCalendar, resolvedPrompt: "just a thought", - source: TaskSource(), reasoning: false) - guard case let .declined(reason) = review else { - return XCTFail("a non-meeting text should decline, got \(review)") - } - XCTAssertEqual(reason, "This text is not a meeting") - } - - // MARK: - Calendar: in-payload applicable:false affordance → decline - - func testCalendarApplicableFalseAffordanceYieldsDecline() async throws { - // A schema-VALID payload that explicitly marks itself not applicable — honored like a decline. - let stub = StubLLMRuntime(structuredScript: .valid( - json: #"{"applicable":false,"reason":"No meeting described"}"#)) - let dispatcher = makeDispatcher(stub: stub) - let review = await dispatcher.prepare(.addToCalendar, resolvedPrompt: "groceries", - source: TaskSource(), reasoning: false) - guard case let .declined(reason) = review else { - return XCTFail("applicable:false should decline, got \(review)") - } - XCTAssertEqual(reason, "No meeting described") - } - - // MARK: - Non-conforming output is repaired, then succeeds - - func testNonConformingRepairedThenSucceeds() async throws { - // First emission misses the required `applicable` key → validation fails → repair succeeds. - let stub = StubLLMRuntime(structuredScript: .invalidThenRepaired( - bad: #"{"title":"Sync"}"#, - good: #"{"applicable":true,"title":"Sync","start":"2026-06-09T15:00"}"#)) - let dispatcher = makeDispatcher(stub: stub) - let review = await dispatcher.prepare(.addToCalendar, resolvedPrompt: "p", source: TaskSource(), reasoning: false) - XCTAssertTrue(review.isAction, "the repaired emission yields a valid action") - XCTAssertEqual(stub.lastAttemptCount, 2, "one repair attempt converged") - } - - // MARK: - Persistently invalid → unavailable (NO malformed side effect) - - func testPersistentlyInvalidYieldsUnavailableNotMalformedAction() async throws { - let stub = StubLLMRuntime(structuredScript: .alwaysInvalid(json: #"{"title":"Sync"}"#), - maxRepairAttempts: 3) - let calendar = FakeCalendarSink() - let dispatcher = makeDispatcher(stub: stub, calendar: calendar) - let review = await dispatcher.prepare(.addToCalendar, resolvedPrompt: "p", source: TaskSource(), reasoning: false) - guard case .unavailable = review else { - return XCTFail("persistently invalid output must be .unavailable, got \(review)") - } - // Executing an unavailable review fires nothing. - try await dispatcher.execute(review) - XCTAssertTrue(calendar.created.isEmpty, "no event is created for an unavailable review") - } - - // MARK: - execute fires the side effect ONLY for a confirmed .action - - func testExecuteFiresCalendarSinkForConfirmedAction() async throws { - let stub = StubLLMRuntime(structuredScript: .valid( - json: #"{"applicable":true,"title":"Sync","start":"2026-06-09T15:00"}"#)) - let calendar = FakeCalendarSink() - let dispatcher = makeDispatcher(stub: stub, calendar: calendar) - let review = await dispatcher.prepare(.addToCalendar, resolvedPrompt: "p", source: TaskSource(), reasoning: false) - - XCTAssertTrue(calendar.created.isEmpty, "prepare alone fires no side effect") - try await dispatcher.execute(review) - XCTAssertEqual(calendar.created.count, 1, "execute creates the event for a confirmed action") - XCTAssertEqual(calendar.created.first?.title, "Sync") - } - - func testExecuteIsNoOpForDeclinedAndUnavailable() async throws { - let stub = StubLLMRuntime(structuredScript: .valid(json: #"{"applicable":true,"title":"x"}"#)) - let calendar = FakeCalendarSink() - let dispatcher = makeDispatcher(stub: stub, calendar: calendar) - try await dispatcher.execute(.declined(reason: "no")) - try await dispatcher.execute(.unavailable(reason: "no")) - XCTAssertTrue(calendar.created.isEmpty, "execute is a no-op for non-action reviews") - } - - // MARK: - Save to project: routes to the store with project + source - - func testSaveToProjectRoutesToStore() async throws { - let stub = StubLLMRuntime(structuredScript: .valid( - json: #"{"applicable":true,"content":"a refined note"}"#)) - let projects = FakeProjectStore() - let dispatcher = makeDispatcher(stub: stub, projects: projects) - let source = TaskSource(appName: "Notes", url: URL(string: "https://x.test"), - timestamp: Date(timeIntervalSince1970: 1_700_000_000)) - let review = await dispatcher.prepare(.saveToProject(project: "Roadmap"), - resolvedPrompt: "save this", source: source, reasoning: false) - guard case let .action(_, fields, _) = review else { - return XCTFail("save-to-project should yield an action, got \(review)") - } - XCTAssertEqual(fields.first(where: { $0.label == "Project" })?.value, "Roadmap") - - try await dispatcher.execute(review) - let routed = try XCTUnwrap(projects.appended.first) - XCTAssertEqual(routed.project, "Roadmap") - XCTAssertEqual(routed.content, "a refined note") - XCTAssertEqual(routed.source.appName, "Notes") - } - - // MARK: - Open tool: routes to the opener with the generated payload - - func testOpenToolRoutesToOpener() async throws { - let stub = StubLLMRuntime(structuredScript: .valid( - json: #"{"applicable":true,"payload":"generated prompt"}"#)) - let tools = FakeToolOpener() - let dispatcher = makeDispatcher(stub: stub, tools: tools) - let review = await dispatcher.prepare(.openToolWithPayload(tool: "MyTool"), - resolvedPrompt: "idea", source: TaskSource(), reasoning: false) - try await dispatcher.execute(review) - let routed = try XCTUnwrap(tools.opened.first) - XCTAssertEqual(routed.tool, "MyTool") - XCTAssertEqual(routed.payload, "generated prompt") - } - - // MARK: - Send to: routes to the sender with the (refined) content - - func testSendToRoutesToSender() async throws { - let stub = StubLLMRuntime(structuredScript: .valid( - json: #"{"applicable":true,"content":"the refined message"}"#)) - let senders = FakeDestinationSender() - let dispatcher = makeDispatcher(stub: stub, senders: senders) - let review = await dispatcher.prepare(.sendTo(.shortcut(name: "Log")), - resolvedPrompt: "msg", source: TaskSource(), reasoning: false) - try await dispatcher.execute(review) - let routed = try XCTUnwrap(senders.sent.first) - XCTAssertEqual(routed.destination, .shortcut(name: "Log")) - XCTAssertEqual(routed.content, "the refined message") - } - - // MARK: - Each kind: a clean decline path - - func testEveryKindCanDecline() async throws { - let kinds: [TaskKind] = [ - .addToCalendar, - .saveToProject(project: "P"), - .openToolWithPayload(tool: "T"), - .sendTo(.urlScheme("x://{content}")) - ] - for kind in kinds { - let stub = StubLLMRuntime(structuredScript: .decline(reason: "not applicable")) - let dispatcher = makeDispatcher(stub: stub) - let review = await dispatcher.prepare(kind, resolvedPrompt: "p", source: TaskSource(), reasoning: false) - guard case .declined = review else { - return XCTFail("\(kind) should be able to decline, got \(review)") - } - } - } - - // MARK: - Save-to-project against the REAL on-disk store (temp dir) - - func testDiskProjectStoreAppendsToTempDirNote() async throws { - let dir = URL(fileURLWithPath: NSTemporaryDirectory()) - .appendingPathComponent("tfs-projects-\(UUID().uuidString)", isDirectory: true) - defer { try? FileManager.default.removeItem(at: dir) } - let store = DiskProjectStore(directory: dir) - - let source = TaskSource(appName: "Safari", url: URL(string: "https://example.test"), - timestamp: Date(timeIntervalSince1970: 1_700_000_000)) - try store.append(project: "Roadmap", content: "first note", source: source) - try store.append(project: "Roadmap", content: "second note", source: source) - - let url = store.noteURL(for: "Roadmap") - let text = try String(contentsOf: url, encoding: .utf8) - XCTAssertTrue(text.contains("first note"), "the first content is appended") - XCTAssertTrue(text.contains("second note"), "the second content is appended after the first") - XCTAssertTrue(text.contains("Safari"), "the source app is recorded") - XCTAssertTrue(text.contains("https://example.test"), "the source URL is recorded") - // Ordering: first note appears before second (append, not overwrite). - let firstRange = try XCTUnwrap(text.range(of: "first note")) - let secondRange = try XCTUnwrap(text.range(of: "second note")) - XCTAssertTrue(firstRange.lowerBound < secondRange.lowerBound, "content is appended in order") - } - - // MARK: - Pure helpers - - func testDiskProjectStoreFileNameSanitizes() { - XCTAssertEqual(DiskProjectStore.fileName(for: "Roadmap"), "Roadmap.md") - XCTAssertEqual(DiskProjectStore.fileName(for: "a/b:c"), "a-b-c.md") - XCTAssertEqual(DiskProjectStore.fileName(for: " "), "project.md", "empty/blank falls back") - } - - func testEntryBlockCarriesSourceAndTimestamp() { - let source = TaskSource(appName: "Mail", url: URL(string: "mailto:x"), - timestamp: Date(timeIntervalSince1970: 0)) - let block = DiskProjectStore.entryBlock(content: "hi", source: source) - XCTAssertTrue(block.contains("hi")) - XCTAssertTrue(block.contains("from Mail")) - XCTAssertTrue(block.contains("1970"), "the ISO timestamp is present") - } - - func testUrlSchemeSubstitution() { - XCTAssertEqual( - AdapterDestinationSender.substitute("a b", into: "x://note?text={content}"), - "x://note?text=a%20b", "content is substituted + percent-encoded into the placeholder") - XCTAssertEqual( - AdapterDestinationSender.substitute("z", into: "x://note?text="), - "x://note?text=z", "with no placeholder, content is appended") - } - - func testCalendarDateParsing() { - XCTAssertNotNil(EventKitCalendarSink.parseDate("2026-06-09T15:00"), "local no-tz form parses") - XCTAssertNotNil(EventKitCalendarSink.parseDate("2026-06-09T15:00:00Z"), "ISO with tz parses") - XCTAssertNil(EventKitCalendarSink.parseDate(nil)) - XCTAssertNil(EventKitCalendarSink.parseDate("not a date")) - } - - // MARK: - TaskError carries a human-facing message (guards the raw-enum-string regression) - - func testCalendarPermissionDeniedHasHumanFacingDescription() { - let description = TaskError.calendarPermissionDenied.errorDescription - XCTAssertNotNil(description, "the error carries a localized description, not just an enum case") - XCTAssertTrue(description?.contains("Calendar") ?? false, - "the message mentions Calendar (never the raw 'calendarPermissionDenied')") - XCTAssertNotEqual(description, "calendarPermissionDenied", - "the message is human-facing, not the raw enum case name") - } - - // MARK: - Honesty (D5): a failed tool open is surfaced, not swallowed - - func testWorkspaceToolOpenerSurfacesOpenFailure() async { - // An injected open handler that fails (stands in for a failed NSWorkspace.open / non-zero - // `shortcuts run` exit) must propagate as a clean TaskError — never a silent success. - let opener = WorkspaceToolOpener(openHandler: { _, _ in - throw TaskError.sinkFailed("Could not open “MyApp”.") - }) - do { - try await opener.open(tool: "MyApp.app", payload: "hello") - XCTFail("a failed open must throw, not silently succeed") - } catch let e as TaskError { - guard case let .sinkFailed(message) = e else { - return XCTFail("expected .sinkFailed, got \(e)") - } - XCTAssertEqual(message, "Could not open “MyApp”.") - } catch { - XCTFail("unexpected error: \(error)") - } - } - - func testWorkspaceToolOpenerSucceedsWhenOpenLands() async throws { - var openedWith: (tool: String, file: URL)? - let opener = WorkspaceToolOpener(openHandler: { tool, file in openedWith = (tool, file) }) - try await opener.open(tool: "MyTool", payload: "payload-text") - let landed = try XCTUnwrap(openedWith) - XCTAssertEqual(landed.tool, "MyTool") - XCTAssertTrue(landed.file.lastPathComponent.hasPrefix("tfs-payload-"), "a payload file was written") - } - - // MARK: - in-payload applicable:false decline validates for save-to-project (Fix 4) - - func testSaveToProjectApplicableFalseAffordanceYieldsDecline() async throws { - // A schema-VALID payload (under the loosened required:["applicable"] schema) that explicitly - // marks itself not applicable — must be honored as a decline, not failed/unavailable. - let stub = StubLLMRuntime(structuredScript: .valid( - json: #"{"applicable":false,"reason":"Nothing to save here"}"#)) - let projects = FakeProjectStore() - let dispatcher = makeDispatcher(stub: stub, projects: projects) - let review = await dispatcher.prepare(.saveToProject(project: "Roadmap"), - resolvedPrompt: "groceries", source: TaskSource(), reasoning: false) - guard case let .declined(reason) = review else { - return XCTFail("applicable:false should decline (not .unavailable), got \(review)") - } - XCTAssertEqual(reason, "Nothing to save here") - try await dispatcher.execute(review) - XCTAssertTrue(projects.appended.isEmpty, "a declined save fires no side effect") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/ToolRoutingTests.swift b/Tests/ThreeFingerSwitcherTests/ToolRoutingTests.swift deleted file mode 100644 index 466438f..0000000 --- a/Tests/ThreeFingerSwitcherTests/ToolRoutingTests.swift +++ /dev/null @@ -1,372 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// Tests for the model-driven tool-routing slice (`ai-tool-routing`, tasks §2–§6): the `structured()` -/// router's outcome mapping, registry aggregation/dispatch, the bridge into the UNCHANGED task -/// machinery (with the approval gate), candidate ranking, and the bounded route→execute→continue loop -/// (plain-answer, multi-hop, cap, loop-guards, widen, cancel). All driven by scripted fakes — no model. -@MainActor -final class ToolRoutingTests: XCTestCase { - - // MARK: - Fakes - - /// A runtime whose `structured` dequeues scripted route JSON per call and whose `generate` streams a - /// scripted final answer — so a multi-hop loop is fully deterministic. - private final class RoutingRuntime: LLMRuntime, @unchecked Sendable { - let capabilities: Set = [.text] - private var routes: [String] - private let answerTokens: [String] - private let lock = NSLock() - init(routes: [String], answer: [String] = ["the answer"]) { self.routes = routes; self.answerTokens = answer } - - func generate(_ request: LLMRequest) -> AsyncThrowingStream { - let toks = answerTokens - return AsyncThrowingStream { c in - let task = Task { - for (i, t) in toks.enumerated() { - if Task.isCancelled { c.finish(throwing: RuntimeError.cancelled); return } - c.yield(Token(t, isFinal: i == toks.count - 1)) - } - c.finish() - } - c.onTermination = { _ in task.cancel() } - } - } - - func structured(_ request: LLMRequest, schema: StructuredSchema, - as type: T.Type) async throws -> StructuredOutcome { - let json: String = { lock.lock(); defer { lock.unlock() }; return routes.isEmpty ? "{\"tool\":\"\"}" : routes.removeFirst() }() - let decoded = try JSONDecoder().decode(T.self, from: Data(json.utf8)) - return .value(decoded) - } - } - - /// A contributor returning scripted step results by tool name (loop tests, decoupled from TaskKind). - private struct FakeContributor: ToolContributor { - let descriptorsList: [ToolDescriptor] - var results: [String: ToolStepResult] = [:] - func descriptors() -> [ToolDescriptor] { descriptorsList } - func canHandle(_ tool: String) -> Bool { descriptorsList.contains { $0.name == tool } } - func run(_ call: RoutedCall, gate: ApprovalGate) async -> ToolStepResult { - results[call.descriptor.name] - ?? ToolStepResult(tool: call.descriptor.name, status: .done, summary: "did \(call.descriptor.name)") - } - } - - private final class ScriptedApprovalGate: ApprovalGate, @unchecked Sendable { - private var decisions: [ApprovalDecision] - private let lock = NSLock() - init(_ decisions: [ApprovalDecision] = []) { self.decisions = decisions } - func awaitDecision(for review: TaskReview) async -> ApprovalDecision { - lock.lock(); defer { lock.unlock() } - return decisions.isEmpty ? .approve : decisions.removeFirst() - } - } - - /// A fake `TaskDispatching` (bridge tests): returns a scripted review and records executes. - private final class FakeDispatcher: TaskDispatching { - var reviewToReturn: TaskReview = .action(title: "Event", - fields: [ReviewField("Title", "Lunch")], - payload: .openTool(tool: "x", - action: ParsedOpenTool(applicable: true, reason: nil, payload: "p"))) - var executeError: Error? - private(set) var prepared: [(TaskKind, String)] = [] - private(set) var executed = 0 - func prepare(_ kind: TaskKind, resolvedPrompt: String, source: TaskSource, reasoning: Bool) async -> TaskReview { - prepared.append((kind, resolvedPrompt)); return reviewToReturn - } - func execute(_ review: TaskReview) async throws { - if let executeError { throw executeError } - executed += 1 - } - } - - private struct AutoResolver: WritePolicyResolving { - func effectiveTier(for descriptor: ToolDescriptor) -> WritePolicyTier { .auto } - } - - private func descriptor(_ name: String, keywords: [String] = [], tier: WritePolicyTier = .confirm) -> ToolDescriptor { - ToolDescriptor(name: name, summary: "tool \(name)", - argsSchema: StructuredSchema(name: name, json: "{\"type\":\"object\"}"), - writePolicy: tier, keywords: keywords) - } - - private func ctx(_ userText: String, allowed: [String] = []) -> RouteContext { - RouteContext(messages: [AgentMessage(role: .user, text: userText)], allowedTools: allowed) - } - - // MARK: - Router outcome mapping - - func testRouterPlainAnswerOnEmptyTool() async { - let rt = StubLLMRuntime(structuredScript: .valid(json: "{\"tool\":\"\"}")) - let r = await ToolRouter.route(context: ctx("hi"), candidates: [descriptor("add_to_calendar")], runtime: rt, reasoning: false) - guard case let .route(route) = r else { return XCTFail("expected .route") } - XCTAssertTrue(route.isPlainAnswer) - } - - func testRouterRoutesToMatchingCandidate() async { - let rt = StubLLMRuntime(structuredScript: .valid(json: "{\"tool\":\"add_to_calendar\",\"argumentsJSON\":\"{}\",\"rationale\":\"a meeting\"}")) - let r = await ToolRouter.route(context: ctx("lunch tomorrow"), candidates: [descriptor("add_to_calendar")], runtime: rt, reasoning: false) - guard case let .route(route) = r else { return XCTFail("expected .route") } - XCTAssertEqual(route.tool, "add_to_calendar") - XCTAssertEqual(route.rationale, "a meeting") - } - - func testRouterUnknownToolDegradesToPlainAnswer() async { - let rt = StubLLMRuntime(structuredScript: .valid(json: "{\"tool\":\"nonexistent\"}")) - let r = await ToolRouter.route(context: ctx("x"), candidates: [descriptor("add_to_calendar")], runtime: rt, reasoning: false) - guard case let .route(route) = r else { return XCTFail("expected .route") } - XCTAssertTrue(route.isPlainAnswer, "a tool not in the candidate set degrades to a plain answer, never dispatched") - } - - func testRouterDeclineIsPlainAnswer() async { - let rt = StubLLMRuntime(structuredScript: .decline(reason: "no tool fits")) - let r = await ToolRouter.route(context: ctx("x"), candidates: [descriptor("add_to_calendar")], runtime: rt, reasoning: false) - guard case let .route(route) = r else { return XCTFail("expected .route") } - XCTAssertTrue(route.isPlainAnswer, "a decline is the first-class 'just talk' plain answer") - } - - func testRouterCouldNotProduceValidFallsBackToPlainAnswer() async { - let rt = StubLLMRuntime(structuredScript: .alwaysInvalid(json: "{}"), maxRepairAttempts: 2) // no "tool" key → exhausts - let r = await ToolRouter.route(context: ctx("x"), candidates: [descriptor("a")], runtime: rt, reasoning: false) - guard case let .route(route) = r else { return XCTFail("expected .route, not a failure") } - XCTAssertTrue(route.isPlainAnswer, "a malformed route is never a fabricated tool call") - } - - // MARK: - Registry - - func testRegistryAggregatesAndDedupesByNameFirstWins() { - let a = FakeContributor(descriptorsList: [descriptor("x"), descriptor("y")]) - let b = FakeContributor(descriptorsList: [descriptor("y"), descriptor("z")]) - let registry = ToolRegistry([a, b]) - XCTAssertEqual(registry.allDescriptors().map(\.name), ["x", "y", "z"], "deduped by name, first contributor wins") - } - - func testRegistryDispatchesToOwningContributorAndUnknownIsDefensiveFailed() async { - let a = FakeContributor(descriptorsList: [descriptor("x")], - results: ["x": ToolStepResult(tool: "x", status: .done, summary: "ran x")]) - let registry = ToolRegistry([a]) - let okCall = RoutedCall(descriptor: descriptor("x"), route: ToolRoute(tool: "x"), userText: "", source: TaskSource()) - let ok = await registry.run(okCall, gate: ScriptedApprovalGate()) - XCTAssertEqual(ok.summary, "ran x") - - let unknownCall = RoutedCall(descriptor: descriptor("ghost"), route: ToolRoute(tool: "ghost"), userText: "", source: TaskSource()) - let unknown = await registry.run(unknownCall, gate: ScriptedApprovalGate()) - guard case .failed = unknown.status else { return XCTFail("an unowned tool returns a defensive .failed") } - } - - // MARK: - Bridge to the task machinery - - private func bridgeCall(_ kind: TaskKind) -> RoutedCall { - RoutedCall(descriptor: TaskKindToolContributor.descriptor(for: kind), - route: ToolRoute(tool: TaskKindToolContributor.name(for: kind), argumentsJSON: "{}"), - userText: "lunch with sam tomorrow", source: TaskSource()) - } - - func testBridgeAutoTierExecutesImmediatelyWithoutGate() async { - let dispatcher = FakeDispatcher() - let c = TaskKindToolContributor(dispatcher: dispatcher, resolver: AutoResolver(), kinds: [.addToCalendar]) - let result = await c.run(bridgeCall(.addToCalendar), gate: ScriptedApprovalGate([.skip])) // gate must NOT be consulted - XCTAssertEqual(result.status, .done) - XCTAssertEqual(dispatcher.executed, 1, "an auto-tier action fires immediately") - } - - func testBridgeConfirmApproveExecutes() async { - let dispatcher = FakeDispatcher() - let c = TaskKindToolContributor(dispatcher: dispatcher, kinds: [.addToCalendar]) // default resolver → .confirm - let result = await c.run(bridgeCall(.addToCalendar), gate: ScriptedApprovalGate([.approve])) - XCTAssertEqual(result.status, .done) - XCTAssertEqual(dispatcher.executed, 1) - } - - func testBridgeConfirmSkipDeclinesAndFiresNothing() async { - let dispatcher = FakeDispatcher() - let c = TaskKindToolContributor(dispatcher: dispatcher, kinds: [.addToCalendar]) - let result = await c.run(bridgeCall(.addToCalendar), gate: ScriptedApprovalGate([.skip])) - XCTAssertEqual(result.status, .declined(reason: "skipped")) - XCTAssertEqual(dispatcher.executed, 0, "a skipped step fires no side effect") - } - - func testBridgeDeclinedReviewMapsToDeclined() async { - let dispatcher = FakeDispatcher() - dispatcher.reviewToReturn = .declined(reason: "not a meeting") - let c = TaskKindToolContributor(dispatcher: dispatcher, kinds: [.addToCalendar]) - let result = await c.run(bridgeCall(.addToCalendar), gate: ScriptedApprovalGate([.approve])) - XCTAssertEqual(result.status, .declined(reason: "not a meeting")) - XCTAssertEqual(dispatcher.executed, 0) - } - - func testBridgeUnavailableMapsToFailed() async { - let dispatcher = FakeDispatcher() - dispatcher.reviewToReturn = .unavailable(reason: "Couldn't produce a valid action.") - let c = TaskKindToolContributor(dispatcher: dispatcher, kinds: [.addToCalendar]) - let result = await c.run(bridgeCall(.addToCalendar), gate: ScriptedApprovalGate([.approve])) - XCTAssertEqual(result.status, .failed(headline: "Couldn't produce a valid action.")) - } - - func testBridgeExecuteThrowMapsToFailedNotFalseDone() async { - let dispatcher = FakeDispatcher() - dispatcher.executeError = TaskError.calendarPermissionDenied - let c = TaskKindToolContributor(dispatcher: dispatcher, kinds: [.addToCalendar]) - let result = await c.run(bridgeCall(.addToCalendar), gate: ScriptedApprovalGate([.approve])) - guard case let .failed(headline) = result.status else { return XCTFail("a thrown sink is .failed, never a false Done") } - XCTAssertTrue(headline.contains("Calendar"), "the clean TaskError headline, not a raw dump") - } - - func testBridgeConfiguredTaskBindsConfigInDescriptorName() { - let save = TaskKindToolContributor.name(for: .saveToProject(project: "Roadmap")) - XCTAssertEqual(save, "save_to_project:Roadmap", "config is bound in the descriptor identity (Decision Q2)") - let c = TaskKindToolContributor(dispatcher: FakeDispatcher(), kinds: [.saveToProject(project: "Roadmap")]) - XCTAssertTrue(c.canHandle("save_to_project:Roadmap")) - XCTAssertFalse(c.canHandle("save_to_project:Other")) - } - - // MARK: - Candidate retrieval - - func testCandidateRankingPrefersKeywordMatch() { - let source = KeywordToolCandidateSource(all: [ - descriptor("weather", keywords: ["weather", "forecast"]), - descriptor("calendar", keywords: ["meeting", "calendar", "schedule"]), - ]) - let result = source.candidates(for: ctx("schedule a meeting"), limit: 5) - XCTAssertEqual(result.first?.name, "calendar", "the keyword-matching tool ranks first") - } - - func testCandidateAlwaysIncludesAllowedTools() { - let source = KeywordToolCandidateSource(all: [ - descriptor("weather", keywords: ["weather"]), - descriptor("special", keywords: ["zzz"]), - ]) - let result = source.candidates(for: ctx("nothing relevant", allowed: ["special"]), limit: 1) - XCTAssertTrue(result.contains { $0.name == "special" }, "the active skill's allowed tool is always offered") - } - - func testCandidateRespectsAdditiveCap() { - let many = (0..<20).map { descriptor("t\($0)") } - let source = KeywordToolCandidateSource(all: many, additiveCap: 8) - XCTAssertLessThanOrEqual(source.candidates(for: ctx("x"), limit: 100).count, 8) - } - - // MARK: - The bounded loop - - private func loop(_ rt: RoutingRuntime, contributor: FakeContributor, gate: ApprovalGate = ScriptedApprovalGate(), - maxToolSteps: Int = 8, onThinking: @escaping @Sendable (String) -> Void = { _ in }) -> AgentLoop { - AgentLoop(runtime: rt, registry: ToolRegistry([contributor]), - candidateSource: KeywordToolCandidateSource(all: contributor.descriptorsList), - gate: gate, reasoning: false, maxToolSteps: maxToolSteps, onThinking: onThinking) - } - - func testLoopPlainAnswerOneShotZeroSteps() async { - let rt = RoutingRuntime(routes: ["{\"tool\":\"\"}"], answer: ["Hello!"]) - let result = await loop(rt, contributor: FakeContributor(descriptorsList: [descriptor("a")])).run(context: ctx("hi")) - XCTAssertEqual(result.outcome, .answered(text: "Hello!")) - XCTAssertTrue(result.steps.isEmpty, "a plain answer runs zero tool steps") - } - - func testLoopSingleToolThenAnswer() async { - let rt = RoutingRuntime(routes: ["{\"tool\":\"a\",\"argumentsJSON\":\"{}\"}", "{\"tool\":\"\"}"], answer: ["Done."]) - let contributor = FakeContributor(descriptorsList: [descriptor("a")], - results: ["a": ToolStepResult(tool: "a", status: .done, summary: "ran a")]) - let result = await loop(rt, contributor: contributor).run(context: ctx("do a")) - XCTAssertEqual(result.outcome, .answered(text: "Done.")) - XCTAssertEqual(result.steps.map(\.summary), ["ran a"], "one tool step ran, then the model answered") - } - - func testLoopFailedStepEndsFailed() async { - let rt = RoutingRuntime(routes: ["{\"tool\":\"a\"}"], answer: ["unused"]) - let contributor = FakeContributor(descriptorsList: [descriptor("a")], - results: ["a": ToolStepResult(tool: "a", status: .failed(headline: "Disk full."), summary: "failed a")]) - let result = await loop(rt, contributor: contributor).run(context: ctx("do a")) - XCTAssertEqual(result.outcome, .failed(headline: "Disk full."), "a side effect that didn't land ends the loop failed") - } - - func testLoopRepeatedStepGuardStops() async { - // The same tool+args twice in a row → repeatedStep guard. - let rt = RoutingRuntime(routes: ["{\"tool\":\"a\",\"argumentsJSON\":\"{}\"}", "{\"tool\":\"a\",\"argumentsJSON\":\"{}\"}"], answer: ["best effort"]) - let contributor = FakeContributor(descriptorsList: [descriptor("a")], - results: ["a": ToolStepResult(tool: "a", status: .done, summary: "ran a")]) - let result = await loop(rt, contributor: contributor).run(context: ctx("do a")) - guard case let .stopped(reason, text) = result.outcome else { return XCTFail("expected .stopped") } - XCTAssertEqual(reason, .repeatedStep) - XCTAssertEqual(text, "best effort", "a guard stop still streams a best-effort final answer") - } - - func testLoopCapReached() async { - // Two distinct auto tools, never a plain answer, cap = 2 → capReached. - let rt = RoutingRuntime(routes: ["{\"tool\":\"a\"}", "{\"tool\":\"b\"}", "{\"tool\":\"a\"}"], answer: ["capped answer"]) - let contributor = FakeContributor(descriptorsList: [descriptor("a"), descriptor("b")]) - let result = await loop(rt, contributor: contributor, maxToolSteps: 2).run(context: ctx("go")) - XCTAssertEqual(result.outcome, .capReached(text: "capped answer")) - XCTAssertEqual(result.steps.count, 2, "exactly maxToolSteps tool steps ran before the cap") - } - - func testLoopThinkingCarriesPlanResponseCarriesAnswerOnly() async { - let rt = RoutingRuntime(routes: ["{\"tool\":\"a\",\"rationale\":\"because reasons\"}", "{\"tool\":\"\"}"], answer: ["Final."]) - let contributor = FakeContributor(descriptorsList: [descriptor("a")], - results: ["a": ToolStepResult(tool: "a", status: .done, summary: "ran a")]) - let captured = ThinkingSink() - let result = await loop(rt, contributor: contributor, onThinking: { captured.append($0) }).run(context: ctx("do a")) - XCTAssertEqual(result.outcome, .answered(text: "Final."), "the committed answer is response-channel only") - let plan = captured.text - XCTAssertTrue(plan.contains("because reasons"), "the route rationale rides the thinking channel") - XCTAssertTrue(plan.contains("ran a"), "the tool-step summary rides the thinking channel") - XCTAssertFalse(plan.contains("Final."), "the final answer is NOT in the thinking plan") - } - - func testLoopWidenEnlargesNextTurnsCandidates() async { - let rt = RoutingRuntime(routes: ["{\"tool\":\"widen_candidates\"}", "{\"tool\":\"\"}"], answer: ["ok"]) - let spy = SpyCandidateSource(inner: KeywordToolCandidateSource(all: [descriptor("a")])) - let agent = AgentLoop(runtime: rt, registry: ToolRegistry([FakeContributor(descriptorsList: [descriptor("a")])]), - candidateSource: spy, gate: ScriptedApprovalGate(), maxToolSteps: 8) - let result = await agent.run(context: ctx("need more")) - XCTAssertEqual(result.outcome, .answered(text: "ok")) - XCTAssertEqual(spy.limits.count, 2, "two route turns") - XCTAssertGreaterThan(spy.limits[1], spy.limits[0], "a widen step enlarges the next turn's candidate limit") - } - - func testLoopCancellationViaGateIsQuietNotFailed() async { - let rt = RoutingRuntime(routes: ["{\"tool\":\"a\"}"], answer: ["unused"]) - let contributor = FakeContributor(descriptorsList: [descriptor("a")]) // default .confirm → consults the gate - let result = await loop(rt, contributor: contributor, gate: ScriptedApprovalGate([.cancel])).run(context: ctx("do a")) - // The contributor's FakeContributor ignores the gate; so emulate cancel via the bridge instead: - _ = result - let dispatcher = FakeDispatcher() - let bridge = TaskKindToolContributor(dispatcher: dispatcher, kinds: [.addToCalendar]) - let agent = AgentLoop(runtime: RoutingRuntime(routes: ["{\"tool\":\"add_to_calendar\"}"], answer: ["x"]), - registry: ToolRegistry([bridge]), - candidateSource: KeywordToolCandidateSource(all: bridge.descriptors()), - gate: ScriptedApprovalGate([.cancel])) - let r = await agent.run(context: ctx("lunch tomorrow")) - XCTAssertEqual(r.outcome, .stopped(reason: .cancelled, text: ""), "a gate cancel ends the loop quietly") - XCTAssertEqual(dispatcher.executed, 0, "cancel fires no side effect") - } - - /// `refactor-park-and-background-agents`: the terminal "task complete" classification was RETIRED — - /// a settled outcome never marks a session for removal. The outcome cases remain distinct values - /// (the engine maps them to settlements; the paused case never fabricates text). - func testAgentLoopOutcomeCasesStayDistinct() { - XCTAssertNotEqual(AgentLoopOutcome.answered(text: "x"), .capReached(text: "x")) - XCTAssertNotEqual(AgentLoopOutcome.stopped(reason: .cancelled, text: ""), .pausedAwaitingUser) - XCTAssertNotEqual(AgentLoopOutcome.pausedAwaitingUser, .failed(headline: "x")) - } - - // MARK: - Test helpers - - private final class ThinkingSink: @unchecked Sendable { - private var buf = "" - private let lock = NSLock() - func append(_ s: String) { lock.lock(); buf += s; lock.unlock() } - var text: String { lock.lock(); defer { lock.unlock() }; return buf } - } - - private final class SpyCandidateSource: ToolCandidateSource, @unchecked Sendable { - let inner: ToolCandidateSource - private(set) var limits: [Int] = [] - private let lock = NSLock() - init(inner: ToolCandidateSource) { self.inner = inner } - func candidates(for context: RouteContext, limit: Int) -> [ToolDescriptor] { - lock.lock(); limits.append(limit); lock.unlock() - return inner.candidates(for: context, limit: limit) - } - } -} diff --git a/Tests/ThreeFingerSwitcherTests/VideoAnimationTests.swift b/Tests/ThreeFingerSwitcherTests/VideoAnimationTests.swift deleted file mode 100644 index 9dd3afa..0000000 --- a/Tests/ThreeFingerSwitcherTests/VideoAnimationTests.swift +++ /dev/null @@ -1,418 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// `ai-video-animation-generation` — the VIDEO backend behind the `MediaRuntime` seam: the `VideoProvider` -/// selector + master-gated validity, the `VideoUploadDisclosure` redaction, the rolling-24h `VideoBudget` -/// (relaunch-surviving ledger + refund + degrade-not-drop), the effective-tier + over-budget degrade -/// resolution, the audit-per-attempt with redacted summary, the two scripted video stubs (progress → -/// finished video asset + img2video seed), parking, the error taxonomy, and the swap-in contract. All -/// MLX-/network-free Core, driven against the scripted stubs (no weights, no hosted API). -final class VideoAnimationTests: XCTestCase { - - // MARK: - Helpers - - private func png() -> Data { Data(MediaSeedValidation.pngMagic + [0x01, 0x02, 0x03]) } - - private func videoAsset() -> MediaAsset { - MediaAsset.video(url: FileManager.default.temporaryDirectory.appendingPathComponent("clip.mp4"), - width: 1024, height: 576, durationMs: 4000) - } - - // MARK: - 1.1 VideoProvider - - func testVideoProviderDefaultIsCloud() { - XCTAssertEqual(VideoProvider.defaultProvider, .cloud) - XCTAssertTrue(VideoProvider.cloud.isCloud) - XCTAssertFalse(VideoProvider.localLTXV.isCloud) - } - - func testVideoProviderCodableRoundTrip() throws { - for p in VideoProvider.allCases { - let data = try JSONEncoder().encode(p) - let back = try JSONDecoder().decode(VideoProvider.self, from: data) - XCTAssertEqual(back, p) - } - } - - func testLocalProviderRequiresMasterToggle() { - // Cloud is always selectable; local needs BOTH master flags ON. - XCTAssertTrue(VideoProvider.cloud.isSelectable(fullPotentialEnabled: false, mediaGenEnabled: false)) - XCTAssertFalse(VideoProvider.localLTXV.isSelectable(fullPotentialEnabled: false, mediaGenEnabled: true)) - XCTAssertFalse(VideoProvider.localLTXV.isSelectable(fullPotentialEnabled: true, mediaGenEnabled: false)) - XCTAssertTrue(VideoProvider.localLTXV.isSelectable(fullPotentialEnabled: true, mediaGenEnabled: true)) - } - - func testResolvedFallsBackToCloudWhenLocalForbidden() { - // A persisted `.localLTXV` with the master off degrades to the calm `.cloud` default. - let r = VideoProvider.resolved(rawValue: "localLTXV", fullPotentialEnabled: false, mediaGenEnabled: false) - XCTAssertEqual(r, .cloud) - // With the master on, the stored choice is honored. - let r2 = VideoProvider.resolved(rawValue: "localLTXV", fullPotentialEnabled: true, mediaGenEnabled: true) - XCTAssertEqual(r2, .localLTXV) - // An absent/garbage value resolves to the default. - XCTAssertEqual(VideoProvider.resolved(rawValue: nil, fullPotentialEnabled: true, mediaGenEnabled: true), .cloud) - XCTAssertEqual(VideoProvider.resolved(rawValue: "bogus", fullPotentialEnabled: true, mediaGenEnabled: true), .cloud) - } - - // MARK: - 1.2 / 7.x VideoUploadDisclosure - - func testCloudDisclosureFlagsUploadAndRedactsPrompt() { - let secretPrompt = "make a clip token=SUPERSECRETVALUE123 of a sunset" - let d = VideoUploadDisclosure.make(provider: .cloud, prompt: secretPrompt, seedPresent: true) - XCTAssertTrue(d.bytesLeaveDevice) - XCTAssertTrue(d.seedPresent) - // The full prompt + the raw secret never appear in the audit summary. - XCTAssertFalse(d.auditSummary.contains("SUPERSECRETVALUE123"), "raw secret leaked into summary") - XCTAssertFalse(d.auditSummary.contains(secretPrompt), "full prompt leaked into summary") - XCTAssertTrue(d.auditSummary.contains("cloud")) - XCTAssertTrue(d.auditSummary.contains("seed:yes")) - } - - func testLocalDisclosureDoesNotClaimUpload() { - let d = VideoUploadDisclosure.make(provider: .localLTXV, prompt: "a dog running", seedPresent: false) - XCTAssertFalse(d.bytesLeaveDevice) - XCTAssertFalse(d.seedPresent) - XCTAssertTrue(d.auditSummary.contains("localLTXV")) - XCTAssertTrue(d.auditSummary.contains("seed:no")) - } - - func testCloudConfirmLineStatesUploadAndCost() { - let d = VideoUploadDisclosure.make(provider: .cloud, prompt: "x", seedPresent: true) - let line = d.cloudConfirmLine(perClipCostOrder: "~$0.20") - XCTAssertTrue(line.contains("remote video service")) - XCTAssertTrue(line.contains("~$0.20")) - XCTAssertTrue(line.contains("source image"), "seed-present run mentions the uploaded image") - } - - func testLocalCostLineDisclosesResidencyLatencyEvictionThermal() { - let line = VideoUploadDisclosure.localCostLine - XCTAssertTrue(line.contains("gigabytes")) // residency - XCTAssertTrue(line.contains("minutes")) // latency - XCTAssertTrue(line.contains("evicted")) // chat eviction - XCTAssertTrue(line.lowercased().contains("hot")) // thermal - XCTAssertTrue(line.contains("Nothing is uploaded")) - } - - // MARK: - 2.1 VideoBudget under cap - - func testBudgetUnderCapAllows() { - var b = VideoBudget(maxCallsPerWindow: 3) - let now = Date() - XCTAssertTrue(b.allows(now: now)) - b.record(at: now); b.reap() - XCTAssertTrue(b.allows(now: now)) // 1 of 3 used - } - - func testZeroCapNeverAllows() { - let b = VideoBudget(maxCallsPerWindow: 0) - XCTAssertFalse(b.allows(now: Date())) - } - - func testConcurrencyCapBlocksInFlight() { - var b = VideoBudget(maxCallsPerWindow: 5, maxConcurrent: 1) - let now = Date() - b.record(at: now) // in-flight = 1, not reaped - XCTAssertFalse(b.allows(now: now), "concurrency cap blocks a second concurrent gen") - b.reap() - XCTAssertTrue(b.allows(now: now)) - } - - // MARK: - 2.2 Rolling window, NOT calendar reset - - func testRollingWindowNotGamedAcrossMidnight() { - // Two spends straddling midnight but within 24h count TOGETHER (no calendar reset). - let cal = Calendar(identifier: .gregorian) - let base = cal.date(from: DateComponents(year: 2026, month: 6, day: 22, hour: 23, minute: 59))! - let afterMidnight = cal.date(from: DateComponents(year: 2026, month: 6, day: 23, hour: 0, minute: 1))! - var b = VideoBudget(maxCallsPerWindow: 2) - b.record(at: base); b.reap() - b.record(at: afterMidnight); b.reap() - // Both inside the rolling 24h window ending just after midnight → cap (2) reached. - XCTAssertEqual(b.callsInLast24h(afterMidnight), 2) - XCTAssertFalse(b.allows(now: afterMidnight), "midnight does NOT reset the rolling window") - // A spend OUTSIDE the window (25h later) no longer counts. - let later = afterMidnight.addingTimeInterval(25 * 60 * 60) - XCTAssertEqual(b.callsInLast24h(later), 0) - XCTAssertTrue(b.allows(now: later)) - } - - // MARK: - 2.3 Ledger survives relaunch (replay) - - func testLedgerSurvivesRelaunchWithinWindow() { - let now = Date() - let store = InMemoryVideoLedgerStore() - let first = RollingVideoBudget(cap: { 2 }, store: store) - XCTAssertTrue(first.hasRemaining(now: now)) - first.consume(now: now); first.reap() - first.consume(now: now); first.reap() - XCTAssertFalse(first.hasRemaining(now: now), "cap reached") - // A NEW box reading the SAME store replays the ledger → prior spends still count. - let relaunched = RollingVideoBudget(cap: { 2 }, store: store) - XCTAssertFalse(relaunched.hasRemaining(now: now), "relaunch does not reset the rolling cap") - XCTAssertEqual(relaunched.snapshot().callsInLast24h(now), 2) - } - - // MARK: - 2.4 Refund + degrade-not-drop - - func testFailedLaunchRefundLeavesCapUnchanged() { - let now = Date() - let b = RollingVideoBudget(cap: { 1 }) - XCTAssertTrue(b.hasRemaining(now: now)) - let at = now - b.consume(now: at) // spend recorded - XCTAssertFalse(b.hasRemaining(now: now), "cap consumed") - b.refund(at: at) // launch failed → refund - XCTAssertTrue(b.hasRemaining(now: now), "refund restores the cap") - XCTAssertEqual(b.snapshot().callsInLast24h(now), 0) - XCTAssertEqual(b.snapshot().inFlight, 0) - } - - func testOverBudgetReturnsDegradeSignalNotDrop() { - // The resolver returns a DEGRADE gate (never a silent drop) when cloud is over budget. - let resolver = VideoTierResolver(provider: .cloud) - let gate = resolver.gate(budgetHasRoom: false) - XCTAssertEqual(gate, .cloudOverBudget) - XCTAssertTrue(gate.spendsCloudBudget) - } - - func testLiveCapBumpTakesEffect() { - // A settings bump to the cap takes effect immediately (the box reads the cap live). - let now = Date() - var capValue = 1 - let b = RollingVideoBudget(cap: { capValue }) - b.consume(now: now); b.reap() - XCTAssertFalse(b.hasRemaining(now: now)) - capValue = 5 - XCTAssertTrue(b.hasRemaining(now: now), "raising the cap admits more without losing recorded spend") - XCTAssertEqual(b.snapshot().callsInLast24h(now), 1, "the existing spend is preserved") - } - - // MARK: - 3.1 Effective tier - - func testCloudVideoResolvesDangerous() { - let desc = ToolDescriptor(name: MediaTool.generateVideo, summary: "", - argsSchema: StructuredSchema(name: MediaTool.generateVideo, json: "{}"), - writePolicy: .dangerous) - let resolver = VideoTierResolver(provider: .cloud, - resolver: BackgroundPolicyResolver(whitelist: .empty)) - XCTAssertEqual(resolver.effectiveTier(for: desc), .dangerous, "cloud video is never lowered") - } - - func testLocalVideoTierIsConfirmAndMasterGated() { - let desc = ToolDescriptor(name: MediaTool.generateVideo, summary: "", - argsSchema: StructuredSchema(name: MediaTool.generateVideo, json: "{}"), - writePolicy: .dangerous) - // Local: confirm tier, off the spend axis. - let onResolver = VideoTierResolver(provider: .localLTXV, - isFullPotentialEnabled: { true }, isMediaGenEnabled: { true }) - XCTAssertEqual(onResolver.effectiveTier(for: desc), .confirm) - XCTAssertEqual(onResolver.gate(budgetHasRoom: true), .localConfirm) - // Master off → provider disabled (a settings desync degrades to disabled, never runs). - let offResolver = VideoTierResolver(provider: .localLTXV, - isFullPotentialEnabled: { false }, isMediaGenEnabled: { true }) - XCTAssertEqual(offResolver.gate(budgetHasRoom: true), .providerDisabled) - } - - // MARK: - 3.2 Over-budget degrade messages - - func testOverBudgetActiveConfirmAndParkedNeedsYouMessages() { - let active = VideoTierResolver.overBudgetReason(parked: false) - let parked = VideoTierResolver.overBudgetReason(parked: true) - XCTAssertTrue(active.contains("budget is used up")) - XCTAssertTrue(active.contains("Approve"), "active degrades to a foreground confirm") - XCTAssertTrue(parked.contains("budget is used up")) - XCTAssertNotEqual(active, parked, "parked message differs (needs-you)") - } - - // MARK: - 4.1 Audit per attempt, redacted - - func testAuditRecordCarriesRedactedSummaryNeverFullPrompt() { - let log = InMemoryAuditLog() - let sessionID = AgentSessionID() - let prompt = "animate token=LEAK_ME_NOW_PLEASE the logo spinning" - let disclosure = VideoUploadDisclosure.make(provider: .cloud, prompt: prompt, seedPresent: false) - // Emit one record per attempt with the redacted summary (the sink's pattern, exercised directly). - log.record(AuditRecord(sessionID: sessionID, tool: MediaTool.generateVideo, policy: .dangerous, - argumentsSummary: disclosure.auditSummary, outcome: .done, - wasBackground: true)) - let recent = log.recent(limit: 10) - XCTAssertEqual(recent.count, 1, "exactly one record per attempt") - let rec = recent[0] - XCTAssertTrue(rec.wasBackground, "wasBackground set when parked") - XCTAssertFalse(rec.argumentsSummary.contains("LEAK_ME_NOW_PLEASE"), "raw secret never in the summary") - XCTAssertFalse(rec.argumentsSummary.contains(prompt), "full prompt never in the summary") - XCTAssertTrue(rec.argumentsSummary.contains("cloud")) - XCTAssertTrue(rec.argumentsSummary.contains("seed:no")) - } - - // MARK: - 5.1 Stub progress → finished video asset - - func testCloudStubStreamsThenFinishesVideoAsset() async throws { - let asset = videoAsset() - let runtime = StubCloudVideoRuntime.make( - script: .successWithPreviews(count: 3, total: 3, preview: png(), asset: asset)) - XCTAssertEqual(runtime.capabilities, [.video]) - var steps = 0 - var finished: MediaAsset? - for try await p in runtime.generate(MediaRequest(prompt: "a sunrise", kind: .video)) { - switch p { - case .step: steps += 1 - case let .finished(a): finished = a - } - } - XCTAssertEqual(steps, 3) - XCTAssertEqual(finished?.kind, .video) - XCTAssertNotNil(finished?.durationMs, "a finished video asset carries durationMs") - XCTAssertEqual(finished?.durationMs, 4000) - } - - // MARK: - 5.2 img2video seed threads through + sets disclosure - - func testSeedThreadsThroughToRequestAndDisclosure() async throws { - let asset = videoAsset() - let runtime = StubCloudVideoRuntime.make(script: .success(asset)) - let req = MediaRequest(prompt: "animate this", seed: png(), kind: .video, - parameters: MediaParameters(durationMs: 2000)) - for try await _ in runtime.generate(req) {} - XCTAssertEqual(runtime.receivedRequests.count, 1) - XCTAssertNotNil(runtime.receivedRequests.first?.seed, "the seed reached the backend as the first frame") - // A seed-present run sets the disclosure flag (cloud → bytes leave too). - let d = runtime.disclosure(for: req) - XCTAssertTrue(d.seedPresent) - XCTAssertTrue(d.bytesLeaveDevice) - } - - func testLocalStubSeedSetsDisclosureButNoUpload() async throws { - let runtime = StubLocalVideoRuntime.make(script: .success(videoAsset())) - let req = MediaRequest(prompt: "animate this", seed: png(), kind: .video) - for try await _ in runtime.generate(req) {} - let d = runtime.disclosure(for: req) - XCTAssertTrue(d.seedPresent) - XCTAssertFalse(d.bytesLeaveDevice, "local never uploads even with a seed") - } - - // MARK: - 6.1 Parking — slow job parks, completion glows; over-budget escalates - - func testVideoParkingFeedsSchedulerAndGlowsOnFinish() { - let id = AgentSessionID() - let scheduler = SerialParkScheduler(sessions: [ParkedSession(id: id, title: "v", state: .parked)]) - let feed = MediaParkFeed(scheduler: scheduler) - feed.reportPainting(id, tool: MediaTool.generateVideo) - feed.reportFinished(id, tool: MediaTool.generateVideo, asset: videoAsset()) - let after = scheduler.snapshot().first { $0.id == id } - XCTAssertEqual(after?.state, .idle, - "completion idles with the unseen result — never a terminal state that removes it") - XCTAssertGreaterThanOrEqual(after?.badgeCount ?? 0, 1) - } - - func testParkedOverBudgetEscalatesNeedsYou() { - let id = AgentSessionID() - let scheduler = SerialParkScheduler(sessions: [ParkedSession(id: id, title: "v", state: .parked)]) - let feed = MediaParkFeed(scheduler: scheduler) - feed.reportNeedsYou(id, reason: VideoTierResolver.overBudgetReason(parked: true)) - XCTAssertEqual(scheduler.snapshot().first { $0.id == id }?.state, .needsYou) - } - - // MARK: - 6.2 Finished clip → Files-band asset + canvas player (CONSUMED seam) - - func testFinishedVideoLandsAsFilesBandEntryAndCanvasPlayer() throws { - // The finished clip is durable in the gallery (output #1) as an .mp4 .fileEntry, and the canvas - // state model (output #2) advances to `.finished` with a playable video asset — both CONSUMED from - // the `ai-media-runtime` seam, unchanged by this slice. - let root = FileManager.default.temporaryDirectory - .appendingPathComponent("VideoGalleryTests-\(UUID().uuidString)", isDirectory: true) - let gallery = MediaGallery(root: root) - let asset = try gallery.write(Data([0, 1, 2]), kind: .video, width: 1024, height: 576, durationMs: 4000) - XCTAssertEqual(asset.kind, .video) - XCTAssertEqual(asset.url.pathExtension, "mp4") - let entry = asset.fileEntry() - XCTAssertEqual(entry.kind, .video, "the clip lists as a video Files-band entry") - XCTAssertFalse(entry.isDirectory) - - // The canvas advances to a finished player on the .finished progress. - var state = MediaJobState.idle - state.advance(.step(index: 0, total: 1, preview: nil)) - state.advance(.finished(asset)) - XCTAssertTrue(state.isTerminal) - XCTAssertEqual(state.asset?.kind, .video) - XCTAssertEqual(state.asset?.durationMs, 4000) - } - - // MARK: - 8.1 / 8.2 Error taxonomy + translator - - func testVideoErrorsMapToMediaTaxonomyAndTranslateClean() { - let cases: [MediaError] = [.cloudBudgetExhausted, .videoProviderDisabled, .cloudUnavailable, - .generationFailed(headline: "The upload failed.")] - for c in cases { - let presented = AIError.message(for: c) - XCTAssertFalse(presented.headline.isEmpty) - XCTAssertEqual(presented.headline, c.errorDescription) - } - } - - func testGenerationFailedHeadlineNeverCarriesRawText() { - // A boundary maps an NSURLError into a clean headline; raw text never reaches the headline. - let raw = "NSURLErrorDomain Code=-1009 \"The Internet connection appears to be offline.\"" - let mapped = MediaError.generationFailed(headline: "The video service isn't reachable right now.") - let presented = AIError.message(for: mapped) - XCTAssertFalse(presented.headline.contains("NSURLErrorDomain")) - XCTAssertFalse(presented.headline.contains(raw)) - XCTAssertEqual(presented.headline, "The video service isn't reachable right now.") - } - - func testFailedMidFlightStreamThrowsNotFinished() async { - let runtime = StubCloudVideoRuntime.make(script: .failMidFlight(steps: 2, headline: "The render failed.")) - var thrown: Error? - var finished = false - do { - for try await p in runtime.generate(MediaRequest(prompt: "x", kind: .video)) { - if case .finished = p { finished = true } - } - } catch { thrown = error } - XCTAssertFalse(finished, "a failed render never yields a finished asset (failed, not a false Done)") - XCTAssertNotNil(thrown) - if case MediaError.generationFailed = (thrown as? MediaError) ?? .cloudUnavailable {} else { - XCTFail("a failure maps into the MediaError taxonomy") - } - } - - // MARK: - 9.1 Swap-in contract - - func testSecondBackendJoinsSameSeamUnchanged() async throws { - // Both backends are interchangeable `MediaRuntime` conformers selected by `videoProvider`. Feature - // code drives them identically — the ONLY observable difference is the disclosure flag. - func drive(_ runtime: MediaRuntime) async throws -> MediaAsset? { - var out: MediaAsset? - for try await p in runtime.generate(MediaRequest(prompt: "same", kind: .video)) { - if case let .finished(a) = p { out = a } - } - return out - } - let cloud: MediaRuntime = StubCloudVideoRuntime.make(script: .success(videoAsset())) - let local: MediaRuntime = StubLocalVideoRuntime.make(script: .success(videoAsset())) - // Same seam, same capability, same finished shape — the sink/canvas/Files path is identical. - XCTAssertEqual(cloud.capabilities, local.capabilities) - let a = try await drive(cloud) - let b = try await drive(local) - XCTAssertEqual(a?.kind, .video) - XCTAssertEqual(b?.kind, .video) - } - - // MARK: - AppSettings persistence (1.3) - - @MainActor - func testAppSettingsVideoKeysDefaultAndPersist() { - let suite = "VideoAnimationTests-\(UUID().uuidString)" - let defaults = UserDefaults(suiteName: suite)! - defer { defaults.removePersistentDomain(forName: suite) } - let settings = AppSettings(defaults: defaults) - XCTAssertEqual(settings.videoProvider, .cloud, "default provider is cloud") - XCTAssertEqual(settings.mediaVideoBudgetPerDay, 3, "conservative default cap") - settings.videoProvider = .localLTXV - settings.mediaVideoBudgetPerDay = 7 - // A fresh instance over the same store reads back the persisted values. - let reloaded = AppSettings(defaults: defaults) - XCTAssertEqual(reloaded.videoProvider, .localLTXV) - XCTAssertEqual(reloaded.mediaVideoBudgetPerDay, 7) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/VoiceSessionControllerTests.swift b/Tests/ThreeFingerSwitcherTests/VoiceSessionControllerTests.swift deleted file mode 100644 index fe1705a..0000000 --- a/Tests/ThreeFingerSwitcherTests/VoiceSessionControllerTests.swift +++ /dev/null @@ -1,165 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// `add-voice-computer-use-agent`: the controller executing the pure model's effects through the -/// stub seams — the spec scenario "Voice logic verifies with the stub": capture → transcript → turn -/// → spoken reply → barge-in, deterministically, no Speech framework involved. -@MainActor -final class VoiceSessionControllerTests: XCTestCase { - - private final class TurnScriptBox { - var tokens: [Token] = [] - var started = 0 - var cancelled = false - } - - private func makeController(transcriber: StubTranscriber, - synthesizer: StubSynthesizer, - micAuthorized: Bool = true, - turns: TurnScriptBox) -> VoiceSessionController { - VoiceSessionController( - transcriberFactory: { transcriber }, - synthesizer: synthesizer, - micAuthorizer: { micAuthorized }, - turnStarter: { _ in - turns.started += 1 - let scripted = turns.tokens - return AsyncThrowingStream { continuation in - let task = Task { - for token in scripted { - try Task.checkCancellation() - continuation.yield(token) - await Task.yield() - } - continuation.finish() - } - continuation.onTermination = { _ in - task.cancel() - turns.cancelled = true - } - } - }) - } - - /// Drain main-actor tasks so async effect chains settle deterministically. - private func settle() async { - for _ in 0..<20 { await Task.yield() } - } - - func testEndToEndSpokenTurn() async { - let transcriber = StubTranscriber(partials: ["read"], final: "read the window") - let synthesizer = StubSynthesizer() - let turns = TurnScriptBox() - turns.tokens = [Token("Here is the text. ", isFinal: false), - Token("It says hello.", isFinal: true)] - let controller = makeController(transcriber: transcriber, synthesizer: synthesizer, turns: turns) - - controller.pttDown() - await settle() - XCTAssertTrue(transcriber.isCapturing, "mic opens on press") - XCTAssertEqual(controller.phase, .listening) - - controller.pttUp() - await settle() - XCTAssertFalse(transcriber.isCapturing, "mic closes on release") - XCTAssertEqual(turns.started, 1, "the finalized transcript starts the turn") - - await settle() - XCTAssertEqual(synthesizer.spoken.first, "Here is the text.", - "the first sentence speaks before the stream ends") - XCTAssertTrue(synthesizer.spoken.contains("It says hello.")) - - synthesizer.finishAll() - await settle() - XCTAssertEqual(controller.phase, .idle, "settled + drained → idle") - } - - func testThinkingChannelIsNeverSpoken() async { - let transcriber = StubTranscriber(final: "think about it") - let synthesizer = StubSynthesizer() - let turns = TurnScriptBox() - turns.tokens = [Token("secret reasoning", isFinal: false, channel: .thinking), - Token("The answer.", isFinal: true)] - let controller = makeController(transcriber: transcriber, synthesizer: synthesizer, turns: turns) - - controller.pttDown(); await settle() - controller.pttUp(); await settle(); await settle() - XCTAssertEqual(synthesizer.spoken, ["The answer."], - "thinking never reaches the synthesizer") - } - - func testMicDenialSurfacesCleanCard() async { - let transcriber = StubTranscriber(final: "irrelevant") - let synthesizer = StubSynthesizer() - let controller = makeController(transcriber: transcriber, synthesizer: synthesizer, - micAuthorized: false, turns: TurnScriptBox()) - controller.pttDown() - await settle() - XCTAssertEqual(controller.phase, .idle) - XCTAssertEqual(controller.lastFailure?.headline, - VoiceError.micDenied.errorDescription, - "the clean headline, never raw vendor text") - } - - func testOSTooOldWhenFactoryReturnsNil() async { - let synthesizer = StubSynthesizer() - let controller = VoiceSessionController( - transcriberFactory: { nil }, - synthesizer: synthesizer, - micAuthorizer: { true }, - turnStarter: { _ in AsyncThrowingStream { $0.finish() } }) - controller.pttDown() - await settle() - XCTAssertEqual(controller.lastFailure?.headline, VoiceError.osTooOld.errorDescription) - XCTAssertEqual(controller.phase, .idle) - } - - func testBargeInStopsSynthesizerAndCancelsTurn() async { - let transcriber = StubTranscriber(final: "long story") - let synthesizer = StubSynthesizer() - let turns = TurnScriptBox() - // An endless-ish stream (many sentences) so the barge-in lands mid-turn. - turns.tokens = (0..<50).map { Token("Sentence number \($0). ", isFinal: false) } - let controller = makeController(transcriber: transcriber, synthesizer: synthesizer, turns: turns) - - controller.pttDown(); await settle() - controller.pttUp(); await settle() - XCTAssertGreaterThan(synthesizer.spoken.count, 0, "speaking began") - - let spokenBeforeBarge = synthesizer.spoken.count - controller.pttDown() // barge-in - await settle() - XCTAssertGreaterThanOrEqual(synthesizer.stopCount, 1, "TTS stopped immediately") - XCTAssertEqual(controller.phase, .listening, "barge-in listens") - await settle() - XCTAssertEqual(synthesizer.spoken.count, spokenBeforeBarge, - "no late tokens are spoken after the barge-in") - } - - func testHumanTouchAborts() async { - let transcriber = StubTranscriber(final: "act on it") - let synthesizer = StubSynthesizer() - let turns = TurnScriptBox() - turns.tokens = (0..<50).map { Token("Working on step \($0). ", isFinal: false) } - let controller = makeController(transcriber: transcriber, synthesizer: synthesizer, turns: turns) - - controller.pttDown(); await settle() - controller.pttUp(); await settle() - controller.humanTouch() - await settle() - XCTAssertEqual(controller.phase, .idle, "touch aborts to idle — the mic does not open") - XCTAssertGreaterThanOrEqual(synthesizer.stopCount, 1) - } - - func testIsConversationActiveFeedsQuiescence() async { - let transcriber = StubTranscriber(final: "hello") - let synthesizer = StubSynthesizer() - let turns = TurnScriptBox() - turns.tokens = [Token("Hi.", isFinal: true)] - let controller = makeController(transcriber: transcriber, synthesizer: synthesizer, turns: turns) - XCTAssertFalse(controller.isConversationActive) - controller.pttDown(); await settle() - XCTAssertTrue(controller.isConversationActive, - "a live voice phase counts as a foreground conversational surface") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/VoiceTurnModelTests.swift b/Tests/ThreeFingerSwitcherTests/VoiceTurnModelTests.swift deleted file mode 100644 index cc067ea..0000000 --- a/Tests/ThreeFingerSwitcherTests/VoiceTurnModelTests.swift +++ /dev/null @@ -1,195 +0,0 @@ -import XCTest -@testable import ThreeFingerSwitcherCore - -/// `add-voice-computer-use-agent`: the pure voice-turn lifecycle (spec "Barge-in stops speech and -/// cancels the turn as a discard" — "The voice turn lifecycle is pure and tested"), the sentence -/// chunker, and the audio-seam refusal contract. -@MainActor -final class VoiceTurnModelTests: XCTestCase { - - private let t = Date(timeIntervalSinceReferenceDate: 500_000) - private var model = VoiceTurnModel() - - private func feed(_ e: VoiceTurnModel.Event) -> [VoiceTurnModel.Effect] { - model.handle(e, at: t) - } - - // MARK: - The happy path - - func testFullHappyLifecycle() { - XCTAssertEqual(feed(.pttDown), [.startCapture]) - XCTAssertEqual(model.phase, .listening) - XCTAssertEqual(feed(.pttUp), [.stopCapture]) - XCTAssertEqual(model.phase, .transcribing) - XCTAssertEqual(feed(.transcriptFinal("read that window")), - [.sendTurn("read that window", epoch: 1)]) - XCTAssertEqual(model.phase, .thinking) - XCTAssertEqual(feed(.chunkReady("Here it is.", epoch: 1)), [.speak("Here it is.")]) - XCTAssertEqual(model.phase, .speaking) - XCTAssertEqual(feed(.chunkReady("Second sentence.", epoch: 1)), [.speak("Second sentence.")]) - XCTAssertEqual(feed(.turnSettled(epoch: 1)), []) - XCTAssertEqual(model.phase, .speaking, "still draining") - XCTAssertEqual(feed(.speechDrained), []) - XCTAssertEqual(model.phase, .idle, "settled + drained → idle") - } - - func testEmptyTranscriptIsANoOp() { - _ = feed(.pttDown); _ = feed(.pttUp) - XCTAssertEqual(feed(.transcriptFinal(" ")), []) - XCTAssertEqual(model.phase, .idle) - } - - func testSettleWithNoSpokenOutputGoesStraightIdle() { - _ = feed(.pttDown); _ = feed(.pttUp); _ = feed(.transcriptFinal("do it")) - XCTAssertEqual(feed(.turnSettled(epoch: 1)), []) - XCTAssertEqual(model.phase, .idle) - } - - // MARK: - Barge-in (spec scenario: barge-in mid-reply) - - func testBargeInWhileSpeakingStopsCancelsAndListens() { - _ = feed(.pttDown); _ = feed(.pttUp); _ = feed(.transcriptFinal("hi")) - _ = feed(.chunkReady("Hello there.", epoch: 1)) - XCTAssertEqual(model.phase, .speaking) - XCTAssertEqual(feed(.pttDown), [.stopSpeaking, .cancelTurn, .startCapture]) - XCTAssertEqual(model.phase, .listening) - // Late chunks from the barged-in turn are DROPPED, never spoken. - XCTAssertEqual(feed(.chunkReady("late token", epoch: 1)), []) - XCTAssertEqual(feed(.turnSettled(epoch: 1)), []) - XCTAssertEqual(model.phase, .listening) - // The corrected turn gets a NEW epoch. - _ = feed(.pttUp) - XCTAssertEqual(feed(.transcriptFinal("actually, summarize it")), - [.sendTurn("actually, summarize it", epoch: 2)]) - } - - func testBargeInWhileThinking() { - _ = feed(.pttDown); _ = feed(.pttUp); _ = feed(.transcriptFinal("hi")) - XCTAssertEqual(model.phase, .thinking) - XCTAssertEqual(feed(.pttDown), [.stopSpeaking, .cancelTurn, .startCapture]) - XCTAssertEqual(model.phase, .listening) - } - - // MARK: - Human touch = abort, never talk - - func testHumanTouchAbortsSpeakingToIdle() { - _ = feed(.pttDown); _ = feed(.pttUp); _ = feed(.transcriptFinal("hi")) - _ = feed(.chunkReady("Hello.", epoch: 1)) - XCTAssertEqual(feed(.humanTouch), [.stopSpeaking, .cancelTurn]) - XCTAssertEqual(model.phase, .idle, "touch aborts; it does not open the mic") - } - - func testHumanTouchWhileListeningCancelsDictation() { - _ = feed(.pttDown) - XCTAssertEqual(feed(.humanTouch), [.cancelCapture]) - XCTAssertEqual(model.phase, .idle) - } - - // MARK: - Failures are clean - - func testCaptureFailurePresentsAndReturnsIdle() { - _ = feed(.pttDown) - XCTAssertEqual(feed(.voiceFailed(.micDenied)), - [.cancelCapture, .presentFailure(.micDenied)]) - XCTAssertEqual(model.phase, .idle) - } - - func testTurnFailureReturnsIdleWithoutSpeaking() { - _ = feed(.pttDown); _ = feed(.pttUp); _ = feed(.transcriptFinal("hi")) - XCTAssertEqual(feed(.turnFailed(epoch: 1)), []) - XCTAssertEqual(model.phase, .idle) - } -} - -// MARK: - SentenceChunker - -final class SentenceChunkerTests: XCTestCase { - - func testFirstSentenceClosesEarly() { - var chunker = SentenceChunker() - XCTAssertEqual(chunker.consume("Here is the answer. The rest"), ["Here is the answer."]) - XCTAssertEqual(chunker.consume(" continues!"), []) - XCTAssertEqual(chunker.consume(" And more."), ["The rest continues!"]) - XCTAssertEqual(chunker.flush(), "And more.") - } - - func testParagraphBreakCloses() { - var chunker = SentenceChunker() - XCTAssertEqual(chunker.consume("First paragraph\n\nsecond"), ["First paragraph"]) - XCTAssertEqual(chunker.flush(), "second") - } - - func testCodeFenceIsSummarizedNeverRead() { - var chunker = SentenceChunker() - let chunks = chunker.consume("Look:\n```swift\nlet a = 1\nlet b = 2\n```\nDone now.") - XCTAssertEqual(chunks, ["Look:", "Code block, 2 lines."]) - XCTAssertEqual(chunker.flush(), "Done now.") - } - - func testUnterminatedFenceStillSummarizesOnFlush() { - var chunker = SentenceChunker() - _ = chunker.consume("```\ncode line\n") - XCTAssertEqual(chunker.flush(), "Code block, 1 line.") - } - - func testLongUnpunctuatedRunFlushesAtWhitespace() { - var chunker = SentenceChunker(maxChunkLength: 20) - let chunks = chunker.consume("one two three four five six seven") - XCTAssertFalse(chunks.isEmpty, "length guard must flush") - for chunk in chunks { - XCTAssertFalse(chunk.hasSuffix(" "), "no dangling whitespace") - XCTAssertLessThanOrEqual(chunk.count, 20) - } - } - - func testFlushEmptyIsNil() { - var chunker = SentenceChunker() - XCTAssertNil(chunker.flush()) - } -} - -// MARK: - Audio seam refusal contract (delta: on-device-ai-runtime) - -@MainActor -final class AudioSeamTests: XCTestCase { - - func testNonEmptyAudioIsRefusedNeverDropped() async { - let stub = StubLLMRuntime(capabilities: [.text, .vision, .audio]) - let request = LLMRequest(prompt: "hear this", audio: [Data([1, 2, 3])]) - do { - _ = try await stub.generateText(request) - XCTFail("audio must be refused until a conformer serves it") - } catch let error as RuntimeError { - XCTAssertEqual(error, .unsupportedModality(.audio)) - } catch { - XCTFail("unexpected error: \(error)") - } - } - - func testChatCarriesAudioIntoTheRefusal() async { - let stub = StubLLMRuntime(capabilities: [.text]) - let request = LLMChatRequest(messages: [AgentMessage(role: .user, text: "hi")], - audio: [Data([9])]) - do { - for try await _ in stub.chat(request) {} - XCTFail("chat must thread audio into the refusal contract") - } catch let error as RuntimeError { - XCTAssertEqual(error, .unsupportedModality(.audio)) - } catch { - XCTFail("unexpected error: \(error)") - } - } - - func testEmptyAudioChangesNothing() async throws { - let stub = StubLLMRuntime(capabilities: [.text]) - stub.scriptedTokens = ["ok"] - let out = try await stub.generateText(LLMRequest(prompt: "hi")) - XCTAssertEqual(out, "ok") - } - - func testAudioSelectsForAudioCapability() throws { - let catalog = ModelCatalog.standard - let descriptor = try catalog.selectModel(requiring: [.audio]) - XCTAssertTrue(descriptor.capabilities.contains(.audio)) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/WizardTourBandsTests.swift b/Tests/ThreeFingerSwitcherTests/WizardTourBandsTests.swift index de337cf..221d03b 100644 --- a/Tests/ThreeFingerSwitcherTests/WizardTourBandsTests.swift +++ b/Tests/ThreeFingerSwitcherTests/WizardTourBandsTests.swift @@ -3,7 +3,7 @@ import XCTest /// Unit tests for the playground tour's fixed band composition (Onboarding/WizardTourBands.swift): /// flame (every app across the user's bands, deduped), display (the twelve window actions), -/// sparkles only when AI is on, clipboard only when provided — and nothing more. +/// clipboard only when provided — and nothing more. final class WizardTourBandsTests: XCTestCase { private func app(_ name: String, _ path: String) -> LaunchItem { LaunchItem(title: name, icon: .appDefault, @@ -24,24 +24,15 @@ final class WizardTourBandsTests: XCTestCase { ] } - private func seededAI() -> ContextBand { - ContextBand(name: "AI", color: ItemColor(red: 1, green: 0, blue: 1), - icon: .sfSymbol("sparkles"), - items: [LaunchItem(title: "Seeded", icon: .sfSymbol("sparkles"), - kind: .script(.shell("echo")))]) - } - func testBaseCompositionIsFlameThenDisplayAndNothingMore() { - let bands = WizardTourBands.compose(userBands: userBands(), aiOn: false, - seededAIBand: seededAI, clipboardBand: nil) + let bands = WizardTourBands.compose(userBands: userBands(), clipboardBand: nil) XCTAssertEqual(bands.count, 2, "flame + display — nothing more") XCTAssertEqual(bands[0].icon, .sfSymbol("flame.fill")) XCTAssertEqual(bands[1].icon, .sfSymbol("display")) } func testFlameGathersEveryAppOnceAndOnlyApps() { - let bands = WizardTourBands.compose(userBands: userBands(), aiOn: false, - seededAIBand: seededAI, clipboardBand: nil) + let bands = WizardTourBands.compose(userBands: userBands(), clipboardBand: nil) let flame = bands[0] XCTAssertEqual(flame.items.map(\.title), ["Terminal", "Mail"], "deduped by bundle URL, original order, URLs/scripts excluded") @@ -51,8 +42,7 @@ final class WizardTourBandsTests: XCTestCase { } func testDisplayHoldsExactlyTheTwelveWindowActions() { - let bands = WizardTourBands.compose(userBands: [], aiOn: false, - seededAIBand: seededAI, clipboardBand: nil) + let bands = WizardTourBands.compose(userBands: [], clipboardBand: nil) let display = bands[1] XCTAssertEqual(display.items.count, 12, "two exact rows of the six-column grid") for item in display.items { @@ -64,32 +54,10 @@ final class WizardTourBandsTests: XCTestCase { XCTAssertEqual(Set(WizardTourBands.windowActions).count, 12, "no duplicates") } - func testAIBandUsesTheUsersOwnCommandsWhenTheyHaveAny() { - var bands = userBands() - let owned = LaunchItem(title: "My command", icon: .sfSymbol("sparkles"), - kind: .aiCommand(AICommand(name: "My command", - icon: .sfSymbol("sparkles"), - input: .selection, - promptTemplate: "{selection}", - output: .replaceSelection))) - bands[0].items.append(owned) - let composed = WizardTourBands.compose(userBands: bands, aiOn: true, - seededAIBand: seededAI, clipboardBand: nil) - XCTAssertEqual(composed.count, 3) - XCTAssertEqual(composed[2].items.map(\.title), ["My command"]) - } - - func testAIBandFallsBackToTheSeededSet() { - let composed = WizardTourBands.compose(userBands: userBands(), aiOn: true, - seededAIBand: seededAI, clipboardBand: nil) - XCTAssertEqual(composed[2].items.map(\.title), ["Seeded"]) - } - func testClipboardBandComesLastWhenProvided() { let clipboard = ClipboardBandBuilder.build(from: WizardSampleContent.clipboardEntries()) - let composed = WizardTourBands.compose(userBands: userBands(), aiOn: true, - seededAIBand: seededAI, clipboardBand: clipboard) - XCTAssertEqual(composed.count, 4) - XCTAssertTrue(ClipboardBandBuilder.isClipboardBand(composed[3])) + let composed = WizardTourBands.compose(userBands: userBands(), clipboardBand: clipboard) + XCTAssertEqual(composed.count, 3) + XCTAssertTrue(ClipboardBandBuilder.isClipboardBand(composed[2])) } } diff --git a/docs/ai-agent-v2-addendum-compute-media-fleet.md b/docs/ai-agent-v2-addendum-compute-media-fleet.md deleted file mode 100644 index 3f4b2c8..0000000 --- a/docs/ai-agent-v2-addendum-compute-media-fleet.md +++ /dev/null @@ -1,296 +0,0 @@ -# AI Agent V2 — Addendum: Compute Tiers · Generative Media · Model Fleet - -**Status:** binding on the SIX new slice architects (`ai-compute-tiers`, `ai-media-runtime`, -`ai-local-image-generation`, `ai-video-animation-generation`, `ai-model-fleet`, -`ai-full-potential-toggle`). This EXTENDS `docs/ai-agent-v2-blueprint.md` — read that FIRST. Every -convention there (naming, target-split, error taxonomy, gesture compass, reuse-don't-reinvent, OpenSpec -authoring rules §7) still binds. This addendum only pins the NEW shared types these six slices add, and -the decisions that resolve how they weld onto the existing nine. - -This is the second evolution wave: from a single-GPU, single-model, text-only agent into a -**two-lane (GPU+CPU), multi-model, media-generating** companion — still Apple-Silicon M5/M4 only, still -behind one master opt-in. - ---- - -## 0. What this wave EVOLVES (never forks) - -| Existing seam / slice | V2.5 role | -|---|---| -| `LLMRuntime` (`AI/LLMRuntime.swift`) | UNCHANGED as the text seam. The CPU ternary model is **another `LLMRuntime` conformer**, selected by lane, not a new protocol. | -| `ai-batched-runtime-and-context` (§3.6 `BatchedLLMRuntime`) | Stays the **GPU lane**. `ai-compute-tiers` adds the CPU lane beside it + cross-lane budget; it AMENDS the GPU-only assumption, it does not rewrite the batched slice. | -| `ParkScheduler` (§3.5) | Gains an optional **lane-affinity hint** so a parked subagent can be dispatched to the CPU lane CONCURRENTLY with a foreground GPU generation. Additive; `runnableSessions(now:maxSlots:)` signature unchanged. | -| `ToolRegistry` / `ToolDescriptor` / route loop (§3.3) | Media generation is a **tool**: `generate_image` / `generate_video` are `ToolDescriptor`s with a `WritePolicyTier`; `MediaGenSink` executes via the existing route→execute→continue loop. No new control flow. | -| `WritePolicyTier` / `AuditLog` (§3.7) | Media writes + cloud-video escalation ride the SAME tiers + audit. Image gen = `.confirm`; cloud video = `.dangerous` + budget cap (mirrors Claude handoff §3.8). | -| `ModelManager` / `ModelProvisioner` (`AI/ModelManager.swift`) | `ai-model-fleet` evolves it from one-resident-runtime to a **registry of descriptors with residency/eviction**. The provisioner seam is reused; a fleet-of-one is today's behavior. | -| `files-band` (`Files/`, `Overlay/FilesBandView.swift`) | Generated assets land as **Files-band entries** (the gallery). Reuse, do not build a new browser. | -| `ai-claude-handoff` (§3.8) gating | The **template** for cloud-media escalation: confirm-by-default, per-day budget cap, audited, fire-and-forget. | -| `DockPreviewOverlay` pattern | The media canvas preview/player + any reveal surface reuse it (non-activating, synchronous `orderOut`). | - ---- - -## 1. Naming (binding, extends blueprint §1) - -- **New change dirs / slice names (verbatim):** `ai-compute-tiers`, `ai-media-runtime`, - `ai-local-image-generation`, `ai-video-animation-generation`, `ai-model-fleet`, - `ai-full-potential-toggle`. -- **New capability spec:** `ai-generative-media` (owned by `ai-media-runtime`; the two backend slices - write deltas against it). Compute/fleet/toggle slices write deltas against existing capabilities - (`on-device-ai-runtime`, `configuration-hub`, `tunable-settings`). -- **New Swift types:** PascalCase, no `V2` suffix. New homes under - `Sources/ThreeFingerSwitcher/AI/Compute/`, `AI/Media/`, `AI/Fleet/` (Core, MLX-free for seams + value - types + policy logic). Native-linked backends live in the `GemmaRuntime` target or a sibling - framework. -- **New error enums (one per domain, only if `RuntimeError`/`TaskError` cannot carry it):** - `MediaError`, `FleetError`. A `ComputeError` only if lane dispatch genuinely needs its own case; - prefer extending `RuntimeError`. All route through `AIError.message(for:)` → `AIPresentedError`. - Map vendor/OS errors (mflux, LTXV/ComfyUI, bitnet.cpp, `Process`, `NSURLError`) at the layer boundary. -- **New persisted keys (camelCase, agent-scoped):** `fullPotentialEnabled`, `cpuLaneEnabled`, - `mediaGenEnabled`, `fleetCloudEscalationEnabled`, `mediaVideoBudgetPerDay`, `imageModelID`, - `videoProvider`. - ---- - -## 2. Target-split & verification (binding, extends blueprint §2) - -- **MLX-free Core (`swift build` + `swift test`):** `ComputeLane` + the role→lane policy; the - `MediaRuntime` PROTOCOL + all media value types; `MediaGenSink` routing; `ModelRegistry` + - `ModelDescriptor` value types + residency/eviction MATH; `fullPotentialEnabled` + gating logic; - `MediaError`/`FleetError`. **This is the majority of every new slice.** A `StubMediaRuntime` + - `StubTernaryRuntime` make it all test-verified without real weights. -- **Native-linked (`xcodebuild` COMPILE-VERIFY ONLY for an agent; real correctness needs the user's - stable-signed build):** the bitnet.cpp-class **ternary CPU runtime**; the **mflux/FLUX image - runtime**; the **LTXV video runtime**; real residency/eviction with real weights; the canvas media - **player overlay**. Each new `design.md` MUST state per-component which target it lives in and how it - is verified. Agents NEVER build/sign/install the `.app` (ad-hoc signing breaks TCC; the - `*.bundle` metallib copy in `build-app.sh` must not regress). - ---- - -## 3. SHARED CONTRACTS (canonical sketches — binding) - -Sketches, not final code. The OWNER slice writes the real type; CONSUMERS import it as written. - -### §A1. Compute lane — OWNER: `ai-compute-tiers` - -```swift -// Core. Which physical lane a runtime/work-unit uses. The GPU does heavy generation + diffusion; -// the CPU ternary lane does short/frequent/structured work (routing, classify, memory-index, -// parked subagents) — concurrently, because ternary weights are bandwidth-frugal. -public enum ComputeLane: String, Codable, Sendable { case gpu, cpuTernary } - -// The pure role→lane policy. Heavy generation → .gpu; router turn / classification / -// memory retrieval / parked subagent advance → .cpuTernary. Pure + testable. -public protocol LaneRouting: Sendable { - func lane(for role: AgentWorkRole) -> ComputeLane -} -public enum AgentWorkRole: String, Codable, Sendable { - case foregroundGeneration // the main reply → GPU - case mediaDiffusion // image/video gen → GPU (evicts chat; see §C1) - case toolRoute // structured() route turn → CPU ternary - case classify // cheap decisions (should-park / needs-you / which-skill) → CPU ternary - case memoryRetrieval // index/TOC retrieval → CPU ternary - case parkedSubagent // background advance of a parked session → CPU ternary -} -``` - -- The **CPU ternary model is an `LLMRuntime` conformer** (`TernaryCPURuntime`), NOT a new protocol. It - carries a SMALL ternary/BitNet-class model. Selected by lane via the registry (§C1). -- `ai-compute-tiers` MUST justify, with the measured M5 facts (prefill up to ~4× on the GPU neural - accelerators; token-gen bandwidth-bound at ~153 GB/s; ternary = ~32× smaller, bandwidth-frugal), why - the CPU lane breaks the single-GPU serialization for light work — and be HONEST that CPU per-token is - slower, so the lane is for short structured bursts, never the long reply. -- It AMENDS `ai-batched-runtime-and-context` (the GPU lane) and adds the lane-affinity hint consumed by - `ParkScheduler` — as MODIFIED requirements on `on-device-ai-runtime`, not by editing the batched - slice's files. - -### §B1. Generative-media seam — OWNER: `ai-media-runtime` - -```swift -// Core. A SECOND runtime seam, parallel to LLMRuntime — NOT a token stream. A long async job -// with step-progress that ends in bytes/a file. -public enum MediaKind: String, Codable, Sendable { case image, video } - -public struct MediaParameters: Codable, Equatable, Sendable { - public var size: MediaSize // width × height - public var steps: Int // diffusion steps - public var seedNumber: UInt64? // RNG seed for reproducibility - public var guidance: Double? - public var durationMs: Int? // video only -} - -public struct MediaRequest: Sendable { - public var prompt: String - public var seed: Data? // optional SEED IMAGE (PNG) for img2img / img2video — the - // screen-region / clipboard capture becomes the first frame - public var kind: MediaKind - public var parameters: MediaParameters -} - -public enum MediaProgress: Sendable { - case step(index: Int, total: Int, preview: Data?) // streamed diffusion progress + optional preview - case finished(MediaAsset) -} - -public struct MediaAsset: Codable, Equatable, Identifiable, Sendable { - public let id: UUID - public var url: URL // the written file (becomes a Files-band entry) - public var kind: MediaKind - public var width: Int - public var height: Int - public var durationMs: Int? -} - -public protocol MediaRuntime: Sendable { - var capabilities: Set { get } - func generate(_ request: MediaRequest) -> AsyncThrowingStream -} -``` - -- `generate_image` and `generate_video` are **`ToolDescriptor`s** (§3.3) registered in the - `ToolRegistry`; `MediaGenSink` is the side-effecting executor invoked by the route loop. Write-policy - tiers: **image `.confirm`**, **cloud video `.dangerous` + budget-capped** (mirrors handoff §3.8). -- Result lands as a **Files-band asset** (the gallery) AND a **canvas preview/player**; swipe-DOWN - extracts (save / paste / set-as), per the canonical compass. Generation is **slow → parks** via - `ParkScheduler`; the notch glows on completion / on `needsYou`. -- `MediaError` taxonomy, mapped at the boundary, surfaced bounded + non-blocking (never `NSAlert`, never - raw error in a headline). `ai-media-runtime` OWNS the seam + tools + sink + Files/canvas output; it - does NOT own the concrete backends. - -### §B2. Image backend (in scope NOW) — OWNER: `ai-local-image-generation` -- A concrete `MediaRuntime` for **local image** via mflux/FLUX-class MLX (native-linked, - compile-verify-only; descriptor + residency math + a stub are `swift test`-verified). Quant variants - (Q4 ~7 GB → FP16 ~24 GB). M5 diffusion is compute-bound → neural-accelerator sweet spot (~3.8× vs M4). - img2img/inpaint from the seed. Residency vs Gemma is DECIDED by the fleet (§C1) and consumed here. - -### §B3. Video/animation backend (frontier + escalation) — OWNER: `ai-video-animation-generation` -- **Honest default = CLOUD escalation** (LTX Studio API / hosted), gated EXACTLY like Claude handoff - (§3.8): confirm-by-default, per-day budget/rate cap, audited, fire-and-forget with progress. **Local - LTXV** is a FRONTIER option behind the full-potential toggle (ComfyUI/MPS, 35 GB+, minutes/clip — NOT - in-process MLX; document the cost). img2video from the seed. Build the seam so a local LTXV backend - drops into the SAME `MediaRuntime` later, exactly as `LLMRuntime` lets Gemma be swapped. - -### §C1. Model fleet — OWNER: `ai-model-fleet` - -```swift -// Core. Evolves ModelManager from one-resident-runtime to a registry with residency/eviction. -public enum ModelRole: String, Codable, Sendable { - case chat // Gemma on the GPU lane - case ternaryChat // the small CPU-lane model - case image, video // generative-media backends - case cloudEscalation // Claude, GLM-5.2 — NOT resident locally -} - -public struct ModelDescriptor: Codable, Equatable, Identifiable, Sendable { - public let id: String - public var name: String - public var role: ModelRole - public var lane: ComputeLane? // nil for cloud - public var provider: ModelProvider // .onDevice / .cloud - public var residencyBytes: UInt64 // for the eviction budget (0 for cloud) - public var maxContextTokens: Int? // chat/ternary only - public var capabilities: Set // text/vision/image/video tags -} -public enum ModelProvider: String, Codable, Sendable { case onDevice, cloud } - -public protocol ModelRegistry: Sendable { - func descriptors() -> [ModelDescriptor] - func resident() -> [ModelDescriptor] - func ensureResident(_ id: String) throws // may EVICT under the 48GB budget -} -``` - -- Residency math is pure/testable: chat (GPU) + ternary (CPU) + image (Q4) + KV can co-reside; a - **video gen or FP16 image EVICTS chat** (the companion goes quiet while it paints — document it). -- **Claude and GLM-5.2 are `provider: .cloud`, `role: .cloudEscalation`** — never resident; they ride - the handoff/escalation paths. GLM-5.2 (753B MoE / 1M ctx / MIT) does NOT fit 48 GB — local is not an - option; it is a cloud fleet member only. -- Reuse `ModelProvisioner`/`runtimeFactory`. A fleet-of-one MUST remain valid (today's behavior). - -### §D1. The master gate — OWNER: `ai-full-potential-toggle` - -```swift -// Core (AppSettings + gating logic). Default OFF → V2 ships calm; ON lights up the fleet. -// master: -var fullPotentialEnabled: Bool // default false -// sub-capability flags, each gated under the master: -var cpuLaneEnabled: Bool // ai-compute-tiers -var batchedRuntimeEnabled: Bool // ai-batched-runtime-and-context (existing heavy slice) -var mediaGenEnabled: Bool // ai-media-runtime + backends -var backgroundAutonomyEnabled: Bool // ai-background-autonomy (existing heavy slice) -var fleetCloudEscalationEnabled: Bool // ai-model-fleet cloud members (Claude / GLM-5.2) -``` - -- The gating logic is Core + `swift test`-verified. Each heavy slice CHECKS its flag before activating. -- **Disclosure UX (the project's honest-surface ethos, applied to capability cost):** each sub-toggle - states its RAM / heat / latency / $-cost in the same breath it offers the capability — no hidden - fans-screaming surprise. Progressive enablement; the master gate is one Hub page. - ---- - -## 4. Dependency weld onto the existing nine (binding) - -``` -existing Wave 1–2 (conversation-runtime, tool-routing, batched-runtime) land FIRST - │ -ai-compute-tiers ──amends──> batched-runtime (GPU lane) + parked-sessions (lane affinity) - │ adds the CPU ternary LLMRuntime conformer + role→lane policy - │ -ai-model-fleet ──evolves──> ModelManager (registry/residency); consumes ComputeLane; - │ registers cloud members (Claude, GLM-5.2) - │ -ai-media-runtime ──new seam──> registers generate_image/video tools in ToolRegistry; - │ output → Files band; parks via ParkScheduler - ├──> ai-local-image-generation (concrete image MediaRuntime; in scope NOW) - └──> ai-video-animation-generation (cloud escalation default; local LTXV frontier) - │ -ai-full-potential-toggle ──gates──> compute-tiers, batched-runtime, background-autonomy, - media (runtime+backends), fleet cloud escalation -``` - -**Implementation order (after the existing nine's waves):** -1. `ai-compute-tiers` (CPU lane beside the GPU batched runtime). -2. `ai-model-fleet` (registry/residency — needs the lane concept). -3. `ai-media-runtime` (the seam + tools + sink + output). -4. `ai-local-image-generation`, `ai-video-animation-generation` (backends, parallel; depend on the seam + fleet). -5. `ai-full-potential-toggle` (last — gates everything new + the existing heavy slices). - -Architects plan in PARALLEL now; the order above is the *implementation* sequence so a later slice may -assume an earlier one's types exist. - ---- - -## 5. Cross-cutting decisions (adopted — every new architect honors) - -1. **Hardware floor is M5 (M4 min). No regressions, no degraded paths.** These features exist BECAUSE - the hardware serves them. Cite the measured M5 facts where relevant. -2. **GLM-5.2 is cloud-only** (753B MoE / 1M ctx / MIT — datacenter scale). The local CPU model is a - **small ternary/BitNet-class** model. Do not propose GLM-5.2 as a resident local runtime. -3. **Two lanes, one process:** GPU = heavy craftsman (chat reply, diffusion); CPU ternary = fast clerks - (route, classify, remember, parked subagents). Ternary is bandwidth-frugal → low contention on the - shared 153 GB/s bus. CPU per-token is slower → short structured work only. -4. **Media is a tool, parked because it's slow, gated by write-policy, audited.** Image `.confirm`; - cloud video `.dangerous` + budget cap (the Claude-handoff pattern). Output is a Files-band asset. -5. **A heavy gen EVICTS chat** under the 48 GB budget — state it honestly in the fleet + media specs; - surface "the assistant is busy painting" rather than pretend co-residency. -6. **Default OFF.** V2 ships calm; the master toggle is the user's deliberate "release full potential." - Every sub-capability discloses its cost. No silent escalation, no surprise spend (cloud members are - off until `fleetCloudEscalationEnabled`). -7. **All blueprint invariants still bind:** one error taxonomy + one translator, mapped at the - boundary, bounded + non-blocking, never `NSAlert`/raw-error-in-headline, a side effect that did not - land is `.failed` never a false Done; non-activating overlays with synchronous `orderOut`; the - canonical gesture compass; reuse-don't-reinvent. - ---- - -## 6. OpenSpec authoring rules (every new slice — same as blueprint §7) - -- Mirror `openspec/changes/add-gesture-previews-and-bindings/` structure EXACTLY: `proposal.md` - (Why / What Changes / Capabilities [New + Modified] / Impact), `design.md` (Context / Goals-NonGoals / - numbered Decisions / per-component target-split + verification), `tasks.md` (numbered `## N.` with - `- [ ]` + verification notes), `specs//spec.md` deltas using - `## ADDED/MODIFIED/REMOVED Requirements` → `### Requirement:` → `#### Scenario:` WHEN/THEN. -- Read the existing spec under `openspec/specs//` first so your delta is a TRUE delta. -- Add `.openspec.yaml` (`schema: spec-driven`, `created: 2026-06-23`). -- **Do NOT write Swift / application code in the planning run — OpenSpec markdown artifacts only.** -- Each change MUST pass `openspec validate --strict`. diff --git a/docs/ai-agent-v2-blueprint.md b/docs/ai-agent-v2-blueprint.md deleted file mode 100644 index cb2e75d..0000000 --- a/docs/ai-agent-v2-blueprint.md +++ /dev/null @@ -1,479 +0,0 @@ -# AI Agent V2 — Shared Contracts Blueprint - -**Status:** binding on the 9 parallel slice architects. Read this BEFORE writing your slice's -`proposal.md` / `design.md` / `tasks.md` / spec delta. It pins the shared TYPES so we do not invent -nine conflicting `Message`/`Conversation`/`AgentSessionID`/route/index/audit shapes. - -This is the V2 evolution of the on-device AI feature: from preset, one-shot, fire-then-commit -commands into a **conversational, tool-using, memory-bearing, background-capable agent companion** on -Apple-Silicon M5. The decomposition is 9 slices, each planned by a separate architect in parallel. -This document is the only thing they all share. - ---- - -## 0. Ground truth in the existing code (what V2 EVOLVES, never forks) - -Read these before your slice. V2 reuses every one of these seams; it does not reinvent them. - -| Existing seam | File | V2 role | -|---|---|---| -| `LLMRuntime` protocol (`generate`, `structured`, `capabilities`) | `AI/LLMRuntime.swift` | The single model seam. V2 ADDS a messages/conversation entry point alongside the existing prompt-string one — additive, never breaking. | -| `LLMRequest` (`prompt`, `image: Data?`, `parameters`, `reasoning`) | `AI/LLMRuntime.swift` | The per-turn request. V2 adds a messages-bearing constructor + optional per-turn image (already present). | -| `Token` / `TokenChannel` (`.response` / `.thinking`) | `AI/LLMRuntime.swift` | The thinking-channel split ALREADY EXISTS. V2 reuses it verbatim for the conversation runtime. Do NOT add a third channel without cross-slice sign-off. | -| `StructuredSchema` / `StructuredOutcome` (`.value` / `.declined`) | `AI/LLMRuntime.swift` | The router (`ai-tool-routing`) reuses `structured()` as the tool-selection mechanism. Routing is a `structured()` call against a route schema. | -| `RuntimeError` (LocalizedError, Equatable taxonomy) | `AI/LLMRuntime.swift` | THE error taxonomy. Each slice adds at most one `Error` LocalizedError; everything maps through `AIError.message(for:)`. | -| `AIError.message(for:) -> AIPresentedError` | `AI/AIError.swift` | THE single translator. Every new error surface routes through it. Never raw-interpolate an error into a headline. | -| `AICommandExecutor` + `State` machine | `AI/AICommandExecutor.swift` | The conversational canvas (`ai-conversational-canvas`) EXTENDS this state machine; it does not replace it. New states are additive cases. | -| `TaskDispatching` / `TaskDispatcher` / `TaskReview` / `PreparedAction` / `ReviewField` / `ParsedActions` | `AI/Tasks/*` | The tool-execute layer. A "tool" in V2 is a `TaskKind`-shaped capability; the route loop calls `prepare`/`execute`. Reuse `TaskReview`/`PreparedAction` for tool-step review. | -| `ModelManager` (lifecycle, residency, registry, provisioner) | `AI/ModelManager.swift` | Owns weights/residency. The batched runtime (`ai-batched-runtime-and-context`) plugs in behind `ModelProvisioner` / `runtimeFactory`. Context-size tuning lives adjacent. | -| `AICommand` / `TaskKind` / `Destination` / `OutputTarget` / `ModelSelector` | `AI/AICommand.swift` | The persisted command value model. `claudeHandoff` per-skill config and skill-as-file migration evolve from here. | -| `AICommandCatalog` (presets → categories) | `AI/AICommandCatalog.swift` | The seed corpus migrated into skills-as-files (`ai-skills-as-files`). | -| `GestureRecognizer.trackCanvasResolution` → `launcherCanvasResolve(dx:dy:)` + `canvasResolveThreshold` + `canvasAtTop` | `Gesture/GestureRecognizer.swift`, `AICommandExecutor` | The two-finger compass. V2 ADDS overscroll-park (UP past bottom) at the CONSUMER (AppCoordinator), not in the recognizer. Recognizer emits raw `±1`; interpretation stays at the seam. | -| `DockPreviewOverlayController` pattern (non-activating, `GlobalCursorMonitor`, orientation anchor, synchronous `orderOut`) | `Overlay/DockPreviewOverlay.swift` | The template for the notch home-zone rail surface (`ai-parked-sessions`). Reuse, do not reinvent. | - ---- - -## 1. Naming conventions (binding) - -- **Capabilities / spec dirs** keep the assigned names: `ai-command-band`, `on-device-ai-runtime`, - `ai-command-tasks`, `ai-command-catalog`, `ai-memory`, `ai-parked-sessions`, - `ai-background-autonomy`, `ai-claude-handoff`. New specs go under `openspec/specs//`; - deltas go under `openspec/changes//specs//spec.md`. -- **Change directory names** = the slice names verbatim: `ai-conversational-canvas`, - `ai-conversation-runtime`, `ai-tool-routing`, `ai-skills-as-files`, `ai-agent-memory`, - `ai-parked-sessions`, `ai-background-autonomy`, `ai-claude-handoff`, - `ai-batched-runtime-and-context`. -- **Swift types:** PascalCase, no `V2` suffix (this IS the codebase now). Prefix agent-era types with - `Agent` ONLY where a bare name would collide with an existing one: `AgentMessage`, - `AgentConversation`, `AgentSessionID`, `AgentTurn`. Where there is no collision and the concept is - clearly the new world, no prefix is needed (`ToolRoute`, `SkillManifest`, `MemoryStore`, - `AuditRecord`, `ParkedSession`). -- **Error taxonomy:** one `enum Error: Error, Equatable` conforming to `LocalizedError` with a - clean per-case `errorDescription`. Allowed new ones: `MemoryError`, `AuditError`, `HandoffError`, - `ParkError` — each only if `RuntimeError`/`TaskError` genuinely cannot carry the case. Map vendor/OS - errors into the taxonomy at the layer boundary. Surface via `AIError.message(for:)`. -- **Persisted keys** (in `AppSettings` / a new store): camelCase, agent-scoped prefix where ambiguous - (`agentContextTokens`, `agentBackgroundWritePolicy`, `claudeHandoffBudgetPerDay`). -- **Files:** new agent code lives under `Sources/ThreeFingerSwitcher/AI/Agent/` (Core, MLX-free), - `AI/Memory/`, `AI/Skills/`, `AI/Parked/`, `AI/Audit/`, `AI/Handoff/`; MLX-linked batched runtime - lives in the `GemmaRuntime` target. Overlays under `Overlay/`. - ---- - -## 2. Target-split & verification (binding, per house rules) - -- **MLX-free `ThreeFingerSwitcherCore`** holds: all value types in this blueprint (messages, - conversation, route schema, skill manifest, memory store + index, audit record, park scheduler, - handoff config), the conversation state machine, the route→execute→continue loop logic, the - retrieval/index logic, the park lifecycle. **Verified by `swift build` + `swift test`.** This is the - majority of every slice. Every shared type below is Core. -- **MLX-linked `GemmaRuntime` / app target** holds: the batched MLX runtime conformer, KV-quant, - context-window wiring, the real `LLMRuntime` messages path, and the overlay panels. **Verified by - `xcodebuild` COMPILE ONLY.** Agents NEVER build/sign/install the `.app` (ad-hoc signing breaks TCC). - The user does real builds. -- Each slice's `design.md` MUST state, per component, which target it lives in and how it is verified. -- To compile-check a subset in isolation, use a throwaway `git worktree` + `swift build` — never the - shared working tree's `.app`. - ---- - -## 3. SHARED CONTRACTS (canonical sketches — binding) - -These are sketches, not final code. The OWNER slice writes the real type; CONSUMERS depend on it as -written here. If a consumer needs a change, it is a cross-slice negotiation, not a local fork. - -### 3.1 Message / Conversation / session identity — OWNER: `ai-conversation-runtime` - -```swift -// Core, MLX-free. The atom of a conversation. Reuses TokenChannel semantics for thinking. -public enum AgentRole: String, Codable, Sendable { case user, assistant, system, tool } - -public struct AgentMessage: Codable, Equatable, Identifiable, Sendable { - public let id: UUID - public var role: AgentRole - public var text: String // the committed response/user text (NEVER the thinking) - public var thinking: String? // reasoning, retained for display only; never re-fed verbatim as ground truth - public var image: Data? // optional per-turn image (PNG); mirrors LLMRequest.image - public var toolCalls: [ToolRoute]? // assistant turn that routed to tools (see 3.3) - public var toolResult: ToolStepResult? // for role == .tool: the executed step's outcome (see 3.3) - public var createdAt: Date -} - -// The session identity threaded through EVERY slice. One opaque value, stable across park/restore. -public struct AgentSessionID: Hashable, Codable, Sendable { public let raw: UUID } - -public struct AgentConversation: Codable, Equatable, Identifiable, Sendable { - public let id: AgentSessionID - public var title: String // short, model- or first-turn-derived; shown on rail/badge - public var messages: [AgentMessage] - public var createdAt: Date - public var updatedAt: Date - public var compactedSummary: String? // compaction output: prefix summary replacing old turns - public var skillID: String? // the active skill (file id) driving this session, if any -} - -// A single conversational turn unit fed to the runtime (after compaction is applied). -public struct AgentTurn: Sendable { - public var messages: [AgentMessage] // the windowed/compacted message list for this generation - public var image: Data? // convenience: latest turn's image - public var reasoning: Bool - public var parameters: GenerationParameters -} -``` - -- **Compaction** is OWNED here: when the windowed token estimate exceeds budget, old turns collapse - into `compactedSummary` (a model call) and drop from `messages`. Core holds the pure - windowing/estimate logic; the actual summarization is an `LLMRuntime` call. -- **Persistence:** `AgentConversation` is `Codable` and stored by the parked-sessions slice's store; - the runtime slice owns the type, the parked slice owns durable storage. Do not duplicate the store. - -### 3.2 Evolved `LLMRuntime` request (messages + per-turn image) — OWNER: `on-device-ai-runtime` (extended by `ai-conversation-runtime` & `ai-batched-runtime-and-context`) - -The existing `LLMRuntime` protocol and `LLMRequest(prompt:image:parameters:reasoning:)` stay. V2 adds -a messages-bearing request and a protocol method — **additive**, default-implemented so existing -conformers (`StubLLMRuntime`, `DevAIRuntime`) keep compiling. - -```swift -public struct LLMChatRequest: Sendable { - public var messages: [AgentMessage] // role-tagged multi-turn context (already compacted upstream) - public var image: Data? // optional per-turn image; vision-capable runtime required if non-nil - public var parameters: GenerationParameters - public var reasoning: Bool - public var tools: [ToolDescriptor]? // advertised tools for route-mode (see 3.3); nil = plain chat -} - -extension LLMRuntime { - // Additive. Default impl flattens messages → a single prompt and calls generate(_:) so old - // conformers work unchanged; the batched MLX conformer overrides for true KV-reuse multi-turn. - func chat(_ request: LLMChatRequest) -> AsyncThrowingStream { /* default flatten */ } -} -``` - -- The **batched runtime** (`ai-batched-runtime-and-context`) is a new `LLMRuntime` conformer that - multiplexes K sessions over one weight read. It implements `chat` + a batched-stream extension - (see 3.6). It plugs into `ModelManager` via the existing `ModelProvisioner` seam — no ModelManager - API change required. -- **Context size** is a runtime/registry property: add `maxContextTokens` to `ModelDescriptor` and a - user-adjustable `agentContextTokens` (clamped to model max), with RAM/speed cost surfaced in the UI. - Owned by `ai-batched-runtime-and-context`; consumed by conversation-runtime's compaction budget. - -### 3.3 Tool ROUTE schema — OWNER: `ai-tool-routing` - -The router uses the EXISTING `structured()` call as the selection mechanism (house rule: reuse seams). -A tool is a `TaskKind`-shaped capability; routing decides which tool(s) to call with which arguments, -then the route→execute→continue loop runs them via `TaskDispatching`. - -```swift -// A tool the model may call. Describes itself to the router (name + JSON-Schema args). -public struct ToolDescriptor: Codable, Equatable, Sendable { - public let name: String // stable id, e.g. "add_to_calendar", "memory.write", "launch_claude" - public let summary: String // one line the model sees - public let argsSchema: StructuredSchema // reuse the runtime's schema type - public let writePolicy: WritePolicyTier // see 3.7 — drives auto/confirm/escalate -} - -// The model's decision for one step. Produced via runtime.structured(routeSchema, as: ToolRoute.self). -public struct ToolRoute: Codable, Equatable, Sendable { - public let tool: String // matches a ToolDescriptor.name, or "" for a plain text answer - public let argumentsJSON: String // JSON object string validated against that tool's argsSchema - public let rationale: String? // short, for the audit log + thinking display -} - -// The outcome of executing one routed step. Reuses TaskReview/PreparedAction underneath. -public struct ToolStepResult: Codable, Equatable, Sendable { - public let tool: String - public let status: ToolStepStatus // .done / .declined(reason) / .failed(headline) / .awaitingApproval - public let summary: String // short human-readable outcome, fed back into the loop as a tool message -} -public enum ToolStepStatus: Codable, Equatable, Sendable { - case done, awaitingApproval - case declined(reason: String) - case failed(headline: String) // clean headline only (AIPresentedError.headline); raw text to logs -} -``` - -- **The loop** (`route → execute → continue`) is OWNED here, pure in Core: ask `structured()` for a - `ToolRoute`; if `tool == ""`, stream a text answer and stop; else look up the `ToolDescriptor`, - validate args, dispatch via `TaskDispatching`, append a `.tool` `AgentMessage` carrying - `ToolStepResult`, and re-enter. Bounded iteration count. -- **Approval grammar (canvas):** a tool step whose `writePolicy` requires confirmation surfaces a - `TaskReview`-backed step; the canvas resolves it with **DOWN=approve / RIGHT=skip** (mirrors - commit/discard). `awaitingApproval` is the observable state. -- Tools are registered in a `ToolRegistry` (Core) that aggregates: the existing `TaskKind` tasks, - memory read/write tools (from `ai-agent-memory`), skill-invocation, and `launch_claude` (from - `ai-claude-handoff`). Each contributor exposes `ToolDescriptor`s; routing slice owns the registry. - -### 3.4 Shared retrieval / index shape — OWNER: `ai-skills-as-files` (the index); CO-OWNED contract with `ai-agent-memory` - -Skills and memory are BOTH declarative files with a table-of-contents + on-demand retrieval. They -share ONE index/retrieval shape so we do not build two retrievers. - -```swift -// One retrievable unit (a skill file, a memory subfile, a memory TOC entry). -public struct IndexedDoc: Codable, Equatable, Identifiable, Sendable { - public let id: String // stable: file path-relative id - public var title: String - public var summary: String // the TOC line — what the retriever ranks/returns first - public var keywords: [String] - public var kind: DocKind // .skill / .memoryCore / .memorySubfile - public var bodyPath: URL // lazily loaded; retrieval returns summaries, body on demand - public var updatedAt: Date -} -public enum DocKind: String, Codable, Sendable { case skill, memoryCore, memorySubfile } - -// The shared retriever seam. Pure, synchronous over an in-memory index; file IO is bridged off-main -// by the owning store (mirrors the Files-band sync model + async cache pattern). -public protocol DocIndex: Sendable { - func allSummaries() -> [IndexedDoc] // the TOC the model always sees - func retrieve(query: String, limit: Int) -> [IndexedDoc] // ranked summaries for on-demand expansion - func body(of id: String) throws -> String // load a doc's full body when the model asks -} -``` - -- **Skills format** (`ai-skills-as-files`): a declarative file (front-matter `title/summary/keywords` - + a prompt/body + optional `claudeHandoff` config + optional `tools` allow-list). The catalog - migration (`AICommandCatalog` presets → skill files) is owned here. The skill folder + index/load is - owned here; it produces `IndexedDoc(kind: .skill)`. -- **Memory** (`ai-agent-memory`): two-tier — a core ground-truth file + TOC, plus named subfiles, - editable by agent and user. It produces `IndexedDoc(kind: .memoryCore/.memorySubfile)` over the SAME - `DocIndex` seam. Memory owns its store + read/write tools; it reuses the retriever contract, it does - not define a second one. -- **Retrieval mechanic:** the model always sees `allSummaries()` (the combined TOC); it requests a - body via a `retrieve`/`read` tool (a `ToolDescriptor`), so retrieval is itself a routed tool step. - -### 3.5 Parked-session scheduler interface — OWNER: `ai-parked-sessions` - -The batched runtime plugs into THIS. Parked sessions are durable conversations the agent advances in -the background; the scheduler decides which session each batch slot serves. - -```swift -public enum ParkState: Codable, Equatable, Sendable { - case active // foreground, in the canvas - case parked // stashed at the notch home zone, may run in background - case needsYou // a dangerous write / approval escalated to foreground (badge) - case idle // nothing pending -} - -public struct ParkedSession: Codable, Equatable, Identifiable, Sendable { - public var id: AgentSessionID // SAME identity as AgentConversation.id (3.1) - public var title: String - public var state: ParkState - public var badgeCount: Int // unseen results / needs-you items, shown on the rail - public var nextRunAt: Date? // scheduler hint (a timed/scheduled continuation) - public var updatedAt: Date -} - -// The seam the batched runtime consumes to pick work. Pure scheduling decisions in Core. -public protocol ParkScheduler: Sendable { - func runnableSessions(now: Date, maxSlots: Int) -> [AgentSessionID] // which parked sessions to advance - func didAdvance(_ id: AgentSessionID, result: ToolStepResult) // feedback after a batch step - func escalate(_ id: AgentSessionID, reason: String) // → .needsYou + badge (dangerous write) -} -``` - -- The **batched runtime** (`ai-batched-runtime-and-context`) calls `runnableSessions(now:maxSlots:)` - to fill its K batch slots, advances each via `chat`, and reports back via `didAdvance`. The scheduler - itself is pure/testable; `now:` is an input (mirrors `DockHoverModel`). -- The **notch home zone + rail** is a `DockPreviewOverlay`-pattern non-activating panel (synchronous - `orderOut`). Park is triggered by **overscroll-past-bottom** (UP excursion past the canvas bottom) — - the consumer (AppCoordinator) interprets the recognizer's raw direction; the recognizer is unchanged. - -### 3.6 Batched runtime extension — OWNER: `ai-batched-runtime-and-context` - -```swift -// MLX-linked. A new LLMRuntime conformer; plugs into ModelManager via ModelProvisioner. -// K streams share ONE weight read per token step. KV cache is quantized per stream. -protocol BatchedLLMRuntime: LLMRuntime { - // Advance up to K sessions one decode step each, sharing the weight read. - func batchStep(_ requests: [AgentSessionID: LLMChatRequest]) -> AsyncThrowingStream<(AgentSessionID, Token), Error> - var maxConcurrentStreams: Int { get } // K -} -``` - -- Consumes `ParkScheduler.runnableSessions` for which sessions to batch; consumes `LLMChatRequest` - (3.2) per session; emits `Token`s keyed by `AgentSessionID`. Foreground active session always gets a - slot. KV-quant + context tuning are internal to this slice; the user-facing context slider - (`agentContextTokens`, RAM/speed cost surfaced) is owned here. - -### 3.7 Write-policy tiers + audit record — OWNER: `ai-background-autonomy` - -```swift -public enum WritePolicyTier: String, Codable, Equatable, Sendable { - case auto // whitelisted/safe: runs without confirm, even when parked (still audited) - case confirm // default: needs foreground approval (DOWN=approve / RIGHT=skip) - case dangerous // always escalates to foreground via needs-you badge, even if parked -} - -// One row in the append-only background audit log. Every tool step (auto or not) writes one. -public struct AuditRecord: Codable, Equatable, Identifiable, Sendable { - public let id: UUID - public let sessionID: AgentSessionID - public let tool: String - public let policy: WritePolicyTier - public let argumentsSummary: String // redacted/short args, NOT raw secrets - public let outcome: ToolStepStatus // reuses 3.3 - public let wasBackground: Bool // true if applied while parked - public let timestamp: Date -} - -public protocol AuditLog: Sendable { - func record(_ r: AuditRecord) - func recent(limit: Int) -> [AuditRecord] -} -``` - -- **User decision (adopted, do not relitigate):** background memory writes + whitelisted writes are - **AUTO even when parked** (still audited). **Dangerous** writes escalate to foreground via the - needs-you badge (`ParkState.needsYou`). The **user whitelist** is owned here: which tools/skills are - `.auto`. The routing slice reads `writePolicy` off the `ToolDescriptor`; this slice owns the - policy resolution (descriptor default ∩ user whitelist → effective tier) and the audit log. - -### 3.8 Per-skill Claude handoff config — OWNER: `ai-claude-handoff` - -```swift -public enum HandoffConfirmMode: String, Codable, Equatable, Sendable { - case confirm // DEFAULT: confirm per call - case auto // per-skill opt-in: fire without confirm (still budget-capped + audited) -} - -public struct ClaudeHandoffConfig: Codable, Equatable, Sendable { - public var enabled: Bool - public var confirmMode: HandoffConfirmMode // default .confirm - public var maxCallsPerDay: Int // budget/rate cap - public var maxConcurrent: Int - // v1 is FIRE-AND-FORGET: launch Claude Code with a prompt/cwd; no structured round-trip back. - public var defaultWorkingDirectory: URL? -} -``` - -- **User decision (adopted):** handoff defaults to **CONFIRM per call**, with **per-skill AUTO - opt-in** and a **budget/rate cap**; **v1 handoff is FIRE-AND-FORGET** (round-trip is documented - future, out of scope). `launch_claude` is a `ToolDescriptor` (write-policy `.confirm` unless the - skill opts into `.auto`); the handoff slice owns launching `claude` (process spawn, like the existing - open-claude-here change). Per-skill config rides on the skill file (3.4) as an optional block. - ---- - -## 4. Dependency DAG & implementation order (coarse, binding) - -``` -on-device-ai-runtime (LLMChatRequest, chat(), context props) ── foundational, do first - │ - ├──> ai-conversation-runtime (AgentMessage/Conversation/SessionID, compaction) ── the type home - │ │ - │ ├──> ai-tool-routing (ToolRoute/ToolDescriptor/ToolStepResult, the loop, ToolRegistry) - │ │ │ - │ │ ├──> ai-skills-as-files (DocIndex, skill files, catalog migration) ┐ - │ │ ├──> ai-agent-memory (DocIndex consumer, memory tools) ├ share DocIndex (3.4) - │ │ └──> ai-claude-handoff (launch_claude tool, ClaudeHandoffConfig) ┘ - │ │ - │ └──> ai-conversational-canvas (extends AICommandExecutor.State, the compass, park-trigger) - │ - └──> ai-batched-runtime-and-context (BatchedLLMRuntime, KV-quant, context tuning) - │ - └──> ai-parked-sessions (ParkedSession/ParkState/ParkScheduler, notch rail, lifecycle) - │ - └──> ai-background-autonomy (WritePolicyTier, AuditRecord, whitelist) -``` - -**Coarse order (waves):** -1. **Wave 1 (foundations, define-the-types):** `on-device-ai-runtime` (chat request + context props), - `ai-conversation-runtime` (message/conversation/session types + compaction). -2. **Wave 2 (routing + batched core):** `ai-tool-routing` (route schema + loop + registry), - `ai-batched-runtime-and-context` (batched conformer + context tuning). -3. **Wave 3 (capability providers, parallel):** `ai-skills-as-files`, `ai-agent-memory`, - `ai-parked-sessions`. -4. **Wave 4 (policy + UX + handoff):** `ai-background-autonomy`, `ai-claude-handoff`, - `ai-conversational-canvas`. - -Architects plan in parallel NOW regardless of wave; the waves are the *implementation* order so a -later slice can assume an earlier slice's types exist. If you OWN a Wave-1/2 type, write it exactly as -3.x; if you CONSUME it, import it as written and do not redefine. - ---- - -## 5. Per-slice interface notes (MUST expose / MUST consume) - -**1. ai-conversational-canvas** (cap `ai-command-band`) -- MUST EXPOSE: additive `AICommandExecutor.State` cases for multi-turn (`.conversing`, `.awaitingApproval(ToolReview)`, `.parked`); the seed → float-up → gesture-compass UX; the park-on-overscroll trigger consumed by `ai-parked-sessions`. -- MUST CONSUME: `AgentConversation`/`AgentMessage` (3.1); the route loop's observable state (3.3); the CANONICAL compass — DOWN=affirm (only when `canvasAtTop`), UP=scroll/overscroll-park, RIGHT=discard, LEFT=reserved, Enter=send; `canvasResolveThreshold`. Reuse `AICommandCanvasView`, `BubbleMorph`, `TaskReview` rendering. - -**2. ai-conversation-runtime** (cap `on-device-ai-runtime`) -- MUST EXPOSE: `AgentMessage`/`AgentRole`/`AgentConversation`/`AgentSessionID`/`AgentTurn` (3.1); the compaction (windowing + summary) logic; the conversation→`LLMChatRequest` assembly. -- MUST CONSUME: `LLMChatRequest`/`chat()` (3.2); `TokenChannel` for thinking; `agentContextTokens` budget from `ai-batched-runtime-and-context`. - -**3. ai-tool-routing** (cap `ai-command-tasks`) -- MUST EXPOSE: `ToolDescriptor`/`ToolRoute`/`ToolStepResult`/`ToolStepStatus` (3.3); the `ToolRegistry`; the pure route→execute→continue loop (bounded); the approval-step contract (DOWN=approve/RIGHT=skip). -- MUST CONSUME: `structured()` + `StructuredSchema`/`StructuredOutcome` as the router; `TaskDispatching`/`TaskReview`/`PreparedAction` for execution; `WritePolicyTier` (3.7) off each descriptor; `AgentMessage` (3.1) to append `.tool` turns. - -**4. ai-skills-as-files** (cap `ai-command-catalog`) -- MUST EXPOSE: the skill file format (front-matter + body + optional `claudeHandoff`/`tools`); the skill folder layout; `IndexedDoc(kind: .skill)` over `DocIndex` (3.4); the `AICommandCatalog` → skill-files migration (idempotent, identity-preserving). -- MUST CONSUME: `DocIndex` (3.4, OWNS the index but shares the contract with memory); `ToolDescriptor` for the skill's allowed tools; `ClaudeHandoffConfig` (3.8) as an optional skill block. - -**5. ai-agent-memory** (cap `ai-memory`) -- MUST EXPOSE: the two-tier store (core ground-truth + TOC, named subfiles); `IndexedDoc(kind: .memoryCore/.memorySubfile)` over the SHARED `DocIndex` (3.4); memory read/write `ToolDescriptor`s (write tools carry `WritePolicyTier`). -- MUST CONSUME: `DocIndex` contract (3.4, do NOT define a second retriever); `WritePolicyTier`/`AuditRecord` (3.7) for edits; `AgentSessionID` (3.1) for attribution. - -**6. ai-parked-sessions** (cap `ai-parked-sessions`) -- MUST EXPOSE: `ParkedSession`/`ParkState`/`ParkScheduler` (3.5); the durable `AgentConversation` store; the notch home-zone rail + badges (DockPreviewOverlay pattern, synchronous teardown); the lifecycle (park/restore/needs-you). -- MUST CONSUME: `AgentConversation`/`AgentSessionID` (3.1); the canvas overscroll-park trigger (slice 1); the batched runtime as the background advancer (3.6); `ParkState.needsYou` escalation from `ai-background-autonomy`. - -**7. ai-background-autonomy** (cap `ai-background-autonomy`) -- MUST EXPOSE: `WritePolicyTier` (3.7); the `AuditRecord`/`AuditLog` (3.7); the user whitelist + effective-tier resolution; the auto-when-parked vs escalate-dangerous decision. -- MUST CONSUME: `ToolDescriptor.writePolicy` (3.3); `ParkScheduler.escalate` (3.5) to raise `.needsYou`; `AgentSessionID` (3.1); `AIError`/`AIPresentedError` for clean headlines in audit + escalation. - -**8. ai-claude-handoff** (cap `ai-claude-handoff`) -- MUST EXPOSE: `ClaudeHandoffConfig` (3.8); the `launch_claude` `ToolDescriptor`; fire-and-forget process launch; the budget/rate-cap enforcement; a `HandoffError` if needed. -- MUST CONSUME: `ToolDescriptor`/route loop (3.3); `WritePolicyTier` (default `.confirm`, per-skill `.auto`); `AuditRecord` (3.7); the per-skill block on the skill file (3.4). Reuse the existing open-claude-here launch path. - -**9. ai-batched-runtime-and-context** (cap `on-device-ai-runtime`) -- MUST EXPOSE: `BatchedLLMRuntime` (3.6); KV-quant + K-stream batching; `ModelDescriptor.maxContextTokens` + the user-adjustable `agentContextTokens` (RAM/speed cost surfaced in UI); plug-in via `ModelProvisioner`. -- MUST CONSUME: `LLMChatRequest` (3.2); `ParkScheduler.runnableSessions` (3.5) to fill batch slots; `AgentSessionID` (3.1) to key streams; existing `ModelManager` residency seam. - ---- - -## 6. Cross-cutting user decisions (adopted — every architect honors) - -1. **Hardware:** Apple-Silicon M5/M4 ONLY (this machine: M5 Pro, 16-core GPU, 48GB unified). No - Intel/low-end fallbacks, no defensive degraded paths. The batched runtime + growable context are in - scope precisely because the hardware can serve them. -2. **Errors:** ONE taxonomy (`RuntimeError`/`TaskError` + at most one `Error`), ONE translator - (`AIError.message(for:)` → `AIPresentedError`), mapped at the layer boundary, surfaced BOUNDED + - NON-BLOCKING. Never `NSAlert.runModal`. Never raw error text in a headline (logs/opt-in details - only). A failure is an observable `.failed`/`.declined` state with a clean headline; a side effect - that did not land is `.failed`, never a false "Done." -3. **Overlays:** non-activating panels, SYNCHRONOUS `orderOut` teardown (ghost-on-Space-switch bug). - The notch rail + any cursor-reveal surface reuse the `DockPreviewOverlay` pattern (edge-gated - `GlobalCursorMonitor`, orientation-aware anchor, mouse-interactive non-key panel). -4. **Canonical gesture compass** (two-finger, post-activation, in the AI canvas): - **DOWN = affirm** (extract latest answer / approve a step) — fires only when `canvasAtTop`. - **UP = scroll**, and **overscroll-past-bottom = PARK** to the notch home zone. - **RIGHT = discard.** **LEFT = reserved** (free for now). **Enter (keyboard) = send the turn.** - Honor `canvasResolveThreshold` (above incidental two-finger scroll). Spatial mnemonic: TOP of canvas - = act on it, BOTTOM = stash it. Tool-step approval reuses DOWN=approve / RIGHT=skip. The recognizer - keeps emitting raw `±1`; interpretation lives at the consumer seam. -5. **Reuse, do not reinvent:** `LLMRuntime`, `AICommandExecutor` + its `State`, `TaskDispatching` / - `TaskReview` / `PreparedAction` / `ParsedActions` / `TaskSinks`, `SelectionProviding` / - `SelectionService`, `ModelManager`, `PromptTemplate`, `BubbleMorph`, `GlobalCursorMonitor`, - `DockHoverModel` anchoring, `ClipboardBandLayout` metrics. -6. **Background autonomy:** background memory writes + whitelisted writes are AUTO even when parked - (still audited); dangerous writes escalate to foreground via the needs-you badge. -7. **Claude handoff:** defaults to CONFIRM per call; per-skill AUTO opt-in; budget/rate cap; v1 is - FIRE-AND-FORGET (round-trip is a documented future). -8. **Batched runtime is in scope NOW** (this is V2). **Context is growable to the model max AND - user-adjustable, with RAM/speed cost surfaced in the UI.** - ---- - -## 7. OpenSpec authoring rules (every slice) - -- Mirror the structure of `openspec/changes/add-gesture-previews-and-bindings/` EXACTLY: - `proposal.md` (Why / What Changes / Capabilities [New + Modified] / Impact), `design.md` - (Context / Goals-Non-Goals / Decisions [numbered] / target-split + verification per component), - `tasks.md` (numbered `## N.` sections, `- [ ]` checkboxes with verification notes), and - `specs//spec.md` deltas using `## ADDED/MODIFIED/REMOVED Requirements` with - `### Requirement:` + `#### Scenario:` WHEN/THEN phrasing. -- Read the relevant existing spec under `openspec/specs//` first so your delta is a TRUE - delta (ADDED/MODIFIED), not a rewrite. -- Add a `.openspec.yaml` (`schema: spec-driven`, `created:` date) to your change dir. -- Do NOT write Swift / application code in the planning run — OpenSpec artifacts only. diff --git a/docs/ai-agent-v2-roadmap.md b/docs/ai-agent-v2-roadmap.md deleted file mode 100644 index 6150edf..0000000 --- a/docs/ai-agent-v2-roadmap.md +++ /dev/null @@ -1,241 +0,0 @@ -# AI Agent V2 — Implementation Roadmap - -**Status:** integration-welded. Nine parallel slice architects each authored an OpenSpec change under -`openspec/changes//`; this document is the integration architect's synthesis — the index, the -shared-contract summary, the sequenced implementation order, the cross-slice conflicts found and their -resolutions, and the open questions still needing a human decision. - -The binding contract sketches live in **`docs/ai-agent-v2-blueprint.md`** (read it first). This roadmap -records how the nine real artifacts line up against that blueprint and the order in which -`/opsx:apply` should walk them. - -All nine changes pass `openspec validate --strict`. - ---- - -## 1. The nine changes (index) - -| # | Change | Capability | Wave | One-line role | -|---|---|---|---|---| -| 1 | [`ai-conversation-runtime`](../openspec/changes/ai-conversation-runtime/) | `on-device-ai-runtime` (delta) | 1 | The **type home**: `AgentMessage`/`AgentConversation`/`AgentSessionID`/`AgentTurn`, `LLMChatRequest` + default-flattened `chat()`, compaction. Seven slices import its types. | -| 2 | [`ai-tool-routing`](../openspec/changes/ai-tool-routing/) | `ai-command-tasks` (delta) | 2 | Inverts control: the **model** picks a tool via a `structured()` route turn; the bounded route→execute→continue loop reuses `TaskDispatcher`/`TaskReview` wholesale. Owns `ToolDescriptor`/`ToolRoute`/`ToolStepResult`/`WritePolicyTier`/`ToolRegistry`. | -| 3 | [`ai-batched-runtime-and-context`](../openspec/changes/ai-batched-runtime-and-context/) | `on-device-ai-runtime` + `tunable-settings` (delta) | 2 | The **MLX** slice: continuous-batching `BatchedLLMRuntime` (one weight read, K streams), KV-quant, RAM-derived concurrency budget, growable user-adjustable context with cost surfaced, the subagent primitive. | -| 4 | [`ai-skills-as-files`](../openspec/changes/ai-skills-as-files/) | `ai-command-catalog` + `ai-skill-index` (new) | 3 | Externalizes the 75-preset catalog into declarative `.skill.md` files; **owns** the shared `DocIndex`/`IndexedDoc`/`DocKind` retriever; idempotent identity-preserving catalog→files migration. | -| 5 | [`ai-agent-memory`](../openspec/changes/ai-agent-memory/) | `ai-memory` (new) | 3 | Two-tier on-disk memory (capped CORE + named subfiles) **over the shared `DocIndex`** (no second retriever); memory read/write tools carrying `WritePolicyTier`. | -| 6 | [`ai-parked-sessions`](../openspec/changes/ai-parked-sessions/) | `ai-parked-sessions` (new) | 3 | `ParkedSession`/`ParkState`/`ParkScheduler`, the durable conversation store, the notch home-zone rail + ambient needs-you glow (DockPreviewOverlay pattern), park/sleep/discard/evict lifecycle. | -| 7 | [`ai-background-autonomy`](../openspec/changes/ai-background-autonomy/) | `ai-background-autonomy` + `configuration-hub` (delta) | 4 | The **policy layer**: blast-radius tiers over `WritePolicyTier`, the user whitelist + effective-tier resolution, the append-only `AuditRecord`/`AuditLog`, the parked auto-vs-escalate gate. | -| 8 | [`ai-claude-handoff`](../openspec/changes/ai-claude-handoff/) | `ai-claude-handoff` (new) | 4 | `launch_claude` as a `.dangerous` `ToolContributor` reusing the open-claude-here `.command` launch; `ClaudeHandoffConfig`, fire-and-forget, rolling-24h budget cap, per-skill `auto` opt-in. | -| 9 | [`ai-conversational-canvas`](../openspec/changes/ai-conversational-canvas/) | `ai-command-band` (delta) | 4 | The **front end**: seed→float-up→multi-turn typed canvas, the canonical two-finger compass, the overscroll-park trigger; extends `AICommandExecutor.State` with `.awaitingSeed` + renders sibling states. | - ---- - -## 2. Shared-contract summary (who owns what, who consumes it) - -| Contract (blueprint §) | Owner | Concrete type / file | Consumers | -|---|---|---|---| -| Message / Conversation / session identity (§3.1) | `ai-conversation-runtime` | `AgentRole`/`AgentMessage`/`AgentSessionID`/`AgentConversation`/`AgentTurn` in `AI/Agent/AgentConversation.swift` | tool-routing, canvas, parked, memory, autonomy, batched, handoff | -| `LLMChatRequest` + `LLMRuntime.chat()` + `ContextBudgetProviding` (§3.2) | `ai-conversation-runtime` (request + injected budget seam); `ai-batched-runtime-and-context` (concrete provider + `ModelDescriptor.maxContextTokens`/`agentContextTokens`) | additive on `AI/LLMRuntime.swift`; default-flattened `chat()` | tool-routing, batched, canvas | -| Tool route schema (§3.3) | `ai-tool-routing` | `ToolDescriptor`/`ToolRoute`/`ToolStepResult`/`ToolStepStatus`/`WritePolicyTier`/`ToolRegistry`/`ApprovalGate`/`WritePolicyResolving` in `AI/Agent/ToolContracts.swift` | skills, memory, handoff, autonomy, canvas, conversation-runtime (placeholder hand-off) | -| Shared retrieval index (§3.4) | `ai-skills-as-files` (OWNS) | `DocIndex`/`IndexedDoc`/`DocKind` + `InMemoryDocIndex`; new `ai-skill-index` capability spec | `ai-agent-memory` (contributes `IndexedDoc`s into the **same merged snapshot**), tool-routing (candidate source mirrors `retrieve(query:limit:)`) | -| Parked scheduler (§3.5) | `ai-parked-sessions` | `ParkState`/`ParkedSession`/`ParkScheduler` + `SerialParkScheduler` | batched (`runnableSessions`/`didAdvance`), autonomy (`escalate`), canvas (restore + overscroll-park) | -| Batched runtime (§3.6) | `ai-batched-runtime-and-context` | `BatchedLLMRuntime` (Core protocol) + `BatchedGemmaMLXRuntime` (GemmaRuntime conformer) | parked (background advancer), conversation-runtime (real KV-reuse `chat()` override) | -| Write-policy + audit (§3.7) | `ai-background-autonomy` (resolution/whitelist/audit/escalation); `ai-tool-routing` (the bare `WritePolicyTier` enum, C1) | `Whitelist`/`BackgroundPolicyResolver`/`BackgroundGate`/`AuditRecord`/`AuditLog` | tool-routing (reads `writePolicy`), memory + handoff (set descriptor tier, emit records), parked (`escalate` target) | -| Claude handoff config (§3.8) | `ai-claude-handoff` | `ClaudeHandoffConfig`/`HandoffConfirmMode` | `ai-skills-as-files` (optional `claudeHandoff:` skill-file block carrier), tool-routing (`launch_claude` descriptor), autonomy (audit + tier) | - -**Cross-cutting invariants honored by all nine:** Apple-Silicon M5/M4 only (no low-end fallbacks); one -error taxonomy (`RuntimeError`/`TaskError` + at most one `Error` — the new ones are `SkillError`, -`MemoryError`, `ParkError`, `AuditError`, `HandoffError`, each justified) routed through the single -`AIError.message(for:)` translator, mapped at the layer boundary, surfaced bounded + non-blocking, never -`NSAlert.runModal`, never raw error text in a headline, a side effect that did not land is `.failed` never -a false Done; overlays are non-activating panels with synchronous `orderOut` teardown reusing the -DockPreviewOverlay pattern; the canonical two-finger compass (DOWN=affirm-only-at-top / UP=scroll & -overscroll-past-bottom=PARK / RIGHT=discard / LEFT=reserved / Enter=send; tool-step approval reuses -DOWN=approve / RIGHT=skip) with interpretation at the `AppCoordinator` consumer seam, recognizer -untouched. - ---- - -## 3. Cross-slice consistency check - -Verified the shared types match across every consuming slice, the dependency graph is acyclic, the gesture -verbs are uniform, and every MLX-linked piece is marked `xcodebuild` compile-verify-only. Findings below; -all are resolvable without re-planning a slice. - -### Verified consistent - -- **Message/Conversation/session** — `ai-conversation-runtime` writes §3.1 verbatim; tool-routing, parked, - canvas, memory, autonomy, batched, handoff all consume `AgentMessage`/`AgentConversation`/`AgentSessionID` - unchanged. `AgentSessionID` is the stable park/restore + batch-stream key everywhere. -- **`DocIndex` retrieval** — `ai-skills-as-files` owns it (new `ai-skill-index` spec); `ai-agent-memory`'s - design and the index spec agree the index is **kind-agnostic** and the combined skills+memory TOC is one - enumeration. No second retriever. Both reference the Files-band sync-model + async-cache bridge identically. -- **`WritePolicyTier` placement (C1)** — defined once with `ToolDescriptor` in `ai-tool-routing`; memory, - skills, handoff, autonomy all *read/set* it, none redefine it. `ai-background-autonomy` owns resolution - (descriptor ∩ whitelist), never widening `.dangerous`. -- **`ParkScheduler` is K-ready** — `runnableSessions(now:maxSlots:)` is the seam; `SerialParkScheduler` - returns ≤1 now, the batched runtime's `ConcurrentParkScheduler` returns ≤K later with no protocol change. - `now:` is an input everywhere (deterministic, DockHoverModel-style). -- **`AuditRecord`/`AuditLog`** — `ai-background-autonomy` writes §3.7 verbatim; memory and handoff emit - records through narrow seams (e.g. `HandoffAuditing`) re-pointed at the real `AuditLog`. `record()` is - non-blocking and never throws into the route loop in all three. -- **`ClaudeHandoffConfig`** — owned by `ai-claude-handoff`, carried as an optional skill-file block by - `ai-skills-as-files`; exactly one definition. Descriptor stays `.dangerous`; `auto` is a per-skill - effective-tier downgrade, never a descriptor change. -- **Gesture compass** — canvas D7, parked D4, and tool-routing's approval contract all use DOWN=affirm/approve - (only at `canvasAtTop`), RIGHT=discard/skip, UP=scroll + overscroll-park, Enter=send. Recognizer untouched - in all; interpretation at `AppCoordinator`. -- **Target split** — every MLX-linked component (the batched decode loop, KV-quant, prefix cache in - `ai-batched-runtime-and-context`; the GemmaRuntime `chat()` override flagged by `ai-conversation-runtime`) - is marked `xcodebuild` compile-verify-only + user run-verify. All other slices are MLX-free Core verified by - `swift test`. Overlay panels (parked rail, canvas float-up) are `xcodebuild` compile-verify + user run-verify. -- **No dependency cycle** — the DAG is acyclic. The only apparent back-edge (conversation-runtime's compaction - needing the batched runtime's context budget) is broken by the injected `ContextBudgetProviding` seam (C3): - conversation-runtime owns the protocol + a default constant provider and lands first; batched supplies the - concrete provider later. -- **Two `on-device-ai-runtime` deltas don't collide** — `ai-conversation-runtime` MODIFIES "Swappable model - runtime abstraction" + ADDs conversation requirements; `ai-batched-runtime-and-context` ADDs disjoint - batched/context requirements. Requirement names are disjoint, so the synced capability spec composes. - -### Conflicts found + resolutions - -1. **`ToolDescriptor` shape divergence (additive).** `ai-tool-routing`'s owned `ToolDescriptor` adds a - `keywords: [String]` field (for its `KeywordToolCandidateSource`) beyond the blueprint §3.3 sketch. - `ai-conversation-runtime` ships a placeholder `ToolDescriptor` "exactly per §3.3" (no `keywords`), and - `ai-skills-as-files`/`ai-agent-memory` project to the §3.3 shape. **Resolution:** `ai-tool-routing`'s - `ToolDescriptor` (with `keywords`) is canonical. The conversation-runtime placeholder is a deliberate - subset that `ai-tool-routing` **replaces wholesale** when it takes ownership (the hand-off is "take - ownership," not "freeze shape"); adding a field is non-breaking. Skills/memory must populate `keywords` - when projecting (they already author `keywords` in front-matter, so the data exists). Low risk. - -2. **Placeholder file vs owned file name.** `ai-conversation-runtime` introduces - `AI/Agent/ToolPlaceholders.swift` (marked `// HAND-OFF: ai-tool-routing takes ownership`); - `ai-tool-routing` lands the real types in `AI/Agent/ToolContracts.swift`. **Resolution:** when - `ai-tool-routing` applies, it must **delete `ToolPlaceholders.swift`** and move the types to - `ToolContracts.swift` (not leave both — duplicate type definitions would not compile). Call this out in - the tool-routing apply step. Mechanical. - -3. **Two slices assumed siblings were "not on disk."** `ai-tool-routing` (Q1) wrote a stand-in - `ConversationSeam` (`ConversationContext`/`ChatStreaming`) and `ai-claude-handoff` wrote - `HandoffAuditing`/`HandoffEscalating` no-op seams because they planned before the owner slices were - committed. **Resolution:** all owner slices ARE now on disk with final types. When applying in wave order - (below), tool-routing binds directly to `AgentMessage`/`chat()` and **deletes `ConversationSeam`**; handoff - re-points its seams at the real `AuditLog`/`ParkScheduler.escalate`. The blueprint §3.1/§3.5/§3.7 sketches - are final enough to bind (confirmed — the committed types match the sketches). No shape renegotiation - needed; only the stand-in seams are removed on bind. - -4. **`InMemoryDocIndex` must accept externally-contributed docs.** `ai-skills-as-files` owns - `InMemoryDocIndex`; `ai-agent-memory` contributes its `IndexedDoc`s into the **same merged snapshot**. - Both flagged the risk that skills might build the index skills-private. **Resolution:** the `ai-skill-index` - spec already requires the index be "kind-agnostic … a combined skills-and-memory table of contents is one - enumeration." `InMemoryDocIndex` must therefore take its snapshot as a **merge of N contributors' - `[IndexedDoc]`** (skills store ∪ memory store), not a skills-internal list. Bind memory's - `MemoryStore.indexedDocs()` into the same snapshot builder. This is the documented contract; flag it in the - skills apply step so the index is built merge-shaped from day one. - -5. **Missing `.openspec.yaml` in three changes (now fixed).** `ai-background-autonomy`, `ai-claude-handoff`, - and `ai-conversational-canvas` were missing the `.openspec.yaml` the blueprint §7 authoring rule requires - (the other six had it). **Resolution:** created all three (`schema: spec-driven`, `created: 2026-06-22`) - to match the existing six. Done as part of this integration pass. - -6. **`canvasAtBottom` ownership.** `ai-conversational-canvas` (open Q) asked whether it or `ai-parked-sessions` - owns the `canvasAtBottom` reporter. **Resolution (settled):** the **canvas** owns the `CanvasAtBottomKey` - reporter + the `executor.canvasAtBottom` field (symmetric to the existing `canvasAtTop`); **parked-sessions** - owns the pure `OverscrollPark.shouldPark(dy:canvasAtBottom:overscrollThreshold:)` decision it feeds. - Both designs already describe exactly this split — no code conflict, just confirm the boundary at apply time. - ---- - -## 4. Implementation order (topological over the dependency DAG) - -Apply in this order. Within a wave, slices are independent and may be applied in any sub-order; the wave -boundary is the hard sequencing constraint (a later wave assumes earlier-wave types exist on disk). - -1. **`ai-conversation-runtime`** — Wave 1, the type home. Everything imports its `AgentMessage`/ - `AgentConversation`/`AgentSessionID`/`AgentTurn` + `LLMChatRequest`/`chat()`. Ships the placeholder tool - types + the injected `ContextBudgetProviding` so it builds standalone before Waves 2–4. -2. **`ai-tool-routing`** — Wave 2. Owns the route schema + loop + `ToolRegistry` + `WritePolicyTier`. On - apply: delete the conversation-runtime placeholder `ToolPlaceholders.swift` + its own `ConversationSeam` - stand-in, bind to the real types (conflicts 2, 3). -3. **`ai-batched-runtime-and-context`** — Wave 2 (parallel with tool-routing; depends only on - conversation-runtime's `LLMChatRequest`). Supplies the concrete `ContextBudgetProviding` + `BatchedLLMRuntime`; - consumes `ParkScheduler.runnableSessions` (a stub until Wave 3's scheduler lands — its pure budget/scheduling - logic is tested against a stub, the MLX decode loop is compile-verify-only). -4. **`ai-skills-as-files`** — Wave 3. Owns `DocIndex`; build `InMemoryDocIndex` as a merge-of-contributors - snapshot (conflict 4). Depends on tool-routing's `ToolDescriptor`. -5. **`ai-agent-memory`** — Wave 3. Consumes the `DocIndex` contract (contributes `IndexedDoc`s into the merged - snapshot); sets memory descriptor tiers (effective `.auto` resolved in Wave 4). -6. **`ai-parked-sessions`** — Wave 3. Owns `ParkScheduler`/`ParkedSession`/`ParkState` + the durable store + - the notch rail. Provides the scheduler the Wave-2 batched runtime fills (the batched slice re-binds its - scheduler stub to `SerialParkScheduler` here). -7. **`ai-background-autonomy`** — Wave 4. Ships the production `WritePolicyResolving` conformer (replaces - tool-routing's `DescriptorWritePolicy` default) + whitelist + `AuditLog` + `BackgroundGate`; calls - `ParkScheduler.escalate`. -8. **`ai-claude-handoff`** — Wave 4. Registers the `launch_claude` `ToolContributor`; re-points - `HandoffAuditing`/`HandoffEscalating` at the real `AuditLog`/`ParkScheduler.escalate` (conflict 3). -9. **`ai-conversational-canvas`** — Wave 4, last. Wires the front end over all earlier waves: renders the - route-loop state, the multi-turn thread, the compass, the overscroll-park trigger into the parked store. - ---- - -## 5. Open questions still needing a human decision - -These are tuning values or small policy choices the architects flagged that are not derivable on paper and -do **not** block applying the first change. Resolve during run-verify or at the relevant wave. - -- **Tuning constants (run-verify, M5 stable-signed build):** `keepRecentTurns` + compaction margin % - (conversation-runtime); `maxToolSteps` default 8 (tool-routing); Gemma 4 real `maxContextTokens`, - Balanced/Long preset token values, `kvBytesPerTokenPerLayer`, sliding-window size + sliding/global layer - split (batched — ground in the actual Gemma 4 `config.json` at code time); `MemoryCap` defaults (~8 KB / ~60 - facts); `agentMaxParkedSessions`, `agentParkIdleTimeout`, `overscrollThreshold` (must sit above - `canvasResolveThreshold`); the needs-you glow radius/period/accent (parked — make-or-break, feel-only); - audit retention cap (500 vs a time window). -- **Conversation title derivation:** the field is on `AgentConversation` (owned by conversation-runtime), but - whether a smart title comes from a model call or the first user turn is left to canvas/parked — confirm who - computes it. -- **Descriptor identity for configured tasks (tool-routing Q2):** encode bound config in the descriptor name - (`save_to_project:`) vs a side table. Proposed name-encoding; confirm. -- **Skill `claudeHandoff:` front-matter spelling (handoff Q4 / skills):** the two slices must agree on the - exact block spelling (`{ auto: true, maxPerDay: 3, dir: … }` vs full `ClaudeHandoffConfig` field names) so - the parse maps cleanly. Owner: `ai-skills-as-files`; type: `ai-claude-handoff`. -- **`PolicyTarget` threading (autonomy Q1):** confirm `ai-tool-routing` passes the extracted `PolicyTarget` - into the additive `effectiveTier(for:target:)` overload at the call site (no protocol change). Without it the - whitelist can only act on the descriptor-only fast path and never lowers `.confirm`. -- **Background compaction trigger:** whether a parked session's compaction runs in the background (batched) or - only at foreground turn time — the pure `plan()`/`summarize()` split supports both; confirm trigger ownership - with parked/batched. -- **Front-matter parser dependency:** hand-rolled YAML subset vs adding a YAML parser to Core (skills) — both - MLX-free-legal; defaulted to the documented subset, confirm. -- **Whitelist command-pattern grain (autonomy Q3):** `argv[0]`/tool-name anchored glob only (proposed) vs full - shell-command-line patterns (rejected as too sharp). Confirm `argv[0]`-only for v1. -- **`nextRunAt` setter (parked Q):** who sets a future scheduled continuation (a skill? a tool? the user?) is - not defined in any one slice — needs a cross-slice owner. -- **Discarded-session audit retention (parked/autonomy):** does a discarded conversation's append-only audit - trail outlive the removed conversation? (Append-only argues yes.) - ---- - -## 6. MLX / build note (binding) - -**`ai-batched-runtime-and-context` requires the user's stable-signed build to validate.** Its pure pieces -(`BatchedLLMRuntime` protocol, `ConcurrencyBudget`/`KVCacheCost` math, `ModelDescriptor.maxContextTokens`, -the `ContextBudgetProviding` provider, the `Subagent` primitive, settings persistence) verify under -`swift test`. But the **batched decode loop, KV-quant, prefix cache, and rotating-window** live in the -MLX-linked `BatchedGemmaMLXRuntime` (GemmaRuntime target) and are **`xcodebuild` compile-verify ONLY** for an -agent. Real correctness — no cross-stream KV bleed, correct padding/masking, prefill interleave, live -batching across foreground + parked slots, the displayed RAM/stream-count/speed cost surface tracking the -slider — can only be confirmed in the **user's stable-signed build** (TCC + Metal + live multi-stream). The -same applies to the deferred real Gemma chat-template `chat()` override flagged by `ai-conversation-runtime`, -and to every overlay panel (the parked notch rail/glow, the canvas float-up/key-main flip). An agent -**never** builds/signs/installs the `.app` — ad-hoc signing breaks TCC grants. The metallib bundle landmine -(`build-app.sh` copies `*.bundle` into `Contents/Resources/`) is unchanged and must not regress. - ---- - -## 7. Readiness - -**Ready for implementation.** All nine changes validate `--strict`, the shared contracts match across every -consumer, the dependency DAG is acyclic, the gesture verbs are uniform, and every MLX-linked piece is marked -compile-verify-only. The conflicts found (§3) are mechanical bind-time actions (delete the two stand-in seams -+ the placeholder file, build the index merge-shaped, populate `keywords` on projection) folded into the -apply order (§4), plus the three `.openspec.yaml` files created during this pass. `/opsx:apply` can start on -`ai-conversation-runtime` (Wave 1) without rework. diff --git a/docs/ai-agent-v2.5-roadmap-addendum.md b/docs/ai-agent-v2.5-roadmap-addendum.md deleted file mode 100644 index f61517f..0000000 --- a/docs/ai-agent-v2.5-roadmap-addendum.md +++ /dev/null @@ -1,284 +0,0 @@ -# AI Agent V2.5 — Roadmap Addendum (Compute Tiers · Media · Fleet · Full Potential) - -**Status:** integration-welded onto the V2 nine-slice plan. Six new slice architects each authored an -OpenSpec change under `openspec/changes/ai-{compute-tiers,media-runtime,local-image-generation, -video-animation-generation,model-fleet,full-potential-toggle}/`; this document is the integration -architect's synthesis for the V2.5 wave — the index, the implementation order welded *after* the -existing nine waves, the cross-slice conflict pass (the six against each other AND against the nine), -and the readiness call. - -This is the companion to **`docs/ai-agent-v2-roadmap.md`** (the nine-slice roadmap) and rides on the -binding contract sketches in **`docs/ai-agent-v2-addendum-compute-media-fleet.md`** (read that first; -it pins the new shared types §A1/§B1/§C1/§D1) and **`docs/ai-agent-v2-blueprint.md`** (the original -nine §3.x contracts). Do **not** edit the existing roadmap — this file is the V2.5 delta. - -**All six new changes pass `openspec validate --strict` (exit 0, re-run during this pass).** - ---- - -## 1. The six new changes (index) - -| # | Change | Capability | One-line role | -|---|---|---|---| -| 1 | [`ai-compute-tiers`](../openspec/changes/ai-compute-tiers/) | `on-device-ai-runtime` (delta) | The **second lane**: `ComputeLane{.gpu,.cpuTernary}` + the pure `LaneRouting` role→lane policy; a CPU ternary `LLMRuntime` conformer for short structured bursts; the additive lane-affinity hint so a parked subagent advances on CPU concurrently with the foreground GPU reply. Owns §A1. | -| 2 | [`ai-model-fleet`](../openspec/changes/ai-model-fleet/) | `on-device-ai-runtime` + `configuration-hub` (delta) | Evolves `ModelManager` from one-resident-runtime into a **registry with residency/eviction**: extends `ModelDescriptor` (`role`/`lane`/`provider`/`residencyBytes`/`maxContextTokens`), the `ModelRegistry` protocol, the pure 48 GB `ResidencyPlanner` (a video/FP16-image gen evicts chat), and the cloud members (Claude, GLM-5.2) that are never resident. Owns §C1. | -| 3 | [`ai-media-runtime`](../openspec/changes/ai-media-runtime/) | `ai-generative-media` (new) + `ai-command-tasks` (delta) | The **second runtime seam**: `MediaRuntime` parallel to `LLMRuntime` (a long async job → a file), the media value types (§B1), `generate_image`/`generate_video` as `ToolDescriptor`s, the `MediaGenSink` route-loop executor, the seed (img2img/img2video) path, and the dual output (Files-band gallery asset + canvas player). Owns the seam/tools/sink/output, not the backends. Owns §B1. | -| 4 | [`ai-local-image-generation`](../openspec/changes/ai-local-image-generation/) | `ai-generative-media` (delta) | A concrete **local image** `MediaRuntime` (mflux/FLUX-class MLX, in-process), the Q4 (~7 GB co-resident) / FP16 (~24 GB evicts-chat) descriptor variants, the pure `ImageResidencyClass` classifier, seed-driven img2img/inpaint, honest RAM/heat/latency disclosure. The honest *local* default (M5 diffusion is compute-bound). | -| 5 | [`ai-video-animation-generation`](../openspec/changes/ai-video-animation-generation/) | `ai-generative-media` (delta) | The **video** backend(s): cloud-escalation default (`CloudVideoRuntime`, `.dangerous` + rolling-24h budget cap + audit, the Claude-handoff template applied to video), the frontier local LTXV backend behind the master toggle (35 GB+, minutes/clip, evicts chat), img2video from the seed, one seam two interchangeable backends. | -| 6 | [`ai-full-potential-toggle`](../openspec/changes/ai-full-potential-toggle/) | `tunable-settings` + `configuration-hub` (delta) | The **master gate**: `fullPotentialEnabled` (default OFF) + five sub-flags, the pure `FullPotentialGate` (`master ∧ subFlag ∧ aiCommandsEnabled`), one Hub page with cost-in-the-same-breath disclosure, panic-off relock. Owns §D1. Each heavy slice consults the gate before activating. | - ---- - -## 2. Implementation order (welded after the existing nine waves) - -The nine-slice roadmap (`ai-agent-v2-roadmap.md` §4) sequences Waves 1–4 ending at -`ai-conversational-canvas`. The V2.5 wave assumes **the existing Wave 1–2 types are already on disk** -(`AgentMessage`/`AgentConversation`/`AgentSessionID` from `ai-conversation-runtime`; `LLMChatRequest`/ -`chat()` and `BatchedLLMRuntime`/`ParkScheduler` from `ai-batched-runtime-and-context` / -`ai-parked-sessions`; `ToolDescriptor`/`ToolRegistry`/`WritePolicyTier`/route loop from -`ai-tool-routing`; `AuditLog`/`WritePolicyResolving` from `ai-background-autonomy`). Apply V2.5 as a -**fifth wave** (addendum §4), in this strict order: - -> **Wave 5:** `ai-compute-tiers` → `ai-model-fleet` → `ai-media-runtime` → -> {`ai-local-image-generation`, `ai-video-animation-generation`} → `ai-full-potential-toggle` - -1. **`ai-compute-tiers`** — first, because it **OWNS `ComputeLane`** (§A1), and every later V2.5 slice - types against it: `ai-model-fleet`'s `ModelDescriptor.lane` *is* `ComputeLane?`; the image/video - backends declare `lane: .gpu`; the toggle gates `cpuLaneEnabled`. It amends the single-GPU assumption - on `on-device-ai-runtime` (MODIFIED "Targets capable hardware only" + ADDed two-lane requirements) and - adds the additive lane-affinity hint consumed by the already-landed `ParkScheduler`/`BatchedLLMRuntime` - shapes — it edits neither sibling's files. -2. **`ai-model-fleet`** — second, because it **needs the lane concept** (`ModelDescriptor.lane: - ComputeLane?`) and supplies the **registry + residency planner** the media slices ask `ensureResident` - on. It registers the ternary descriptor compute-tiers introduced (`role: .ternaryChat`, - `lane: .cpuTernary`) and the image/video descriptors the backends supply, and the cloud members - (Claude, GLM-5.2). Evolves `ModelManager` through the existing `ModelProvisioner` seam (no API change). -3. **`ai-media-runtime`** — third, because it **OWNS the `MediaRuntime` seam + the `generate_*` tools + - `MediaGenSink` + the dual output** (§B1). It registers the media tools in the already-landed - `ToolRegistry` (via `ToolContributor`, no route-loop change), maps output to the existing Files-band - `.fileEntry`, and parks via the existing `ParkScheduler`. The two backends drop into this seam. -4. **`ai-local-image-generation` + `ai-video-animation-generation`** — fourth, **in parallel**: both are - pure `MediaRuntime` conformers behind the seam (depend on it) plus their fleet descriptors (depend on - the registry). They share no types with each other (image is `capabilities:[.image]`, video is - `[.video]`; distinct descriptor variants, distinct provider keys), so order between them is free. -5. **`ai-full-potential-toggle`** — last, because it **gates everything new (and the existing heavy - slices)**: `cpuLaneEnabled`/`batchedRuntimeEnabled`/`mediaGenEnabled`/`backgroundAutonomyEnabled`/ - `fleetCloudEscalationEnabled` under the `fullPotentialEnabled` master. The pure `FullPotentialGate` - must exist for each heavy slice's check-site; landing it last lets every gated slice's flag and cost - line already be on disk to wire into the one Hub page. (Architects planned it in parallel against the - addendum's pinned §D1 keys, so the plan does not block — only the *apply* is last.) - -**Rationale for the spine** (compute → fleet → media → backends → toggle): it is a topological walk of -the §4 weld DAG. Lane is the lowest primitive (a descriptor field); the fleet is the registry that -hosts every model *including* the lane-tagged ternary and the media backends; the media seam needs the -fleet to resolve which image/video runtime is resident; the backends need the seam; the toggle gates the -union. No back-edges: the toggle only *reads* flags the heavy slices consult, never the reverse. - ---- - -## 3. Cross-slice conflict pass - -Verified the new shared types match across the six AND against the nine, the V2.5 weld DAG is acyclic, -the gesture/error/overlay invariants are uniform, and every native-linked piece is `xcodebuild` -compile-verify-only. Findings below — all are **mechanical bind-time actions**, none re-plans a slice -(mirroring `ai-agent-v2-roadmap.md` §3 style). - -### Verified consistent - -- **`ComputeLane` (§A1)** — `ai-compute-tiers` writes it verbatim (`enum ComputeLane: String, Codable, - Sendable { case gpu, cpuTernary }`); `ai-model-fleet` consumes it as `ModelDescriptor.lane: - ComputeLane?`, the image/video backends declare `.gpu`, the toggle gates `cpuLaneEnabled`. Grep-confirmed - no redefinition; `ai-model-fleet` independently states it imports `ComputeLane` "verbatim from §A1." -- **`AgentWorkRole` / `LaneRouting`** — owned by compute-tiers; the six-case role enum and the protocol are - reproduced exactly. No consumer redefines them. -- **`MediaRuntime` + media value types (§B1)** — owned by `ai-media-runtime` verbatim - (`MediaKind`/`MediaSize`/`MediaParameters`/`MediaRequest`/`MediaProgress`/`MediaAsset`/`MediaRuntime`). - Both backend slices conform to the seam "as written," never redefine it: image declares - `capabilities: [.image]`, video `[.video]`; both consume `MediaRequest.seed` as the first frame. The - eight media requirement headers in `ai-media-runtime` and the backend-only requirement headers in the - two backends are disjoint — no merge collision on `ai-generative-media`. -- **`ai-generative-media` is a NEW capability authored by `ai-media-runtime`; the two backends write - `## ADDED Requirements` against it.** Because the base capability does not yet exist in - `openspec/specs/`, `ADDED` is the correct OpenSpec op for the backends (net-new requirements layered on - a capability a sibling introduces in the same wave). Validate is clean (exit 0) on all three. -- **`generate_image`/`generate_video` ride the EXISTING route loop** — they are `ToolDescriptor`s - contributed via the existing `ToolContributor` seam; `MediaGenSink` is just another sink the - route→execute→continue loop dispatches to. No new control flow. The `ai-command-tasks` delta is one - `## ADDED Requirement` ("Media generation is a routed tool executed by a media sink, not a new control - flow") — disjoint from `ai-tool-routing`'s six existing `ai-command-tasks` requirement headers, so the - capability composes additively. -- **Write-policy + audit reuse** — image `.confirm`, cloud video `.dangerous` + budget cap; both reuse - `WritePolicyTier`/`AuditLog`/`WritePolicyResolving` (owned by `ai-background-autonomy`, §3.7) and the - `ai-claude-handoff` budget/rate-cap template (§3.8). No new gating mechanism; the video `VideoBudget` is - the `HandoffBudget` pattern reused (`now:`-injected rolling-24h ledger). -- **Park + Files-band reuse** — media parks via the existing `ParkScheduler` (the notch glows on - completion / `needsYou`); output lands as the existing Files-band `.fileEntry` (no new browser). Both - are consumed shapes from `ai-parked-sessions` / `files-band`, unedited. -- **Gesture compass uniform** — the media canvas player resolves with the canonical two-finger compass - (DOWN=extract only at canvas top → save/paste/set-as, RIGHT=discard, sub-threshold scroll = no-op), - identical to the nine-slice canvas. No new gesture surface; no recognizer edit; interpretation at the - `AppCoordinator` seam. -- **Error taxonomy** — exactly the two addendum-sanctioned new enums appear: `MediaError` (owned by - `ai-media-runtime`, the backends map vendor errors into it at their boundary) and `FleetError` (owned by - `ai-model-fleet`, only the cases `RuntimeError` cannot carry). `ai-compute-tiers` correctly prefers - extending `RuntimeError` over a `ComputeError`. All route through the single `AIError.message(for:)`, - surfaced bounded + non-blocking, never `NSAlert`, never raw error in a headline. -- **`configuration-hub` deltas compose** — `ai-model-fleet` (fleet roster + cloud badge), - `ai-full-potential-toggle` (the Full Potential section), and the existing-wave `ai-background-autonomy` - (whitelist + audit) each `## ADDED` disjoint requirement headers on the AI page. No collision. -- **Master gate keys (§D1) consumed, not redefined** — `ai-full-potential-toggle` OWNS the six keys; - compute-tiers reads `cpuLaneEnabled`, media + backends read `mediaGenEnabled`, fleet reads - `fleetCloudEscalationEnabled`. Each slice's proposal lists them as "consumed, not owned." -- **Native-linked split marked everywhere** — every concrete backend (`TernaryCPURuntime`, - `MFluxImageRuntime`, `CloudVideoRuntime`/`LocalLTXVRuntime`), real residency/eviction with real weights, - and the canvas media player overlay are `xcodebuild` compile-verify-only; the seams, value types, - policy/residency MATH, and stubs (`StubTernaryRuntime`/`StubMediaRuntime`/stub video runtimes) are - MLX-free Core `swift test`-verified. The `*.bundle` metallib copy must not regress. - -### Conflicts found + resolutions - -1. **`ModelDescriptor.maxContextTokens` is ADDed by TWO slices (cross-wave).** The existing-wave - `ai-batched-runtime-and-context` ADDs `ModelDescriptor.maxContextTokens` (for the - `agentContextTokens` budget); the new-wave `ai-model-fleet` *also* lists `maxContextTokens` among the - fields it adds to `ModelDescriptor` (§C1 carries it on the descriptor). **Resolution:** the field has - ONE definition. `ai-batched-runtime-and-context` (earlier wave) lands `maxContextTokens` first; - `ai-model-fleet` adds `role`/`lane`/`provider`/`residencyBytes` **only**, and treats `maxContextTokens` - as **already present** (it carries the field on the descriptor, it does not re-declare it). Fleet's own - design already scopes this honestly ("this slice only *carries* `maxContextTokens` on the descriptor"; - context-tuning sliders stay batched's). At apply time, fleet's `ModelDescriptor` extension is a - **superset patch** over batched's field — adding the four fleet fields next to the existing - `maxContextTokens`, not a second declaration of it. Mechanical; no shape renegotiation. - -2. **`ModelDescriptor.capabilities` type: `Set` (live) vs `Set` (§C1 sketch) vs the - backends' string tags.** §C1 sketches `capabilities: Set`; the live descriptor is - `capabilities: Set`; `ai-local-image-generation` writes `capabilities: ["image","img2img", - "inpaint"]` (string tags). **Resolution:** keep the **live `Set`** (fleet's design D explicitly - honors §C1's *field shape* while keeping the existing `Set` — §C1's `Set` is the - sketch stand-in for the real capability set, not a mandate to change the type). The backends' string - tags (`"img2img"`/`"inpaint"`) bind to **`Modality` cases** (or are carried as a separate descriptor - tag set the image runtime reads) — the image slice must populate `Modality` values, not raw strings, - when it supplies its descriptors. Bind action: at `ai-local-image-generation` apply, map its - `["image","img2img","inpaint"]` tags onto the descriptor's `Modality`-typed capability set (extend - `Modality` with the img2img/inpaint cases if absent). Low risk; the data exists either way. - -3. **`ModelDescriptor` `Codable`: §C1 sketches it `Codable`; the live descriptor is NOT.** §B1's - `MediaAsset` IS `Codable` (it persists), but §C1's `ModelDescriptor` `Codable` is sketch-only — fleet's - design keeps the descriptor non-`Codable` (it is constructed in code; only the selected model **id** - persists via the existing `ModelSelector`). **Resolution:** no consumer may assume - `ModelDescriptor: Codable`. The image/video backends persist their **selection key** (`imageModelID` / - `videoProvider`), never the descriptor. Confirm at apply that no backend serializes a descriptor. - Already honored in every proposal; just flag it so a later author doesn't add `Codable` to satisfy the - §C1 sketch literally. - -4. **`mediaVideoBudgetPerDay` is referenced by `ai-model-fleet` but OWNED by - `ai-video-animation-generation`.** The fleet's Hub roster displays a cloud member's per-day-$ figure - and design D6 references a `$ / per-day cap`, but the key `mediaVideoBudgetPerDay` is owned by the video - slice (addendum §1). Fleet's "Consumes (verbatim)" line names `fleetCloudEscalationEnabled` / - `fullPotentialEnabled` but **not** `mediaVideoBudgetPerDay`. **Resolution:** the key has one owner — - `ai-video-animation-generation`. The fleet roster's cloud-cost display **reads** `mediaVideoBudgetPerDay` - (consume, do not define). Since the video slice lands AFTER the fleet in Wave 5, the fleet's roster - reads the key through a read-seam that is `nil`/"budget set by Video" until the video slice supplies it - — exactly the same forward-reference pattern the nine-slice roadmap used for the - `ContextBudgetProviding` injection. Tightening fleet's "Consumes" line to list `mediaVideoBudgetPerDay` - would remove the ambiguity (markdown-only; non-blocking — flagged by the per-slice review). - -5. **`imageModelID` / `videoProvider` selection keys cross fleet ↔ backend.** The fleet REGISTERS image - and video descriptors but the **selection keys** are owned by the backends: `imageModelID` - (`ai-local-image-generation`) selects the Q4-vs-FP16 image descriptor; `videoProvider` - (`ai-video-animation-generation`) selects cloud-vs-localLTXV. **Resolution:** no conflict, but the - binding direction must be explicit — the **fleet's registry is the source of descriptors; the backends' - keys select among them.** At apply, the fleet's roster reads `imageModelID`/`videoProvider` as the - selected-member pointers (consume), and the backends supply the descriptors the keys point at. Both - land after the fleet, so the fleet roster tolerates an unset key (defaults to the Q4 image / cloud - video) until the backend slice lands. Mechanical forward-reference, identical to conflict 4. - -6. **Lane ordering against the existing `ParkScheduler`/`BatchedLLMRuntime` shapes.** `ai-compute-tiers`'s - lane-affinity hint must attach to `ParkScheduler`'s runnable session and be read by the batched - runtime — but both seams are owned by EARLIER (already-landed) slices and their signatures - (`runnableSessions(now:maxSlots:)`, `batchStep(...)`) are pinned and unchanged. **Resolution:** the hint - is **additive and carried beside** the pinned signatures (a `LaneAffinity` value the scheduler attaches - to a `ParkedSession`/the dispatch, NOT a new parameter on the pinned methods). Compute-tiers amends this - via MODIFIED/ADDed requirements on `on-device-ai-runtime` and does **not** edit the batched or parked - slice files. Confirm at apply that the hint rides as an attached value, not a signature change — the - compute-tiers spec already says exactly this. - -7. **Three slices add `on-device-ai-runtime` deltas this wave; four total across both waves.** Existing - wave: `ai-conversation-runtime` (MODIFIES "Swappable model runtime abstraction" + ADDs conversation - reqs) and `ai-batched-runtime-and-context` (ADDs batched/context reqs). New wave: `ai-compute-tiers` - (MODIFIES "Targets capable hardware only" + ADDs two-lane reqs) and `ai-model-fleet` (ADDs fleet/registry - reqs). **Resolution:** all four compose because their requirement headers are **pairwise disjoint** (the - one MODIFIED header each touches a *different* base requirement: conversation-runtime → "Swappable model - runtime abstraction", compute-tiers → "Targets capable hardware only"; grep-confirmed only - `ai-compute-tiers` modifies the hardware requirement in the active set). The synced capability spec - composes cleanly. No collision. - -### Against the existing nine — no regressions - -- The two `LLMRuntime` consumers added this wave (the CPU ternary conformer, the media seam) honor the - blueprint §0 reuse rule: the ternary model is **another `LLMRuntime` conformer** (not a new protocol); - the media seam is a **parallel** `MediaRuntime`, explicitly NOT bolted onto `LLMRuntime`. Neither forks - the model seam every nine-slice consumer depends on. -- `ai-model-fleet` evolves `ModelManager` through the **existing `ModelProvisioner`/`runtimeFactory`** - (no API change), preserving the nine-slice assumption that the batched runtime plugs in behind the same - seam. A **fleet-of-one stays byte-for-byte today's behavior** (and the nine-slice batched runtime's - plug-in path). -- The master toggle's `batchedRuntimeEnabled` / `backgroundAutonomyEnabled` flags gate the EXISTING heavy - slices. Per `ai-full-potential-toggle`'s design, those slices' check-site requirements are authored in - **their own** files, not edited by the toggle slice — so the weld onto the nine is "each slice consults - the gate," not a rewrite of the nine. - ---- - -## 4. Open questions needing a human decision - -Tuning values and small policy choices the architects flagged as not derivable on paper. None blocks -applying `ai-compute-tiers` (the first V2.5 slice). Resolve during run-verify on the user's stable-signed -build or at the relevant Wave-5 step. - -- **Ternary model choice (compute-tiers):** which concrete small ternary/BitNet-class model the - `TernaryCPURuntime` carries (the addendum pins "small ternary/BitNet-class," not a specific checkpoint). - Needs a named model + its real `residencyBytes` (~0.5 GB target) and a measured CPU per-token rate to - confirm the short-bursts-only constraint holds on M5. -- **Cloud video provider choice (video):** the addendum names "LTX Studio's hosted API / equivalent" — - the specific hosted provider, its API shape (upload prompt + optional seed, poll progress, fetch file), - and the per-clip $ order that calibrates `mediaVideoBudgetPerDay` are a human decision (real money + - data off-device). The seam is built provider-agnostic; the choice is a config, not a re-plan. -- **`mediaVideoBudgetPerDay` default + the per-day-$ display value (video / fleet):** the actual default - cap and the $-figure the fleet roster shows. Ground in the chosen provider's real per-clip price. -- **`imageModelID` default + quant policy (local-image):** Q4 (~7 GB, co-resident) is the documented - default; confirm the FP16 (~24 GB, evicts-chat) opt-in is exposed and that the exact `residencyBytes` - numbers match the chosen mflux/FLUX checkpoint at code time. -- **Image/video tuning (local-image / video):** default diffusion `steps`, `guidance`, `MediaSize` - presets, and the per-clip `durationMs` ceiling — feel-only, calibrated on the signed build against real - latency/heat. -- **The eviction "FP16 threshold" wording (fleet):** fleet design D3 describes a qualitative "FP16 - threshold" for evict-chat. It is arguably **emergent** (the target simply doesn't fit the 48 GB budget - given the resident set + KV) rather than a named constant — confirm the planner stays pure - `residencyBytes`-vs-`budgetBytes` math with no hard-coded threshold (cosmetic; scenarios are correct - either way). -- **`Modality` extension for img2img/inpaint tags (conflict 2):** confirm `Modality` gains - `.img2img`/`.inpaint` cases (vs carrying those as a separate descriptor tag set) — a small type decision - at the `ai-local-image-generation` apply step. -- **Cost-disclosure copy (full-potential):** the exact RAM/heat/latency/$ strings on each sub-toggle row - are make-or-break for the honest-surface ethos and are feel/wording — finalize on the real Hub page in - the signed build. - ---- - -## 5. Readiness - -**Ready for implementation, after the existing nine.** All six V2.5 changes validate `--strict` (exit 0, -re-run this pass), the new shared types (`ComputeLane` §A1, `MediaRuntime`+value types §B1, -`ModelRegistry`/extended `ModelDescriptor` §C1, the master gate §D1) match across the six AND fold onto -the nine without redefining a nine-slice type or forking the `LLMRuntime`/`ModelProvisioner` seams, the -Wave-5 weld DAG is acyclic, and every native-linked backend + overlay is marked `xcodebuild` -compile-verify-only with the real correctness deferred to the user's stable-signed build. - -The seven conflicts in §3 are all **mechanical bind-time actions** folded into the Wave-5 apply order: -treat `maxContextTokens` as already-present when fleet extends the descriptor (1); map the image backend's -string tags onto `Modality` (2); never assume `ModelDescriptor: Codable` (3); read `mediaVideoBudgetPerDay` -through a forward-reference seam owned by the video slice (4); let the fleet roster tolerate unset -`imageModelID`/`videoProvider` until the backends land (5); carry the lane-affinity hint as an attached -value beside the pinned scheduler/batch signatures (6); and rely on the disjoint requirement headers so the -four `on-device-ai-runtime` deltas compose (7). None re-plans a slice. `/opsx:apply` can start on -`ai-compute-tiers` (Wave 5, first) once the existing nine have landed. diff --git a/docs/notch-geometry-reference.md b/docs/notch-geometry-reference.md deleted file mode 100644 index ab7c304..0000000 --- a/docs/notch-geometry-reference.md +++ /dev/null @@ -1,96 +0,0 @@ -# Notch geometry reference (MacBook lineup) + the "behind-the-notch" reveal - -Reference data + the derivation methodology behind `NotchHomeZoneAnchor.notchRect`, captured while -reworking the notch-dock reveal trigger (reveal **only when the cursor crosses *behind* the notch**, -not merely near it). Read this before touching notch geometry or the reveal trigger. - -## The one rule: derive at runtime, NEVER hardcode - -The notch box (position, width, height) is **read from `NSScreen` at runtime** — never hardcoded per -model. The reason is subtle and load-bearing: - -- The **hardware** fixes only the notch's size in **native pixels**. -- macOS reports geometry in **points**, and the point↔pixel ratio is the user's **display-scaling** - choice (System Settings → Displays → "Larger Text … More Space"). Pick "More Space" and every point - value below shrinks proportionally — but `NSScreen` still reports the *current* point-space values - correctly. -- So a per-model point table is a **sanity check only**. The authoritative source is always the live - `NSScreen` read. `NotchHomeZoneAnchor.notchRect(screenFrame:safeAreaTop:auxLeft:auxRight:)` does this. - -### How it's derived (verified correct on real hardware) - -``` -safeAreaTop = NSScreen.safeAreaInsets.top // menu-bar / notch height, 0 ⇒ no notch -auxLeft = NSScreen.auxiliaryTopLeftArea // usable menu-bar strip LEFT of the notch -auxRight = NSScreen.auxiliaryTopRightArea // usable menu-bar strip RIGHT of the notch - -notch (Cocoa, bottom-left origin) = - x: auxLeft.maxX // left edge = right edge of the left strip - width: auxRight.minX - auxLeft.maxX // gap BETWEEN the two strips - y: screenFrame.maxY - safeAreaTop // bottom edge (top edge = physical top) - height: safeAreaTop -``` - -Notch present iff `safeAreaTop > 0 && auxLeft && auxRight && auxRight.minX > auxLeft.maxX`. Absence -(notchless built-in **or** any external display) cleanly selects the top-center **tab** path. - -## Ground truth — measured on this machine - -**MacBook Pro 14" (`Mac17,9`, Apple M5 Pro), default "looks like 1512 × 982" scaling:** - -| Property | Points | Pixels @2× | -|---|---|---| -| `screen.frame` | 1512 × 982 | 3024 × 1964 | -| `visibleFrame` | 1512 × 949 (maxY 949) | — | -| `safeAreaInsets.top` | **32** | 64 | -| `auxiliaryTopLeftArea` | (0, 950, 663, 32) | — | -| `auxiliaryTopRightArea` | (848, 950, 664, 32) | — | -| **derived notchRect** | x 663→848, y 950→982 | — | -| notch **width** | **185** | 370 | -| notch **height** | **32** | 64 | -| centered? | midX 755.5 vs screen 756 → **dead-center (±0.5)** | — | - -Reproduce anytime with `swift` (no app build needed): instantiate `NSApplication` as `.accessory`, -then read `NSScreen.main` `frame` / `visibleFrame` / `safeAreaInsets` / `auxiliaryTop{Left,Right}Area`. - -## Cross-model reference table (sanity check — values are scaling-dependent) - -All notched Macs report **`safeAreaInsets.top = 32 pt` at their _default_ scaling** (the notch height == -the menu-bar height). Native pixels are hardware-fixed; point widths below are at the **default** scale. - -| Model | Native px | Default "looks like" (pt) | Scale | notch/menu H | notch W (pt) | -|---|---|---|---|---|---| -| **14" MBP** (2021 M1 Pro/Max → M2 → M3 → **M4** → M5) | 3024 × 1964 | 1512 × 982 | 2.00× | 32 pt | **185** (measured) | -| **16" MBP** (2021 → M2 → M3 → **M4** → M5) | 3456 × 2234 | 1728 × 1117 | 2.00× | 32 pt | ≈185–200 (est.†) | -| **13" MBA** (M2 2022 / M3 2024 / **M4 2025**) | 2560 × 1664 | 1470 × 956 | ≈1.74× | 32 pt | est.† | -| **15" MBA** (M2 2023 / M3 2024 / **M4 2025**) | 2880 × 1864 | 1470 × 956 | ≈1.96× | 32 pt | est.† | - -† Estimates — **do not hardcode**. The camera housing is physically ~the same module across the -notched line, so in native pixels the notch width is similar everywhere (~370 px); the point value -follows the model's scale factor. The Airs use a **non-integer default scale**, so their point -geometry is *not* a clean 2× of native — one more reason to only ever trust the runtime read. - -Non-notched (always the **tab** path): notchless built-in displays, all external monitors, iMac, -Mac mini / Studio (no built-in display). - -**M4 family specifically:** 14"/16" MBP M4 & M4 Pro/Max (late 2024) reuse the 2021 chassis/panel → -identical notch geometry to the M1–M3 14"/16". 13"/15" MBA M4 (early 2025) keep the M2/M3 Air panels. -No M4 introduces new notch dimensions; the runtime derivation covers them all unchanged. - -## "Behind the notch" is real, usable cursor space - -macOS lets the pointer travel **up into the notch band** — within the notch's x-span the cursor `y` -reaches all the way to `screenFrame.maxY` (the cursor visibly disappears behind the black cutout). -That region is `notchRect` in Cocoa coords, and it's genuinely reachable — which is what makes it a -valid, deliberate hit target rather than a dead zone. - -**Consequence for the reveal trigger:** the dock should reveal **only when the cursor enters -`notchRect`** (crosses *behind* the notch), not when it grazes the strip *below* the notch. The notch -sits on the **physical top edge** → an infinite-depth Fitts's-law target you can slam into and stop -dead, and nothing else lives behind the notch (menu items flank it), so a notch-only trigger can -never collide with reaching for a centered menu item. On **notchless / external** displays we mimic -the same feel: trigger on pushing the cursor to the **very top screen edge** at top-center (a thin -band `visibleFrame.maxY → screenFrame.maxY`, tab-width, centered). - -Keep-open is unchanged — the contiguous live zone already unions nub ∪ notch ∪ panel, so once shown, -moving *down* onto the rail keeps it open; only the hidden→shown **trigger** moves up into the notch. diff --git a/openspec/changes/add-gesture-previews-and-bindings/specs/gesture-bindings/spec.md b/openspec/changes/add-gesture-previews-and-bindings/specs/gesture-bindings/spec.md index b1bc1bc..cc3a59a 100644 --- a/openspec/changes/add-gesture-previews-and-bindings/specs/gesture-bindings/spec.md +++ b/openspec/changes/add-gesture-previews-and-bindings/specs/gesture-bindings/spec.md @@ -1,57 +1,25 @@ ## ADDED Requirements ### Requirement: User-configurable resolution-gesture bindings -The app SHALL let the user choose **which excursion performs which action** for the **resolution** gestures of remappable open surfaces — the AI command canvas, the Files-band drill, and the window switcher's scrub axes. Each surface SHALL have its own **action set** and its own **excursion vocabulary** (the surfaces are deliberately distinct grammars and SHALL NOT be unified into one remap). Bindings SHALL be **persisted** and SHALL **default to exactly today's behavior**. The bindings SHALL be consumed at the existing raw-direction seam (the recognizer's emitted direction is unchanged); only the action a direction maps to is configurable. +The app SHALL let the user choose **which excursion performs which action** for the **resolution** gestures of remappable open surfaces — today, the window switcher's scrub axes. Each surface SHALL have its own **action set** and its own **excursion vocabulary** (the surfaces are deliberately distinct grammars and SHALL NOT be unified into one remap). *(The former AI-canvas and Files-drill binding surfaces were removed with those features — `remove-local-ai`.)* Bindings SHALL be **persisted** and SHALL **default to exactly today's behavior**. The bindings SHALL be consumed at the existing raw-direction seam (the recognizer's emitted direction is unchanged); only the action a direction maps to is configurable. -- **AI canvas:** actions `{commit, dismiss, ignore}` ← excursions `{swipe up, swipe down, swipe left, swipe right}` (two-finger). Default: down = commit, horizontal = dismiss, up = ignore. -- **Files drill:** actions `{open, Open-With, discard}` ← excursions `{lift, +1-finger lift, four-finger horizontal}`. Default: lift = open, +1-finger = Open-With, four-finger horizontal = discard. - **Switcher:** per-axis scrub `{windows axis, Spaces axis}` ∈ `{normal, reversed}`. Default: both normal. #### Scenario: Default bindings reproduce today's grammar - **WHEN** the user has never changed a binding -- **THEN** every surface resolves exactly as it does today (canvas down = commit, etc.) +- **THEN** every surface resolves exactly as it does today (both switcher axes normal) -#### Scenario: A remapped excursion performs the bound action -- **WHEN** the user binds the canvas commit to swipe-right and then swipes right on a ready canvas result -- **THEN** the result is committed (and no longer discarded), and the old default no longer commits - -#### Scenario: Surfaces keep separate vocabularies -- **WHEN** the user configures the Files-drill bindings -- **THEN** only Files-drill excursions/actions are offered there, independent of the canvas bindings - -### Requirement: Bindings are mutually exclusive per surface -Within a single surface, two actions SHALL NOT share one excursion. Assigning an excursion already held by another action SHALL resolve the conflict deterministically (swap the two actions' excursions, or present the taken excursion as unavailable) so the binding set is always **a one-to-one mapping**. The conflict resolution SHALL be a pure, unit-testable verdict. - -#### Scenario: Assigning a taken excursion resolves the conflict -- **WHEN** swipe-down is bound to commit and the user assigns swipe-down to dismiss -- **THEN** the binding set is renormalized so no excursion maps to two actions (e.g. commit takes dismiss's former excursion) - -#### Scenario: The mapping stays one-to-one -- **WHEN** any binding assignment is made -- **THEN** the resulting per-surface mapping has at most one action per excursion +#### Scenario: A remapped axis performs the bound direction +- **WHEN** the user sets the switcher's windows axis to reversed and scrubs right +- **THEN** the highlight steps in the reversed direction (and the normal mapping no longer applies) ### Requirement: Reserved and invalid excursions are never bindable -The binding vocabularies SHALL exclude excursions that must keep a fixed meaning: **single-finger** motion (never a trigger anywhere), and on the AI canvas the **sub-threshold two-finger pan** that scrolls/reads the canvas (below the resolve excursion threshold) SHALL remain "read the canvas" and SHALL NOT be offered as a bindable excursion. **Activation** gestures (which finger-count opens which platform) SHALL NOT be remappable — bindings cover resolution within an already-open surface only. +The binding vocabularies SHALL exclude excursions that must keep a fixed meaning: **single-finger** motion is never a trigger anywhere. **Activation** gestures (which finger-count opens which platform) SHALL NOT be remappable — bindings cover resolution within an already-open surface only. #### Scenario: Single-finger is not offered as a binding - **WHEN** the user opens a binding editor - **THEN** no single-finger excursion is available to bind -#### Scenario: Reading the canvas stays unbindable -- **WHEN** the user opens the canvas binding editor -- **THEN** the sub-threshold two-finger scroll is not offered as a bindable excursion and keeps scrolling the canvas - #### Scenario: Activation finger-counts are not remappable - **WHEN** the user looks for a way to change which finger-count opens the launcher vs. the switcher - **THEN** no such binding is offered (the open/dismiss-vs-act-within finger grammar is fixed) - -### Requirement: Load-bearing resolution guards are binding-independent -Safety guards on a resolution action SHALL apply to **whichever** excursion is bound to that action. The AI canvas commit SHALL still require the canvas to be **scrolled to the top** (a commit-bound excursion mid-scroll is the user scrolling, not committing). The Files-drill discard SHALL **never terminate an already-running application**, regardless of which excursion is bound to it. - -#### Scenario: Commit guard survives a remap -- **WHEN** the canvas commit is rebound to a different excursion and performed while the canvas is scrolled away from the top -- **THEN** nothing is committed (the at-top guard still holds) - -#### Scenario: Discard never kills a running app after a remap -- **WHEN** the Files discard is rebound and performed against an entry whose target app is already running -- **THEN** the pending open is defused but the running application is not terminated diff --git a/openspec/changes/ai-action-context-resolution/design.md b/openspec/changes/ai-action-context-resolution/design.md deleted file mode 100644 index c600f41..0000000 --- a/openspec/changes/ai-action-context-resolution/design.md +++ /dev/null @@ -1,72 +0,0 @@ -# Design — context-driven AI action resolution - -## The one-line model - -`AICommand` stops storing *what to do* and starts storing *what it's allowed to do*. Two capability sets (`inputs`, `outputs`), default all-on. At fire time the executor **senses the live environment**, resolves the active input channel and the commit behavior from it, and picks the model capability from the chosen input. Toggles are guardrails on an automatic decision, not the decision. - -``` -ENVIRONMENT (sensed at fire) COMMAND (capability toggles, default all-on) - • selection present? inputs : {selection, clipboard, clipboardImage} - • clipboard text? outputs : {replaceSelection, pasteAtCursor, previewOnly} - • clipboard image? (or a side-effect: {runTask(k)} / {sendTo(d)}) - │ - ▼ - RESOLVE INPUT ── first ENABLED channel that is live, by priority: - selection ▸ clipboardText ▸ clipboardImage ▸ (none) - │ - ▼ - PICK MODEL ── resolved channel is an image ⇒ vision, else text - │ - ▼ - RUN + STREAM into the canvas (unchanged) - │ - ▼ - RESOLVE OUTPUT ── in-place: wasSelection && replaceSelection enabled → REPLACE - else pasteAtCursor enabled → PASTE - else previewOnly → write nothing - side-effect: accept-step → execute (unchanged) - │ - ▼ - two-finger DOWN commits the resolved behavior · horizontal ALWAYS discards -``` - -## Decisions - -### D1 — Input priority: `selection ▸ clipboardText ▸ clipboardImage` -Text beats image when both exist: text is the overwhelming common case and cheaper, and it keeps a plain "Fix Grammar" a text run. The image channel engages only when there is **no usable text anywhere** — which is exactly the "the last thing I copied is an image" case. Selection always wins over the clipboard (it is the most intentional signal). A command narrows this by disabling channels: a "Describe image" command turns the text channels **off** so only the image channel can win, regardless of a stray selection. - -### D2 — Output is resolved from the input channel, not stored -For an in-place command the commit is derived, not fixed: -``` -if resolvedWasSelection && outputs.contains(.replaceSelection) → replace selection -elif outputs.contains(.pasteAtCursor) → paste at cursor -elif outputs.contains(.replaceSelection) → replace (SelectionService pastes if no live selection) -elif outputs.contains(.previewOnly) → write nothing -else → write nothing (safe default) -``` -This is what makes "swipe down replaces my selection **or** pastes at my cursor" a single behavior. `previewOnly` present without replace/paste = a read-only understanding command. Disabling `replaceSelection` = "never touch my selection, always paste." - -### D3 — Model capability is resolved at runtime (reverses the old static rule) -The prior spec mandated `requiredCapabilities` be derived **statically** from the authored source and never depend on runtime clipboard contents. That is incompatible with "parse the clipboard image when one is present, else run text." So: the executor senses the input first, then requests `[.vision]` only when the **resolved** channel is an image, else `[.text]`. The command's `requiredCapabilities` becomes a **union hint** (does any enabled input *potentially* need vision?) used for informational/editor purposes — it no longer gates the model request. If the only live/enabled channel is an image and no vision model is available, the executor surfaces a clean `.failed` (not a silent text run). - -### D4 — Screen region stays exclusive and region-first -`screenRegion` cannot be passively sensed — it needs the interactive picker to run **before** the canvas opens (existing `selection-io` / launcher flow). So a command whose `inputs` contain `screenRegion` is treated as a region command exactly as today (picker → capture → canvas → fire with the pre-supplied image); it is **not** blended into the ambient cascade. The editor presents `screenRegion` as mutually exclusive with the ambient channels. This keeps the region flow a pure rename (`command.input == .screenRegion` → `command.inputs.contains(.screenRegion)`), no behavior change. - -### D5 — Persistence & skills migrate in place (no schema bump, no data loss) -`AICommand` gains a **custom `Codable`**: `init(from:)` decodes `inputs`/`outputs` when present, else the legacy `input`/`output` scalars, mapping each to a set; `encode(to:)` writes the sets. Legacy → set mapping (behavior-preserving): - -| legacy `input` | → `inputs` | legacy `output` | → `outputs` | -|--------------------|----------------------------------|------------------------|------------------------------------------| -| `selection` | `{selection, clipboard}` | `replaceSelection` | `{replaceSelection, pasteAtCursor}` | -| `clipboard` | `{clipboard}` | `pasteAtCursor` | `{pasteAtCursor}` | -| `clipboardImage` | `{clipboardImage}` | `previewOnly` | `{previewOnly}` | -| `screenRegion` | `{screenRegion}` | `runTask(k)` | `{runTask(k)}` | -| `none` | `{}` (standalone) | `sendTo(d)` | `{sendTo(d)}` | - -`selection → {selection, clipboard}` deliberately makes the old implicit selection→clipboard fallback **explicit** (and now honest: it pastes, not replaces). A **new blank** command defaults to `inputs = {selection, clipboard, clipboardImage}`, `outputs = {replaceSelection, pasteAtCursor, previewOnly}` (all-on). Catalog presets keep authoring with the legacy single `input:`/`output:` via a **convenience init** that runs the same mapping, so `AICommandCatalog`/`AIBand` change only their helper, not every entry. `SkillFile` serializes `inputs:`/`outputs:` (comma-joined) and still parses a legacy `input:`/`output:` line. - -### D6 — Set element vocabulary reuses existing enums -`inputs: Set` reuses `InputSource` (its cases are already right); `.none` is represented by the **empty set**, never a member. `outputs: Set` reuses `OutputTarget`; a side-effecting `runTask`/`sendTo` is a single-element set, and the resolver prefers it when present. `InputSource`, `OutputTarget`, `TaskKind`, and `Destination` gain `Hashable` (all associated values are `String`/`Destination`, so it auto-synthesizes). No new top-level enum types — this keeps the skill serializer, the editor, and `requiredCapabilities` on one vocabulary and holds churn down. - -## What does NOT change -The two-finger resolve grammar (DOWN commits at top / horizontal discards / UP scrolls), the one-shot "a fired command never becomes a conversation" contract, the region picker, the side-effecting accept-step, the model download/opt-in gate, and every non-AI surface. Only *which input is read* and *what DOWN does* become environment-resolved. diff --git a/openspec/changes/ai-action-context-resolution/proposal.md b/openspec/changes/ai-action-context-resolution/proposal.md deleted file mode 100644 index c079e75..0000000 --- a/openspec/changes/ai-action-context-resolution/proposal.md +++ /dev/null @@ -1,33 +0,0 @@ -## Why - -An AI command today pins **one** input source and **one** output target at authoring time (two single-select dropdowns), and the executor obeys them literally. That produces two coupled failures: - -- **The source is wrong in practice.** A command set to `selection` reads the selection via AX, and when AX can't expose it (the common case) it silently falls back to the user's **old clipboard** with no signal — so "run on my selected text" quietly runs on whatever was last copied. -- **The destination is dumb.** `replaceSelection` always replaces and `pasteAtCursor` always pastes, regardless of whether anything is actually selected — so the same verb can't "replace what I highlighted" *and* "paste where my cursor is" depending on context. - -The user's mental model is that input and output are **one decision, driven by the live environment**: if there's a selection, work on it and **replace** it on commit; if there's nothing selected, take the **clipboard** (text *or* image) and **paste** the result at the cursor; a horizontal swipe always discards with no effect. And rather than hard-coding that as the only behavior, each command should expose its input and output options as a **capability toggle list** (default: all on) so a command can be narrowed (e.g. "only ever read the clipboard", "never touch my selection", "preview only"). - -## What Changes - -- **Input becomes a capability set, resolved dynamically.** A command carries `inputs: Set` (the ambient channels it may consume). At fire, the executor **senses the environment** and activates the highest-priority **enabled** channel that is live: `selection ▸ clipboard-text ▸ clipboard-image`. Whichever wins is both the input *and* the remembered channel that drives commit. An empty set means "no input needed" (standalone prompt). -- **Output becomes a capability set, resolved dynamically.** A command carries `outputs: Set`. For in-place commands the commit behavior is **derived from the resolved input**: input was a selection **and** `replaceSelection` is enabled → replace; else `pasteAtCursor` enabled → paste at cursor; else `previewOnly` → write nothing. A side-effecting `runTask`/`sendTo` in the set keeps its own accept-step→execute flow. Nothing is a fixed one-shot toggle; it is resolved every fire from *action nature × environment*. -- **Model capability is resolved at runtime, from the chosen input.** The static "an image source always requires vision" rule is replaced: the executor senses the input first, and requests a **vision** model only when the **resolved** channel is an image (the clipboard actually holds an image), else a text model. This is what lets one command transparently OCR/parse a clipboard image when that's what's present, and stay a cheap text run otherwise. -- **The editor's two dropdowns become two toggle lists.** Inputs and (in-place) outputs are multi-select, defaulting to all-on for a new command; the side-effecting task/destination sub-editor is unchanged. -- **The silent-clipboard bug is designed out.** A failed selection read now *cascades* to the clipboard **and** the commit becomes paste (not a surprise replace of stale clipboard) — the fallback is intentional and honest, never a false "replace" on content the user didn't select. -- **Screen region stays a deliberate, exclusive input.** A command whose inputs contain `screenRegion` is region-first (the pre-canvas picker runs, unchanged); it is not mixed into the ambient cascade. -- **Persistence + skills migrate.** A one-time, in-place decode maps every legacy `input`/`output` scalar to the new sets (e.g. `selection` → `{selection, clipboard}`, `replaceSelection` → `{replaceSelection, pasteAtCursor}`), preserving behavior; the skill-file format gains `inputs:`/`outputs:` and still parses legacy `input:`/`output:`. - -## Capabilities - -### Modified Capabilities - -- `ai-command-band`: the value model's input/output become **capability sets**; **input acquisition** and **in-place output routing** become **environment-resolved** (selection→replace, clipboard→paste, clipboard-image→vision), replacing the static single-source acquisition and the static replace/paste routing; **required model capability** is resolved from the chosen input at runtime rather than derived statically from the authored source. -- `selection-io`: the read path gains a **capability-cascade** framing (probe selection, else clipboard text, else clipboard image) and the write path is chosen by the **resolved input channel**, not a stored output enum. - -## Impact - -- **Code (all MLX-free Core):** `AI/AICommand.swift` (sets + `Hashable` on `InputSource`/`OutputTarget`/`TaskKind`/`Destination`, custom `Codable` migration, legacy convenience init, pure input/output **resolution** helpers, union `requiredCapabilities`); `AI/AICommandExecutor.swift` (sense→cascade input acquisition, runtime capability request, remembered resolved channel, resolved commit); `AI/AICommandCatalog.swift` + `AI/AIBand.swift` (preset helper maps legacy args → sets — call sites barely change); `AI/Skills/SkillFile.swift` (serialize/parse `inputs:`/`outputs:` with legacy fallback); `Hub/BandsCanvas.swift` (two dropdowns → two toggle lists); `Overlay/LauncherOverlayController.swift` + `App/AppCoordinator.swift` (`screenRegion` routing keys off `inputs`). -- **Tests:** `AICommandTests` (set model, migration decode, resolution helpers, union capabilities), `AICommandExecutorTests` (selection→replace, empty-selection→clipboard→paste, clipboard-image→vision, no-input), `AICommandCatalogTests`/`SkillsTests` (set-shaped presets + skill round-trip). -- **Spec-mandated change:** `ai-command-band` currently requires `requiredCapabilities` be **static** and never depend on runtime clipboard contents. This proposal **deliberately reverses that** for the resolved-capability behavior; the requirement is rewritten accordingly. -- **No signing, no permission, no TCC interaction, no gesture relocation.** The resolve/commit grammar (two-finger DOWN commits at top, horizontal discards) is unchanged; only *what DOWN does* becomes contextual. Verified under `swift build` / `swift test`; the live AX/clipboard/vision behavior needs the user's signed build. -- **Out of scope:** mixing `screenRegion` into the ambient cascade; new input sources; changing the side-effecting task/`sendTo` dispatch or its accept-step; any change to the canvas gesture grammar or the parked-session/notch surfaces. diff --git a/openspec/changes/ai-action-context-resolution/specs/ai-command-band/spec.md b/openspec/changes/ai-action-context-resolution/specs/ai-command-band/spec.md deleted file mode 100644 index 9a884d8..0000000 --- a/openspec/changes/ai-action-context-resolution/specs/ai-command-band/spec.md +++ /dev/null @@ -1,66 +0,0 @@ -## MODIFIED Requirements - -### Requirement: AI command value model and persistence -The system SHALL define an AI command as a Codable value type carrying: a stable identifier, a display name, an icon and tint, an **input capability set** (`inputs`: a set drawn from `selection` | `clipboard` | `clipboardImage` | `screenRegion`; the empty set means "no input needed"), a **prompt template** string, an **output capability set** (`outputs`: a set drawn from `replaceSelection` | `pasteAtCursor` | `previewOnly` | `runTask(TaskKind)` | `sendTo(Destination)`), a **model selector** (v1: on-device Gemma 4), and a **confirmBeforeRun** flag. A freshly created command SHALL default both sets **all-on** for its family: `inputs = {selection, clipboard, clipboardImage}` and `outputs = {replaceSelection, pasteAtCursor, previewOnly}`. A command's **potential** required capabilities (`requiredCapabilities`) SHALL be the **union** over its enabled inputs — `vision` when any enabled input is an image source (`clipboardImage`, `screenRegion`), and `text` otherwise — and this union is an informational hint only; it SHALL NOT be the value used to request the model (see "Command input acquisition"). Persistence SHALL migrate legacy single-valued `input`/`output` fields into these sets on decode, behavior-preservingly (`selection`→`{selection, clipboard}`, `replaceSelection`→`{replaceSelection, pasteAtCursor}`, other scalars→their singleton set, `none`→`{}`), preserving the identifier. An AI command SHALL remain a first-class, persisted **band item**: stored **inside the Favorites record** as the item of a context band, persisting across launches, applying immediately when changed, and movable between bands like any other item. Its stable identifier SHALL be preserved across edits and migration. - -#### Scenario: Commands persist across launches -- **WHEN** the user creates AI commands and relaunches the app -- **THEN** the same commands, in the same order within their bands, are present - -#### Scenario: Commands are stored as Favorites band items -- **WHEN** AI commands exist and the Favorites record is inspected -- **THEN** the commands are present as items of context bands (they are no longer kept in a separate store) - -#### Scenario: Legacy scalar input/output migrate to capability sets -- **WHEN** a command persisted with the old single `input: selection` / `output: replaceSelection` fields is decoded -- **THEN** it decodes with `inputs = {selection, clipboard}` and `outputs = {replaceSelection, pasteAtCursor}`, preserving its identifier and prior effective behavior - -#### Scenario: A new command defaults every capability on -- **WHEN** a blank AI command is created -- **THEN** its `inputs` are `{selection, clipboard, clipboardImage}` and its in-place `outputs` are `{replaceSelection, pasteAtCursor, previewOnly}` - -#### Scenario: confirmBeforeRun defaults on for side-effecting output but is honored -- **WHEN** a command whose outputs contain a side-effecting `runTask`/`sendTo` is created without an explicit choice -- **THEN** its `confirmBeforeRun` defaults to true; if the user later sets it false, that stored value is honored at run time (not overridden) - -#### Scenario: requiredCapabilities is the union hint, not the model request -- **WHEN** a command with `inputs = {selection, clipboardImage}` is inspected -- **THEN** its `requiredCapabilities` union includes both `text` and `vision`, but this value does not by itself force a vision model — the model is requested from the input resolved at fire time - -### Requirement: Command input acquisition -When an AI command is fired, the system SHALL acquire its input by **resolving the live environment against the command's enabled input capabilities**, not from a single stored source. The executor SHALL activate the highest-priority **enabled** channel that is live, in the order `selection ▸ clipboard-text ▸ clipboard-image`: a non-empty selection wins; else non-empty clipboard text; else a decodable clipboard image (supplied as the request's PNG image input). The **resolved** channel SHALL be remembered to drive both the model capability and the commit behavior. The model requested SHALL be a **vision** model only when the resolved channel is an image, and a **text** model otherwise — so capability follows the actual input, not the authored superset. A command whose enabled inputs are **empty** requires no input (a standalone prompt). A command whose enabled inputs contain `screenRegion` SHALL instead be acquired **region-first** via the interactive picker before the canvas (unchanged), and SHALL NOT blend region capture into the ambient cascade. If a command requires input but **no** enabled channel is live — no selection, no clipboard text, no clipboard image — the system SHALL surface a clear "no input" state and SHALL NOT invoke the model. - -#### Scenario: Selection wins when text is highlighted -- **WHEN** a command with `selection` enabled is fired with text highlighted in the front app -- **THEN** the highlighted text is used as `{input}` and the resolved channel is the selection - -#### Scenario: Empty selection cascades to the clipboard text -- **WHEN** a command with `selection` and `clipboard` enabled is fired with no current selection but text on the clipboard -- **THEN** the clipboard text is used as `{input}` and the resolved channel is the clipboard (so commit will paste, not replace) - -#### Scenario: No text anywhere cascades to a clipboard image -- **WHEN** a command with `clipboardImage` enabled is fired with no selection and no clipboard text, but an image on the clipboard -- **THEN** the clipboard image is supplied as the request's image input, a vision model is requested, and the vision result streams into the canvas - -#### Scenario: A disabled channel is skipped -- **WHEN** a command with `selection` disabled (only `clipboard` enabled) is fired while text is selected -- **THEN** the selection is ignored and the clipboard text is used - -#### Scenario: No live channel surfaces no input -- **WHEN** an input-requiring command is fired and none of its enabled channels is live -- **THEN** the preview shows a clear "no input" state and the model is not invoked - -### Requirement: In-place output routing -For a command whose outputs are in-place (no `runTask`/`sendTo`), after the model result is committed the system SHALL route it by the **resolved input channel**, honoring the enabled output capabilities: when the resolved input was a **selection** and `replaceSelection` is enabled, it SHALL replace the front app's selected text (via selection replace when settable, else paste); otherwise when `pasteAtCursor` is enabled it SHALL paste the result at the insertion point; otherwise when only `previewOnly` is enabled it SHALL write nothing. Output SHALL be delivered into the app that was frontmost when the launcher opened. A command whose outputs contain a `runTask`/`sendTo` SHALL route through the side-effecting accept-step/execute flow unchanged. - -#### Scenario: Selection input commits as a replace -- **WHEN** an in-place command resolved its input from the selection and `replaceSelection` is enabled, and its result is committed -- **THEN** the front app's selection is replaced by the result, in the app that was frontmost at open - -#### Scenario: Clipboard input commits as a paste -- **WHEN** an in-place command resolved its input from the clipboard (no selection) and `pasteAtCursor` is enabled, and its result is committed -- **THEN** the result is pasted at the insertion point (the selection is not touched) - -#### Scenario: Preview-only never writes -- **WHEN** a command whose only enabled output is `previewOnly` has its result committed -- **THEN** the result is shown but nothing is written into the front app diff --git a/openspec/changes/ai-action-context-resolution/specs/selection-io/spec.md b/openspec/changes/ai-action-context-resolution/specs/selection-io/spec.md deleted file mode 100644 index abdc3cb..0000000 --- a/openspec/changes/ai-action-context-resolution/specs/selection-io/spec.md +++ /dev/null @@ -1,16 +0,0 @@ -## MODIFIED Requirements - -### Requirement: Write output back into the front app -The system SHALL deliver a command's committed result into the app that was frontmost when the launcher opened, choosing the write path from the command's **resolved input channel** rather than a single stored output enum: when the resolved input was a **selection** (and the command enables replacing it), by setting the selection's text via Accessibility when the element is settable, otherwise by pasting; when the resolved input was the **clipboard** (or no selection was used), by pasting the result at the insertion point. The paste path SHALL reuse the existing paste-on-fire mechanism (restore representations + synthesized ⌘V into the captured app). - -#### Scenario: Replace via Accessibility when the input was a settable selection -- **WHEN** the resolved input was the selection, the focused element exposes a settable selected-text attribute, and the result is committed to a replace -- **THEN** the selection is replaced via Accessibility without a paste - -#### Scenario: Fall back to paste when not settable -- **WHEN** the resolved input was the selection but the element is not settable and the result is committed to a replace -- **THEN** the result is delivered by pasting into the captured front app - -#### Scenario: Clipboard input writes by pasting at the cursor -- **WHEN** the resolved input was the clipboard (no selection used) and the result is committed -- **THEN** the result is pasted at the insertion point and the selection is not modified diff --git a/openspec/changes/ai-action-context-resolution/tasks.md b/openspec/changes/ai-action-context-resolution/tasks.md deleted file mode 100644 index a0f90c7..0000000 --- a/openspec/changes/ai-action-context-resolution/tasks.md +++ /dev/null @@ -1,50 +0,0 @@ -# Tasks — context-driven AI action resolution - -## 1. Value model (`AI/AICommand.swift`) -- [x] 1.1 Add `Hashable` to `OutputTarget`, `TaskKind`, `Destination` (`InputSource` is already Hashable via its raw value) -- [x] 1.2 Replace `input: InputSource` → `inputs: Set` and `output: OutputTarget` → `outputs: Set` -- [x] 1.3 Designated init takes the sets; add a **legacy convenience init** `(input:output:)` that maps → sets (behavior-preserving table, design D5) -- [x] 1.4 Custom `Codable`: `init(from:)` decodes `inputs`/`outputs` if present, else legacy `input`/`output` → sets; `encode(to:)` writes the sets; preserve all other fields + `decodeIfPresent` optionals -- [x] 1.5 `defaultConfirmBeforeRun(for: Set)` + `isSideEffecting`/`sideEffect` key off "outputs contains a `runTask`/`sendTo`" -- [x] 1.6 `requiredCapabilities` = union over enabled inputs (vision if any image input, text otherwise; text if empty) — informational hint -- [x] 1.7 Pure resolution helper `inPlaceCommitPlan(resolvedWasSelection:outputs:) -> CommitPlan` used by the executor + tested directly -- [x] 1.8 `sideEffect: OutputTarget?` accessor + `needsInput` derived from the sets - -## 2. Executor (`AI/AICommandExecutor.swift`) -- [x] 2.1 `run()` → **resolve**: `resolveAmbientInput` probes `selection` ▸ `clipboard` text ▸ `clipboardImage`; returns the text/image AND records the channel -- [x] 2.2 Request the model from the **resolved** channel's modality (image→vision else text), not the static union -- [x] 2.3 Empty inputs = standalone (no input required); non-empty but nothing live = `.noInput` -- [x] 2.4 `screenRegion` in inputs → keep the pre-supplied-capture path (region-first) -- [x] 2.5 Remember `resolvedWasSelection` from fire → commit -- [x] 2.6 `commit()` in-place branch uses `AICommand.inPlaceCommitPlan` (replace / paste / preview) -- [x] 2.7 Side-effecting branch reads the `sideEffect` accessor (`taskKind(for: command)`; unchanged accept-step/execute) - -## 3. Catalog & seeding (`AI/AICommandCatalog.swift`, `AI/AIBand.swift`) -- [x] 3.1 `preset(…)` helper's legacy `input:`/`output:` args route through the convenience init — entries unchanged, no edit needed -- [x] 3.2 Seeded/curated bands compile and read the same (catalog tests green) - -## 4. Skills serialization (`AI/Skills/SkillFile.swift`, `AI/Skills/SkillToolProvider.swift`) -- [x] 4.1 Serialize `inputs:`/`outputs:` (comma-joined raw values / encoded sinks) -- [x] 4.2 Parse new `inputs:`/`outputs:`; fall back to legacy `input:`/`output:` single lines -- [x] 4.3 `SkillToolProvider` reads `command.sideEffect` for its task kind + schema; round-trip tests green - -## 5. Region routing (`Overlay/LauncherOverlayController.swift`, `App/AppCoordinator.swift`) -- [x] 5.1 `command.input == .screenRegion` → `command.inputs.contains(.screenRegion)` (region-first, exclusive) - -## 6. Editor UI (`Hub/BandsCanvas.swift`) -- [x] 6.1 Input dropdown → multi-toggle over the ambient channels + `screenRegion` as an exclusive choice -- [x] 6.2 Output dropdown → a "Result" kind picker (in-place / task / send-to) + in-place capability toggles; task/`sendTo` sub-editor unchanged -- [x] 6.3 Crossing the in-place ⇄ side-effect boundary re-derives `confirmBeforeRun` -- [x] 6.4 Helper text explaining "the action reads the first available source; selection→replace, clipboard→paste" - -## 7. Tests -- [x] 7.1 `AICommandTests`: set defaults, legacy-scalar migration (pure + decode), `requiredCapabilities` union, `inPlaceCommitPlan` truth table, `sideEffect` accessor -- [x] 7.2 `AICommandExecutorTests`: default selection→replace vs clipboard→paste, empty-selection→clipboard, clipboard-image→vision, disabled-channel skip, no-live-channel→noInput (existing + new) -- [x] 7.3 `AICommandCatalogTests` / `SkillsTests`: presets set-shaped; skill file round-trips new + legacy -- [x] 7.4 Other construction sites updated (blank-command creation uses the all-on defaults) - -## 8. Verify -- [x] 8.1 `swift build` — clean -- [x] 8.2 `swift test` — 1482 tests, 0 failures -- [x] 8.3 `swift build --product ThreeFingerSwitcher` (the MLX-linked executable) — clean -- [ ] 8.4 Apply `ai-command-band` / `selection-io` deltas to the main specs at archive (`/opsx:archive` or `/opsx:sync`) after review — NOT done here to avoid double-applying the deltas diff --git a/openspec/changes/ai-agent-memory/.openspec.yaml b/openspec/changes/ai-agent-memory/.openspec.yaml deleted file mode 100644 index 38f7628..0000000 --- a/openspec/changes/ai-agent-memory/.openspec.yaml +++ /dev/null @@ -1,2 +0,0 @@ -schema: spec-driven -created: 2026-06-22 diff --git a/openspec/changes/ai-agent-memory/design.md b/openspec/changes/ai-agent-memory/design.md deleted file mode 100644 index 3d2fe4b..0000000 --- a/openspec/changes/ai-agent-memory/design.md +++ /dev/null @@ -1,327 +0,0 @@ -## Context - -Three existing seams shape every decision here, and two sibling V2 slices own the contracts this slice -consumes verbatim. - -**Existing code (V2 evolves, never forks):** - -- `DiskProjectStore` (`AI/Tasks/TaskSinks.swift`) is the exact on-disk pattern memory parallels: an - Application-Support directory under `ThreeFingerSwitcher/`, a sanitized deterministic filename per logical - unit, an append/write that maps every `FileManager`/`FileHandle` throw into a clean `TaskError.sinkFailed` - **at the IO boundary** (raw OS error to the log only), and a pure `static` block-builder (`entryBlock`) - that is unit-testable without disk. Memory reuses this shape wholesale: `defaultDirectory()` → - `…/ThreeFingerSwitcher/memory`, sanitized subfile names, boundary-mapped errors, a pure document builder. -- The **Files-band sync-model + async-cache + coalesced-watch** pattern: a pure synchronous model reads a - cache; a miss spawns an off-main listing that stores and republishes; a folder watch coalesces reloads. - Memory's pure document/index logic never touches `FileManager`; the `MemoryStore` bridges IO off-main. -- `AIError.message(for:) -> AIPresentedError` is the single error translator; every memory failure routes - through it. Raw text rides only in logs / opt-in `details`, never a headline. - -**Consumed contracts (owned elsewhere — do NOT redefine):** - -- `DocIndex` / `IndexedDoc` / `DocKind` — OWNED by `ai-skills-as-files` (blueprint 3.4). The shared retriever: - `allSummaries()` (the combined TOC the router always sees), `retrieve(query:limit:)` (deterministic ranked - summaries), `body(of:)` (lazy body load). `IndexedDoc` carries `id`, `title`, `summary` (the TOC line), - `keywords`, `kind` (`.skill`/`.memoryCore`/`.memorySubfile`), `bodyPath`, `updatedAt`. Memory contributes - `IndexedDoc(kind: .memoryCore/.memorySubfile)` into the **same** in-memory snapshot. **Memory defines no - second retriever** (blueprint C2). -- `ToolDescriptor` / `ToolRoute` / `ToolStepResult` / `ToolStepStatus` + the bare `WritePolicyTier` — OWNED - by `ai-tool-routing` (blueprint 3.3, integration fix C1). `WritePolicyTier` is `.auto`/`.confirm`/ - `.dangerous`, defined WITH `ToolDescriptor`, so each memory descriptor carries its tier with no DAG - back-edge. Memory **reads/sets** the tier; it does not own the enum. -- `AuditRecord` / `AuditLog` + the user whitelist + effective-tier resolution — OWNED by - `ai-background-autonomy` (blueprint 3.7). Effective tier = descriptor default ∩ user whitelist; the - whitelist marks memory writes `.auto`. Every memory write emits an `AuditRecord`. Escalation - (`ParkScheduler.escalate` → `.needsYou`) is owned by `ai-parked-sessions`; memory only *classifies* an op - dangerous so the policy layer escalates it. -- `AgentSessionID` — OWNED by `ai-conversation-runtime` (blueprint 3.1). Stable across park/restore; used to - attribute a memory write in its `AuditRecord`. - -The router never wants the whole memory dumped into its prompt — it wants the cheap CORE facts plus the -subfile TOC, and a subfile body only when it picks one. That is the **identical** progressive-disclosure -shape skills already use, which is exactly why memory reuses the skills retriever rather than building its -own. - -## Goals / Non-Goals - -**Goals:** - -- A **two-tier** on-disk memory: a single hard-capped **CORE** document (ground-truth facts + a table of - contents of subfiles), read every session; **named SUBFILES** holding the details, pulled on demand by - relevance through the shared `DocIndex`. -- A **hard byte/line cap on CORE that physically evicts** detail content into a subfile (replacing it in - CORE with a TOC entry) so the always-read tier can never overflow. -- **Promotion to CORE is propose-then-keep**: the agent proposes via a `memory.promote` step; the cap is the - structural backstop; the user overrides by hand-editing. -- **Memory tools** projected as `ToolDescriptor`s for the `ToolRegistry`: `memory.read` (free, `.auto`, - runs even when parked), `memory.write`/`memory.update`/`memory.forget`/`memory.promote` (side-effecting, - whitelisted → effective `.auto` per the adopted decision, **always audited**, dangerous ops `.dangerous`). -- **Editable by agent (by direction) AND by user (by hand)**, converging on the same Markdown files; the - store watches + reloads off-main. -- A pure `MemoryDocument` (CORE parse/serialize + cap/eviction decision) and `MemorySubfile` (front-matter + - body) in MLX-free Core, `swift test`-able headless. - -**Non-Goals:** - -- The route → execute → continue loop, candidate selection, and `ToolRegistry` aggregation - (`ai-tool-routing`); this slice provides a `MemoryToolProvider` contributor only. -- Defining `DocIndex`/`IndexedDoc`/`DocKind` or any ranking/IO (`ai-skills-as-files` owns the retriever). -- Defining `WritePolicyTier`/`AuditRecord`/`AuditLog`, the whitelist, or effective-tier resolution - (`ai-background-autonomy`); this slice sets descriptor defaults and emits records. -- The needs-you escalation transport and parked lifecycle (`ai-parked-sessions`). -- A Hub UI for browsing/editing memory (v1 is file-based, like dropping a `.md`). -- Embeddings / semantic ranking (a future behind the same `DocIndex` seam — M5 can run it, but the - deterministic keyword ranker the skills retriever already provides is the v1 mechanic). -- Cross-device memory sync; encryption-at-rest beyond the OS default. - -## Decisions - -### 1. Two tiers, two file shapes — CORE document + named subfiles. - -The memory folder (`…/Application Support/ThreeFingerSwitcher/memory/`) contains: - -- **`core.md`** — the single always-read CORE document. Two sections: - - **`## Facts`** — a flat list of short ground-truth lines (one fact per bullet). This is what makes the - agent a companion: it is injected into **every** session's context (as a synthetic `system` prefix turn, - assembled by `ai-conversation-runtime`). - - **`## Contents`** — the **table of contents**: one line per subfile, `name — summary`. This is the - memory half of the combined `DocIndex` TOC; the router sees it alongside the skills TOC and can pull a - subfile body on demand. CORE holds the TOC line, never the subfile's detail. -- **`subfiles/.md`** — one named subfile per detail topic, the **same** front-matter+body shape as a - skill file (so the two doc kinds parse through one familiar idiom and contribute uniformly to the index): - - ``` - --- - name: acme-migration - summary: Notes and decisions on the Acme data-migration project. - keywords: [acme, migration, project, deadline] - updatedAt: 2026-06-22T10:00:00Z - --- - The Acme migration is targeted for Q3. Dana owns the schema work… - ``` - - The body is free Markdown detail. `name` is the stable id (sanitized, deterministic filename); `summary` - is the TOC line that appears in `core.md`'s `## Contents` AND is the `IndexedDoc.summary` the router ranks. - -*Why a single `core.md` rather than two files (facts vs TOC):* the facts and the TOC are the **one** thing -read every session and must round-trip atomically under the cap; splitting them invites a desync between -"what CORE claims it has" and "what subfiles exist." The reconciliation pass (Decision 6) keeps the -`## Contents` block honest against the actual subfiles folder. - -*Why Markdown front-matter for subfiles:* it mirrors the skill file format (`ai-skills-as-files` Decision 1), -so a subfile and a skill index identically; it is the human-authoring sweet spot for hand-editing; it -survives copy/paste and version control. - -### 2. `MemoryDocument` (pure Core) — CORE parse/serialize + the cap/eviction decision. - -``` -struct MemoryFact: Equatable, Sendable { var text: String } -struct MemoryTOCEntry: Equatable, Sendable { var name: String; var summary: String } - -struct MemoryDocument: Equatable, Sendable { - var facts: [MemoryFact] - var contents: [MemoryTOCEntry] // mirrors the subfiles; the table of contents - // pure parse from core.md text; pure serialize back to core.md text - static func parse(_ text: String) -> MemoryDocument - func serialized() -> String - - // The cap, as a pure decision. byteCount/lineCount of serialized() must stay ≤ cap. - func wouldExceedCap(addingFact: MemoryFact, cap: MemoryCap) -> Bool - // Eviction picks the lowest-value fact(s) to move out and returns the residual doc + the evicted text. - func evicting(toFit cap: MemoryCap) -> (kept: MemoryDocument, evicted: [MemoryFact]) -} - -struct MemoryCap: Equatable, Sendable { - var maxBytes: Int // hard cap on serialized() size; default tuned (e.g. 8 KB) - var maxFacts: Int // hard cap on fact count; default tuned (e.g. 60) -} -``` - -- **The cap is on the serialized CORE**, because CORE is what gets injected every session — the cost is the - always-read token budget, not disk. Both a byte cap and a fact-count cap (whichever binds first) make the - decision robust to a few very long facts vs. many short ones. -- **`evicting(toFit:)` is a pure function** so the eviction policy is unit-testable headless: it selects the - facts to move out (v1 policy: **oldest-first among the longest facts** — favor evicting bulky detail-bearing - lines, keep terse identity facts), bundles them into subfile body text, and returns the residual document. - The store turns the evicted text into an actual `subfiles/.md` + a `## Contents` entry. -- On M5 this never needs a defensive "memory too big to load" path — the cap keeps CORE tiny by construction - and subfiles are loaded one at a time on demand. - -### 3. The cap is the backstop; promotion to CORE is propose-then-keep. - -The agent does not silently inflate CORE. Two distinct write intents: - -- **`memory.write`** with `tier: subfile` (the default for detail) → writes/updates a **subfile** and - ensures a `## Contents` TOC entry. CORE's fact list is untouched. This is the common case ("remember the - notes on the Acme project"). -- **`memory.promote`** → the agent **proposes** keeping something as a CORE **fact** ("keep this in core?"). - Promotion is a side-effecting step the policy layer can gate; the **hard cap is the structural backstop** — - even an approved promotion that would breach the cap triggers `evicting(toFit:)`, so CORE stays bounded - regardless. The user **overrides** by hand: edit `core.md`'s `## Facts` directly to add/remove a fact, or - move a line between `## Facts` and a subfile. - -So the chosen policy is: **proposal + hard cap.** The proposal keeps the agent honest about what is -"ground truth" worth reading every session; the cap guarantees the always-read tier never grows unbounded no -matter how aggressively the agent (or a buggy loop) promotes. Documented user-override seam: the files are -plain Markdown the user edits by hand, and the store reconciles (Decision 6). - -### 4. Memory tools as `ToolDescriptor`s — read is free, writes are whitelisted-auto + audited. - -`MemoryToolProvider` (Core) exposes `descriptors() -> [ToolDescriptor]` and -`invoke(tool:argumentsJSON:sessionID:) async -> ToolStepResult`, aggregated by `ai-tool-routing`'s -`ToolRegistry`. The descriptors: - -| Tool | argsSchema (`StructuredSchema`) | `writePolicy` (descriptor default) | Behavior | -|---|---|---|---| -| `memory.read` | `{ query?: string }` | `.auto` | Reads CORE (facts + TOC); if `query` present, `retrieve(query:)` the relevant subfiles and loads their bodies. **Free**: runs even when the session is parked; never escalates. | -| `memory.write` | `{ scope: "fact"\|"subfile", name?: string, summary?: string, content: string }` | `.confirm` (→ effective `.auto` via the memory whitelist) | Writes/updates a fact or a subfile + its TOC entry. | -| `memory.update` | `{ name: string, content: string, summary?: string }` | `.confirm` (→ effective `.auto`) | Replaces a named subfile's body/summary. | -| `memory.forget` | `{ scope: "fact"\|"subfile", name?: string, match?: string }` | `.confirm` (→ `.auto` for single; `.dangerous` for bulk) | Removes a fact or a subfile (+ its TOC entry). A `match` that would remove **many** entries, or a CORE-wide clear, classifies `.dangerous`. | -| `memory.promote` | `{ content: string }` | `.confirm` | Proposes a CORE fact (Decision 3); the cap is the backstop. | - -- **Per the adopted user decision (do NOT relitigate):** memory writes are **whitelisted → effective - `.auto`**, so they run **without a confirm even when the session is parked**, BECAUSE the store is - **contained** (it can only ever touch the memory folder — no calendar, no network, no arbitrary file) and - **every write is audited**. `ai-background-autonomy` owns the whitelist + the effective-tier resolution - (descriptor default ∩ whitelist); this slice sets the descriptor default to `.confirm` so that, absent the - whitelist, a memory write is a normal confirm step — and ships the memory whitelist entry as the thing the - user can toggle off to re-require confirmation. -- **Dangerous escalation.** A bulk `memory.forget` (a `match` hitting many entries) or a destructive CORE - rewrite classifies **`.dangerous`** regardless of the whitelist, so it escalates to the foreground via the - needs-you badge even when parked (`ai-parked-sessions` owns the escalation; memory only sets the tier). -- **Every write emits an `AuditRecord`** (`sessionID`, `tool`, `policy`, a **redacted/short** - `argumentsSummary` — the subfile name + a length, never the full secret content — `outcome`, - `wasBackground`, `timestamp`). `memory.read` is also recorded (audit is for every step) but is `.auto` and - free. -- A write whose side effect **does not land** (disk full, permission, read-only volume) maps the IO throw at - the boundary into `MemoryError`, returns `ToolStepResult(status: .failed(headline:))`, and audits the - failure — **never a false "Done."** - -### 5. `MemoryStore` (Core) — owns IO; bridges to the shared index; contains all writes. - -``` -final class MemoryStore { // owns IO; pure logic lives in MemoryDocument/MemorySubfile - init(directory: URL = MemoryStore.defaultDirectory()) - static func defaultDirectory() -> URL // …/ThreeFingerSwitcher/memory (parallels DiskProjectStore) - - func loadCore() async throws -> MemoryDocument - func indexedDocs() async -> [IndexedDoc] // .memoryCore (one) + .memorySubfile (per subfile) → merged into DocIndex - func body(of id: String) throws -> String // a subfile body, or the CORE facts text, for the shared index - - func write(scope:name:summary:content:) async throws -> MemoryWriteOutcome // fact|subfile, applies cap+eviction - func update(name:content:summary:) async throws -> MemoryWriteOutcome - func forget(scope:name:match:) async throws -> MemoryForgetOutcome // classifies dangerous for bulk - func promote(content:) async throws -> MemoryWriteOutcome // cap is the backstop - - // user-folder writes are out of band (user edits files); the store watches + reloads (coalesced, off-main) -} -``` - -- **Containment is structural:** every path the store writes is rooted at `directory`; a `name` is run - through the same sanitizer `DiskProjectStore.fileName(for:)` uses (alphanumerics + `-_ `, never empty, no - traversal), so `memory.write` can never escape the memory folder. This is what makes auto-when-parked - safe. -- **Sync-model + async-cache bridge** (the Files-band pattern): the pure `MemoryDocument`/`MemorySubfile` - logic never touches `FileManager`. `MemoryStore` reads/writes off-main, then publishes an updated - `[IndexedDoc]` snapshot that the shared `InMemoryDocIndex` (owned by skills) answers `allSummaries()` / - `retrieve()` over synchronously. `body(of:)` reads the file (cached), mirroring the skill store. -- **Memory contributes into the SAME index.** `ai-skills-as-files`'s `SkillStore` and this `MemoryStore` - each produce `[IndexedDoc]`; the combined snapshot is one merged enumeration ranked by the one retriever. - IDs are namespaced by a path-relative id and `kind`, and skills/memory live in different folders, so a - skill-id ↔ memory-name collision is structurally impossible (the documented contract from - `ai-skills-as-files` Decision 4 / its edge case). -- **Watch + reload** off-main (coalesced, like the Files-band cache): a hand-edited `core.md` or a dropped - subfile re-indexes and republishes; user edits and agent edits converge on the same files. - -### 6. Reconciliation — `## Contents` always matches the subfiles folder. - -CORE's `## Contents` block and the actual `subfiles/` folder can drift (the user deletes a subfile by hand; -the agent's write crashed between the subfile write and the TOC update). On load and after every write, the -store runs a pure reconciliation: a subfile with no TOC entry gets one (from its front-matter `summary`); a -TOC entry with no subfile is dropped. The **subfiles folder is the source of truth for existence**; the TOC -is a derived view kept honest. This is the same "projection stays consistent because it is computed from the -files" discipline `ai-skills-as-files` uses for the catalog. - -### 7. Error taxonomy — `MemoryError`, mapped at the boundary, one translator. - -A new `enum MemoryError: Error, Equatable, LocalizedError` ONLY for cases `RuntimeError`/`TaskError` cannot -carry: `.unreadableCore(detail:)`, `.writeFailed(detail:)`, `.subfileNotFound(name:)`, `.capExceeded` (a -defensive guard — eviction should always make room, but a single fact larger than the whole cap is a clean -`.capExceeded`), `.malformedSubfile(name:detail:)`. Each has a clean `errorDescription`; raw OS/parse text -rides only in opt-in `details` / logs (the `DiskProjectStore` boundary-mapping discipline). `FileManager` -throws map into `MemoryError` at the `MemoryStore` IO boundary; Core stays MLX-free. `AIError.message(for:)` -is extended to translate `MemoryError` into an `AIPresentedError` — the single translator; never raw error -text in a headline, never an `NSAlert.runModal`. A memory tool failure becomes a -`ToolStepResult(status: .failed(headline:))` fed back into the route loop — observable, never silent. - -## Target split & verification (per component) - -| Component | Target | Verified by | -|---|---|---| -| `MemoryFact`/`MemoryTOCEntry`/`MemoryDocument` (parse/serialize/cap/evict) | Core | `swift test` (round-trip stability, cap boundary, deterministic eviction order) | -| `MemoryCap` defaults + `wouldExceedCap`/`evicting(toFit:)` | Core | `swift test` (byte vs fact cap, evict-to-fit, single-fact-over-cap → `.capExceeded`) | -| `MemorySubfile` parse/serialize (front-matter + body) | Core | `swift test` (parse fixtures, malformed → `MemoryError.malformedSubfile`, round-trip) | -| `MemoryStore` (IO, containment, write/update/forget/promote, reconcile, watch) | Core | `swift test` against temp dirs (containment/sanitization, evict-on-write, reconcile drift, dangerous classification, coalesced reload) | -| Memory `IndexedDoc` contribution into the shared `DocIndex` | Core | `swift test` (memory docs appear in `allSummaries()`, `retrieve()` ranks them, `body(of:)` loads a subfile; combined-with-skills enumeration is one) | -| `MemoryToolProvider` (descriptors + `invoke` → `ToolStepResult`) | Core | `swift test` against `StubLLMRuntime`/fakes (descriptor tiers, read-free, write-audited, forget-bulk-dangerous, fail → `.failed`) | -| `AuditRecord` emission on every write | Core | `swift test` (a fake `AuditLog` records redacted args + outcome + `wasBackground`) | -| `MemoryError` + `AIError.message(for:)` extension | Core | `swift test` (clean headline, no raw interpolation) | -| Full app link (no MLX code added here) | app/GemmaRuntime | `xcodebuild` compile-verify only; **user** does the real install to hand-edit files end-to-end | - -No piece of this slice links MLX — it is value types + file IO + a stub-driven invoke. The whole slice -verifies under `swift build` / `swift test`. (House rule: an agent never builds/signs/installs the `.app`.) - -## Edge cases - -- **A single fact larger than the whole cap** (a user pastes a paragraph as one "fact") → `evicting(toFit:)` - cannot make room by moving other facts; the store routes it to a **subfile** instead (it is detail, not a - fact) and adds a TOC entry, or, if `memory.promote` forced it, returns `MemoryError.capExceeded` as a clean - `.failed` step — never a false "kept in core." -- **Concurrent agent write + user hand-edit of `core.md`** → the store writes atomically (`.atomic`, like - `DiskProjectStore`) and the watch reload re-parses; last-writer-wins on `core.md`, but subfiles are - independent files so a fact write and a subfile edit don't collide. Documented as acceptable for v1 - (single-user, single-machine). -- **`memory.read` while parked** → runs (`.auto`, free), reads CORE + retrieves subfiles, never escalates, - still audited. This is the "background memory writes + whitelisted reads are auto even when parked" - decision in action. -- **`memory.forget(match:)` hitting many entries** → classified `.dangerous`; escalates to foreground via the - needs-you badge even if parked; applies nothing until approved. -- **A subfile on disk with malformed front-matter** → that subfile is a `MemoryError.malformedSubfile` - bounded problem (excluded from the index, the rest load), surfaced as a bounded non-blocking row — never an - `NSAlert`, never a crash, never a silent drop. -- **TOC entry with no backing subfile** (user deleted the subfile by hand) → reconciliation drops the stale - TOC line on next load; the router never sees a phantom subfile. -- **Subfile with no TOC entry** (agent crashed between writes) → reconciliation re-adds the TOC line from the - subfile's front-matter `summary`; the fact tier is unaffected. -- **Empty / absent memory folder** → store creates it on first write; an empty CORE document is valid (no - facts, no TOC); `memory.read` returns an empty-but-valid result, never an error. -- **A memory write whose disk IO fails** → mapped at the boundary into `MemoryError.writeFailed`, - `ToolStepResult(.failed(headline:))`, audited as a failure; never a false "Done." -- **A memory name colliding with a skill id** → structurally impossible: different folders, `kind`-tagged, - path-relative ids (the contract documented by `ai-skills-as-files`). - -## Rejected alternatives - -- **One ever-growing memory file fed every session.** Rejected: it drowns the routing prompt as it grows - (the same problem skills solved with a TOC). The two-tier split keeps the always-read tier hard-capped and - pulls detail on demand. -- **A second, memory-specific retriever.** Rejected by blueprint C2: skills and memory share ONE `DocIndex`; - memory contributes `IndexedDoc`s, it does not define its own ranking/IO. Two retrievers would duplicate - ranking + IO and let the combined TOC drift. -- **Defining `WritePolicyTier` / `AuditRecord` here.** Rejected by blueprint C1 + the ownership split: the - bare `WritePolicyTier` is defined with `ToolDescriptor` in `ai-tool-routing`; `AuditRecord`/`AuditLog` + - the whitelist are owned by `ai-background-autonomy`. Memory sets the descriptor default and emits records; - it owns neither type. -- **Confirm-every-memory-write even when parked.** Rejected by the adopted user decision: a confirm wall on - every "remember…" makes the companion feel like a form. Containment (memory-folder-only) + a full audit - trail makes auto-when-parked safe; the whitelist toggle is the user's off-switch, and dangerous bulk ops - still escalate. -- **Silent auto-promotion of frequently-mentioned facts into CORE.** Rejected: silent CORE growth is exactly - what the cap exists to prevent, and "what is ground truth" is a judgment the user should see. The agent - **proposes** (`memory.promote`); the cap is the backstop; the user overrides by hand. -- **Embedding-based memory retrieval in v1.** Rejected for v1: the shared deterministic keyword ranker is - unit-testable without a model and is the cheap pre-filter; an embedding ranker is an additive future - behind the same `DocIndex` seam (M5 can run it). -- **A separate JSON store for facts.** Rejected: Markdown facts + Markdown subfiles are hand-editable - (the whole point of "editable by the user"), mirror the skill file format, and survive version control; - JSON is hostile to no-code editing of detail prose. diff --git a/openspec/changes/ai-agent-memory/proposal.md b/openspec/changes/ai-agent-memory/proposal.md deleted file mode 100644 index 0e156a1..0000000 --- a/openspec/changes/ai-agent-memory/proposal.md +++ /dev/null @@ -1,95 +0,0 @@ -## Why - -The V2 agent is meant to be a **companion**, not a stateless command box. Today the on-device AI has no -durable knowledge of the user: `AICommandExecutor.fire(_:)` starts from zero every time, and even the new -multi-turn conversation (`ai-conversation-runtime`) is compacted and eventually discarded — nothing the -user told the agent two sessions ago survives. A companion has to **remember**: stable facts ("I work at -Acme", "my partner is Dana", "I prefer metric units"), and the detail behind those facts ("the full notes -on the Acme migration project") — and it has to be able to **write** that memory itself ("remember that…") -and let the user edit it by hand. - -The naive approach — one ever-growing memory file fed every session — fails on the V2 hardware budget for -the wrong reason: not that an M5 can't hold it, but that **dumping unbounded memory into every routing -prompt drowns the signal**. The router (`ai-tool-routing`) already learned this lesson for skills: it sees -a cheap **table of contents** of one-line summaries and pulls a full body only on demand -(`ai-skills-as-files` owns that `DocIndex` retriever). Memory has the identical shape — a small set of -always-true facts plus a large, named, on-demand detail corpus — so it must reuse the **same** retriever, -not invent a second one. - -This slice builds that memory: a **two-tier store** (a hard-capped CORE of ground-truth facts + a table of -contents of named SUBFILES; the subfiles hold the details, pulled by relevance) with **memory tools** the -agent routes to (`memory.read` is free; `memory.write` / `memory.forget` are side-effecting and audited), -**editable by hand and by direction**. It is Wave 3: it consumes the `DocIndex` contract owned by -`ai-skills-as-files`, the `WritePolicyTier`/`AuditRecord` contract owned by `ai-background-autonomy`, and -`AgentSessionID` from `ai-conversation-runtime`. It is entirely MLX-free Core and verifies under -`swift test`. - -## What Changes - -- **A two-tier on-disk memory store** (`MemoryStore`, MLX-free Core) paralleling `DiskProjectStore`: - under Application Support, a single **CORE** document (`core.md`) holding ground-truth facts **plus a - table of contents** that lists every subfile by name with its one-line summary, and a **subfiles** folder - (`memory/subfiles/.md`) holding the details. CORE is read **every session**; subfiles are pulled - **on demand by relevance**. CORE never holds details — only facts and the TOC of subfiles. -- **A hard size cap on CORE that physically evicts to subfiles.** When a write would push CORE past its - byte/line cap, the store **evicts** the lowest-value detail-bearing content into a subfile and replaces it - in CORE with a TOC entry — the cap is a backstop that cannot be exceeded, so the always-read tier stays - bounded regardless of how much the agent tries to keep. -- **Promotion to CORE is propose-then-keep, with the cap as the backstop.** The agent does not silently - fill CORE: it **proposes** ("keep this in core?") via a dedicated `memory.promote` step; the user (or the - whitelist) decides. The hard cap is the structural guarantee underneath the proposal. The user can - override promotion/eviction by editing the files by hand. -- **Memory tools, exposed as `ToolDescriptor`s into the `ToolRegistry`.** `memory.read` (free — runs even - when the session is parked, carries `WritePolicyTier.auto`, reads CORE + retrieves relevant subfiles); - `memory.write` / `memory.update` / `memory.forget` (side-effecting). **Per the adopted user decision, - memory writes are WHITELISTED → effective `.auto`, so they run without a confirm even when parked** — but - they are **always audited** (every write emits an `AuditRecord`) and the store is **contained** (it can - only touch the memory folder). A dangerous memory operation (a bulk `forget`, a CORE rewrite) escalates to - `.dangerous` and surfaces via the needs-you badge. -- **Editable by agent AND by hand.** The agent edits by direction ("remember…" → `memory.write`, - "forget…" → `memory.forget`); the user edits the same Markdown files directly (no app needed), and the - store **watches + reloads** off-main (the Files-band cache pattern). User edits and agent edits converge on - the same files. -- **Memory contributes `IndexedDoc`s into the SHARED `DocIndex`** (kind `.memoryCore` / `.memorySubfile`), - so the combined skills-and-memory table of contents is **one** enumeration ranked by **one** retriever. - This slice **consumes** the `DocIndex` contract owned by `ai-skills-as-files`; it defines **no** second - retriever (blueprint C2). - -## Capabilities - -### New Capabilities - -- `ai-memory`: the two-tier agent memory — a hard-capped CORE ground-truth document carrying a table of - contents of named subfiles, the on-demand subfile detail tier, the evict-on-cap + propose-to-promote - policy, the `memory.read`/`write`/`update`/`forget`/`promote` tools (writes whitelisted-auto + audited), - hand- and agent-editability with off-main watch/reload, and the contribution of memory `IndexedDoc`s into - the shared `DocIndex`. - -## Impact - -- **Code (MLX-free Core, `swift test`):** new `AI/Memory/MemoryStore.swift` (the on-disk two-tier store, - paralleling `DiskProjectStore`), `AI/Memory/MemoryDocument.swift` (the pure CORE parse/serialize: facts + - TOC block, with the cap + eviction decision as a pure function), `AI/Memory/MemorySubfile.swift` (the - subfile front-matter+body format, mirroring the skill file), `AI/Memory/MemoryToolProvider.swift` (the - memory `ToolDescriptor`s + `invoke` that reads/writes and returns a `ToolStepResult`), and a new - `enum MemoryError: Error, Equatable, LocalizedError` plus an `AIError.message(for:)` extension. -- **Reuse, not rebuild:** the `DocIndex`/`IndexedDoc`/`DocKind` retriever (owned by `ai-skills-as-files`) — - memory contributes its docs into the SAME in-memory snapshot; `ToolDescriptor`/`ToolRoute`/`ToolStepResult` - + `WritePolicyTier` (owned by `ai-tool-routing`); `AuditRecord`/`AuditLog` + the user whitelist + effective- - tier resolution (owned by `ai-background-autonomy`); `AgentSessionID` (owned by `ai-conversation-runtime`) - for write attribution; `DiskProjectStore`'s Application-Support + sanitized-filename + IO-boundary-error - pattern; the Files-band sync-model + async-cache + coalesced-watch pattern; `AIError.message(for:)` as the - one translator. -- **MLX-free Core only:** the store, the parse/serialize, the cap/eviction decision, the ranking - contribution, and the tool provider are all pure value types + file IO + a stub-driven invoke, so the whole - slice verifies under `swift build` / `swift test`. No piece links MLX. The full app link is - `xcodebuild` compile-verify only; the **user** does the real install to exercise hand-editing the files - end-to-end. (House rule: an agent never builds/signs/installs the `.app`.) -- **Out of scope:** the route → execute → continue loop and the `ToolRegistry` aggregation - (`ai-tool-routing` owns them; this slice provides a contributor); defining `DocIndex`/`IndexedDoc` or any - ranking/IO (`ai-skills-as-files` owns the retriever); defining `WritePolicyTier`/`AuditRecord`/`AuditLog` - or the whitelist UI (`ai-background-autonomy` owns them; this slice sets the descriptor default and emits - audit records); the parked-session lifecycle and the needs-you escalation transport (`ai-parked-sessions` - owns `ParkScheduler.escalate`; this slice only marks dangerous ops); a Hub UI for browsing/editing memory - (v1 editing is file-based, like dropping a `.md`); embeddings / semantic memory ranking (a future behind - the same `DocIndex` seam); cross-device memory sync. diff --git a/openspec/changes/ai-agent-memory/specs/ai-memory/spec.md b/openspec/changes/ai-agent-memory/specs/ai-memory/spec.md deleted file mode 100644 index fe3c4ec..0000000 --- a/openspec/changes/ai-agent-memory/specs/ai-memory/spec.md +++ /dev/null @@ -1,150 +0,0 @@ -## ADDED Requirements - -### Requirement: Two-tier memory — capped core ground truth plus a table of contents of subfiles -The system SHALL maintain agent memory in **two tiers**: a single **CORE** document holding ground-truth -**facts** and a **table of contents** of named subfiles, and a set of named **SUBFILES** holding the -details. The CORE SHALL be read **every session** (injected as context). The CORE SHALL hold **only** facts -and the table-of-contents line for each subfile — **never** a subfile's detail body. Each table-of-contents -entry SHALL name a subfile and carry its one-line summary. The detail behind a fact SHALL live in a subfile -pulled **on demand by relevance**, not in the CORE. - -#### Scenario: Core is read every session and holds facts plus a table of contents -- **WHEN** a session begins -- **THEN** the core ground-truth facts and the subfile table of contents are available to the agent, and the subfile detail bodies are not loaded until requested - -#### Scenario: Detail lives in a subfile, not in core -- **WHEN** detail content is remembered for a topic -- **THEN** the detail is written into a named subfile and the core holds only a table-of-contents entry (name + summary) pointing at it - -#### Scenario: A subfile body is pulled only on demand -- **WHEN** the agent needs the detail behind a table-of-contents entry -- **THEN** that subfile's body is loaded at that point, by relevance, not eagerly with every session - -### Requirement: Hard core cap that physically evicts to subfiles -The CORE SHALL have a **hard size cap** (a byte cap and a fact-count cap, whichever binds first) that it -SHALL NOT exceed. When a write would push the CORE past the cap, the system SHALL **evict** the -lowest-value detail-bearing fact(s) into a subfile and replace them in the CORE with a table-of-contents -entry, so the always-read tier stays bounded regardless of how much is written. The eviction selection -SHALL be deterministic. A single fact larger than the entire cap SHALL be routed to a subfile (it is -detail, not a fact) or, if forced into the core, SHALL surface as a clean failure — never silently -truncated and never reported as kept-in-core when it was not. - -#### Scenario: A write over the cap evicts to a subfile -- **WHEN** adding a fact would push the core past its cap -- **THEN** the lowest-value existing fact(s) are moved into a subfile, the core gains a table-of-contents entry in their place, and the core stays within the cap - -#### Scenario: The cap is never exceeded -- **WHEN** any sequence of writes is applied -- **THEN** the serialized core never exceeds its byte cap or its fact-count cap - -#### Scenario: A single oversized fact does not silently break the cap -- **WHEN** a single fact is larger than the whole cap -- **THEN** it is stored as a subfile, or the operation fails with a clean headline, and it is never reported as kept in core when it was not - -### Requirement: Promotion to core is proposed, with the cap as the backstop -The agent SHALL NOT silently grow the CORE. Keeping new content as a CORE fact SHALL be a **proposal** the -agent makes (a `memory.promote` step — "keep this in core?"), gated by the write-policy layer, while the -hard cap remains the structural **backstop**: even an approved promotion that would breach the cap SHALL -trigger eviction so the core stays bounded. The user SHALL be able to **override** promotion and eviction by -editing the memory files by hand (adding/removing a fact, moving content between a fact and a subfile). - -#### Scenario: The agent proposes a promotion rather than silently filling core -- **WHEN** the agent wants to keep new content as a core fact -- **THEN** it raises a promotion proposal that the policy layer can gate, rather than writing the core fact silently - -#### Scenario: The cap backstops an approved promotion -- **WHEN** an approved promotion would push the core past its cap -- **THEN** eviction to a subfile runs so the core stays within the cap - -#### Scenario: The user overrides by hand -- **WHEN** the user edits the core document or a subfile directly -- **THEN** the change is honored on reload, including moving content between the fact tier and a subfile - -### Requirement: Memory tools — free read, side-effecting writes that are whitelisted-auto and audited -The system SHALL expose memory operations as routed tools: a **read** tool (`memory.read`) that is **free** -(write-policy `auto`, runs even when the session is parked, never escalates) reading the core and retrieving -relevant subfiles; and **side-effecting** tools (`memory.write`, `memory.update`, `memory.forget`, -`memory.promote`). Per the adopted policy, memory writes SHALL be **whitelisted to an effective `auto` -tier**, so they apply **without a confirmation even when the session is parked**, BECAUSE the memory store is -**contained** (it can write only within the memory folder) and **every memory operation is audited**. The -user SHALL be able to turn off the whitelist so memory writes again require a foreground confirmation. A -**dangerous** memory operation (a bulk forget, a destructive core rewrite) SHALL classify as `dangerous` -regardless of the whitelist and escalate to the foreground via the needs-you badge even when parked. - -#### Scenario: Read runs free, even when parked -- **WHEN** the agent reads memory while the session is parked -- **THEN** the read runs without a confirmation, returns the core facts and the relevant subfiles, never escalates, and is recorded in the audit log - -#### Scenario: A whitelisted write applies auto and is audited -- **WHEN** the agent writes a memory fact or subfile and the memory whitelist is on -- **THEN** the write applies without a foreground confirmation, even if the session is parked, and an audit record is appended - -#### Scenario: Disabling the whitelist re-requires confirmation -- **WHEN** the user turns off the memory-write whitelist -- **THEN** a memory write becomes a confirm step requiring foreground approval - -#### Scenario: A dangerous memory operation escalates -- **WHEN** the agent attempts a bulk forget or a destructive core rewrite -- **THEN** the operation classifies as dangerous and escalates to the foreground via the needs-you badge, applying nothing until approved - -#### Scenario: A memory write that does not land is a failure, never a false success -- **WHEN** a memory write's disk IO fails -- **THEN** the step is reported failed with a clean headline and audited as a failure, never reported done - -### Requirement: Memory is editable by the agent by direction and by the user by hand -Memory SHALL be editable **by the agent by direction** ("remember…" → a write, "forget…" → a removal) and -**by the user by hand** (editing the same memory files directly, with no app interaction required). Agent -edits and user edits SHALL converge on the **same** files, and the system SHALL pick up out-of-band user -edits by **watching and reloading** the memory folder off the main thread. A malformed hand-edited subfile -SHALL surface as a bounded, non-blocking problem (excluded from the index, the rest still loaded), never a -crash, never an app-modal alert, and never a silent drop of the whole memory. - -#### Scenario: The agent edits by direction -- **WHEN** the user tells the agent to remember or forget something -- **THEN** the agent writes or removes the corresponding fact or subfile - -#### Scenario: The user edits by hand and it is picked up -- **WHEN** the user edits the core document or a subfile on disk directly -- **THEN** the change is reloaded off-main and reflected in the memory the agent reads next - -#### Scenario: A malformed subfile is bounded, not fatal -- **WHEN** a subfile on disk has malformed front-matter -- **THEN** that subfile is reported as a bounded problem and excluded from the index while the remaining memory loads normally - -### Requirement: On-disk store paralleling the project-note store, contained to the memory folder -Memory SHALL be persisted **on disk** in an Application-Support directory paralleling the project-note -store, with a single core document file and a subfiles folder. The store SHALL be **contained**: every -write SHALL be rooted inside the memory folder, and a subfile name SHALL be run through a sanitizer that -prevents path traversal and never yields an empty name, so a memory write can never touch a file outside the -memory folder. The store's table of contents SHALL be **reconciled** against the actual subfiles folder on -load and after every write — a subfile with no table-of-contents entry SHALL gain one, and a -table-of-contents entry with no backing subfile SHALL be dropped — so the always-read core never advertises -a subfile that does not exist. - -#### Scenario: Writes stay inside the memory folder -- **WHEN** a memory write names a subfile with traversal characters or slashes -- **THEN** the resolved file path stays rooted inside the memory folder and never escapes it - -#### Scenario: The table of contents is reconciled with the subfiles -- **WHEN** the memory is loaded and the subfiles folder and the table of contents disagree -- **THEN** a missing entry is added from the subfile's summary and a stale entry with no backing subfile is dropped, so the core's table of contents matches the subfiles - -### Requirement: Memory retrieval reuses the single shared document index -Memory SHALL contribute its documents into the **single shared declarative-document index** (the same -retriever used by skills), tagged with their **kind** (memory-core / memory-subfile), and SHALL NOT define a -second retriever. The combined skills-and-memory table of contents SHALL be **one** enumeration ranked by -**one** retriever, and a memory subfile body SHALL be loaded on demand through that same index. Memory -document identifiers SHALL be namespaced (by a path-relative id and kind, in a separate folder from skills) -so a memory name and a skill identifier cannot collide. - -#### Scenario: Memory and skills share one index -- **WHEN** both skill documents and memory documents are present -- **THEN** they appear in one combined index enumeration, each tagged with its kind, ranked by one retriever, with no second memory-specific retriever - -#### Scenario: A memory subfile is retrieved and its body loaded on demand -- **WHEN** a query matches a memory subfile -- **THEN** the subfile's summary is ranked among the results and its body is loaded on demand through the shared index - -#### Scenario: Memory and skill identifiers cannot collide -- **WHEN** a memory subfile name equals a skill identifier -- **THEN** they remain distinct in the index because memory documents are namespaced by a separate folder and kind diff --git a/openspec/changes/ai-agent-memory/tasks.md b/openspec/changes/ai-agent-memory/tasks.md deleted file mode 100644 index 6f5154d..0000000 --- a/openspec/changes/ai-agent-memory/tasks.md +++ /dev/null @@ -1,45 +0,0 @@ -> Wave 3. Consumes `DocIndex`/`IndexedDoc`/`DocKind` (owned by `ai-skills-as-files`), `ToolDescriptor`/`ToolRoute`/`ToolStepResult`/`WritePolicyTier` (owned by `ai-tool-routing`), `AuditRecord`/`AuditLog` + whitelist/effective-tier (owned by `ai-background-autonomy`), and `AgentSessionID` (owned by `ai-conversation-runtime`). Do NOT redefine any of these. §1–§3 are the pure substrate (do first); §4–§6 are the store + index + tools; §7 is the error seam; §8 verifies. - -## 1. CORE document model (pure Core) - -- [x] 1.1 Add `MemoryFact`, `MemoryTOCEntry`, and `MemoryDocument` (`AI/Memory/MemoryDocument.swift`, MLX-free Core): a `## Facts` list + a `## Contents` (TOC) list, with pure `parse(_:) -> MemoryDocument` and `serialized() -> String`. Keep it pure/`Sendable`, no `FileManager`. *Verify: `swift test` round-trips parse↔serialize byte-stably, including empty doc, facts-only, TOC-only.* -- [x] 1.2 Add `MemoryCap{maxBytes, maxFacts}` with tuned defaults, and the pure cap decision: `wouldExceedCap(addingFact:cap:)` and `evicting(toFit:) -> (kept, evicted: [MemoryFact])` selecting **oldest-among-longest** facts to move out. *Verify: `swift test` — byte cap vs fact cap (whichever binds first), evict-to-fit makes room, deterministic eviction order, a single fact larger than the whole cap surfaces as the documented `.capExceeded` boundary (not an infinite evict loop).* - -## 2. Subfile model (pure Core) - -- [x] 2.1 Add `MemorySubfile` (`AI/Memory/MemorySubfile.swift`, Core): the front-matter (`name`/`summary`/`keywords`/`updatedAt`) + Markdown body shape, mirroring the skill file format, with pure parse/serialize. *Verify: `swift test` — parse fixtures, round-trip stability.* -- [x] 2.2 Map a malformed subfile (missing `name`/`summary`, bad front-matter) at the parse boundary into a bounded problem (`MemoryError.malformedSubfile(name:detail:)` — defined in §7), excluded from the index, the rest still loadable. *Verify: `swift test` — a malformed fixture yields a problem, not a throw-through; the good fixtures still parse.* - -## 3. Reconciliation (pure Core) - -- [x] 3.1 Add the pure reconcile pass: given a `MemoryDocument` and the set of actual subfile front-matters, return a normalized `MemoryDocument` whose `## Contents` matches the subfiles folder (add a missing TOC entry from a subfile's `summary`; drop a TOC entry with no backing subfile). The subfiles folder is the existence source of truth. *Verify: `swift test` — TOC-with-no-subfile dropped, subfile-with-no-TOC re-added, already-consistent is a no-op.* - -## 4. `MemoryStore` (Core, owns IO) - -- [x] 4.1 Add `MemoryStore` (`AI/Memory/MemoryStore.swift`, Core) paralleling `DiskProjectStore`: `defaultDirectory()` → `…/Application Support/ThreeFingerSwitcher/memory`, a test/seam `init(directory:)`, and the sanitized deterministic subfile filename (reuse the `DiskProjectStore.fileName(for:)` sanitizer — alphanumerics + `-_ `, never empty, no traversal). *Verify: `swift test` against a temp dir — directory created on first write; a hostile `name` (`../`, slashes) stays rooted inside the memory folder (containment).* -- [x] 4.2 `loadCore()` + `indexedDocs() -> [IndexedDoc]`: read `core.md` off-main into a `MemoryDocument`, list subfiles, run §3 reconcile, and produce the memory `IndexedDoc`s — one `kind: .memoryCore` (facts) + one `kind: .memorySubfile` per subfile — for merging into the shared `DocIndex` snapshot. `body(of:)` loads a subfile body / the CORE facts text (cached). *Verify: `swift test` — memory docs appear with correct `kind`/`summary`; `body(of:)` returns a subfile body; reconcile runs on load.* -- [x] 4.3 `write(scope:name:summary:content:)` (fact|subfile), `update(name:content:summary:)`, `promote(content:)`: apply through `MemoryDocument` (cap + eviction on a fact write/promote — evicted facts become a subfile + TOC entry), write atomically (`.atomic`), reconcile, republish the snapshot. *Verify: `swift test` — a fact write over the cap evicts to a subfile; a subfile write adds a TOC entry; `promote` over the cap still evicts (cap is the backstop); a single-fact-over-cap promote returns `.capExceeded`.* -- [x] 4.4 `forget(scope:name:match:)`: remove a fact / a subfile (+ its TOC entry); classify a `match` hitting **many** entries or a CORE-wide clear as **dangerous** (returned in the outcome so the tool sets `.dangerous`). *Verify: `swift test` — single forget removes one + its TOC line; a broad `match` is flagged dangerous; a no-match forget is a clean no-op, not a throw.* -- [x] 4.5 Coalesced off-main watch + reload (the Files-band cache pattern): a hand-edited `core.md` / dropped subfile re-indexes and republishes. *Verify: `swift test` — after an external file change, a reload yields the updated snapshot; rapid changes coalesce to one reload.* — RELOAD verified (`testWatchReloadPicksUpExternalEdit`: an out-of-band `core.md` edit is reflected on the next `loadCore()`, since the store always re-reads from disk). The live FSEvents watcher + debounce that fires the reload automatically on a file-system event is deferred to the publish-loop owner (`ai-tool-routing`/`ai-background-autonomy`) and is the live-OS-event piece an agent cannot validate headless; the reload mechanic it drives is implemented + tested. - -## 5. Memory `IndexedDoc` contribution into the shared `DocIndex` - -- [x] 5.1 Wire `MemoryStore.indexedDocs()` into the **shared** `InMemoryDocIndex` snapshot (owned by `ai-skills-as-files`) so skills + memory form **one** combined `allSummaries()` enumeration ranked by the **one** retriever — no second retriever. *Verify: `swift test` — a combined snapshot of skills + memory ranks both via `retrieve()`; a memory subfile is selectable and `body(of:)`-loadable; ids are namespaced so a skill-id ↔ memory-name collision is structurally impossible.* - -## 6. `MemoryToolProvider` (Core) — descriptors + invoke - -- [x] 6.1 Add `MemoryToolProvider` (`AI/Memory/MemoryToolProvider.swift`, Core) exposing `descriptors() -> [ToolDescriptor]`: `memory.read` (`.auto`, `{query?}`), `memory.write` (`.confirm`, `{scope,name?,summary?,content}`), `memory.update` (`.confirm`, `{name,content,summary?}`), `memory.forget` (`.confirm`; `.dangerous` for bulk, `{scope,name?,match?}`), `memory.promote` (`.confirm`, `{content}`). Reuse `StructuredSchema` for each `argsSchema`. *Verify: `swift test` — each descriptor's name/summary/argsSchema/`writePolicy` default matches the table; `memory.read` is `.auto`.* -- [x] 6.2 `invoke(tool:argumentsJSON:sessionID:) async -> ToolStepResult`: validate args, dispatch to the matching `MemoryStore` op, return `.done`/`.declined`/`.failed(headline:)`. A read runs free even when parked; a write/forget routes through the store. *Verify: `swift test` against fakes — read returns CORE + retrieved subfiles; write applies + returns `.done`; an IO failure returns `.failed` (never a false `.done`); a malformed-args route declines.* -- [x] 6.3 Emit an `AuditRecord` for **every** invoke (read included) with a **redacted/short** `argumentsSummary` (subfile name + content length, never the raw secret), the `sessionID`, the effective `policy`, the `outcome`, and `wasBackground`. *Verify: `swift test` — a fake `AuditLog` records a redacted summary + outcome + `wasBackground` for read, write, and a failed write.* -- [x] 6.4 Set the descriptor defaults so that, absent the whitelist, a memory write is a `.confirm` step; ship the memory-write whitelist entry (the user's off-switch) as the thing `ai-background-autonomy` resolves to effective `.auto`. Classify dangerous ops `.dangerous` regardless of the whitelist. *Verify: `swift test` — descriptor default is `.confirm`; a bulk forget is `.dangerous`; (effective-tier resolution itself is `ai-background-autonomy`'s test, not duplicated here).* - -## 7. Error taxonomy + one translator - -- [x] 7.1 Add `enum MemoryError: Error, Equatable, LocalizedError` (`.unreadableCore`, `.writeFailed`, `.subfileNotFound`, `.capExceeded`, `.malformedSubfile`) with clean `errorDescription`s; map `FileManager`/parse throws into it at the `MemoryStore` boundary (raw text to the log only). *Verify: `swift test` — each case has a clean, non-raw description; an injected IO failure surfaces `.writeFailed` with the raw error absent from the message.* -- [x] 7.2 Extend `AIError.message(for:)` to translate `MemoryError` → `AIPresentedError` (the single translator); raw text only in opt-in `details`/logs, never a headline; failures surface bounded + non-blocking (no `NSAlert.runModal`). *Verify: `swift test` — `AIError.message(for: MemoryError.…)` yields a clean headline and no raw interpolation.* - -## 8. Verify - -- [x] 8.1 `swift build` + `swift test` green: the pure `MemoryDocument`/cap/evict + `MemorySubfile` + reconcile + `MemoryStore` (temp-dir containment/evict/reconcile/watch) + the shared-index contribution + `MemoryToolProvider` (descriptors/invoke/audit) + `MemoryError`/translator are all covered. The full `ThreeFingerSwitcher` product builds + links (Core + GemmaRuntime/MLX) — no MLX added here. *Verify: `swift test`; `xcodebuild` compile-verify the full app target (compile only — agent never signs/installs).* -- [x] 8.2 `openspec validate --strict` passes; the `ai-memory` spec (ADDED-only, brand-new capability) matches the implementation, and the consumed contracts (`DocIndex`, `ToolDescriptor`/`WritePolicyTier`, `AuditRecord`, `AgentSessionID`) are referenced, never redefined. *Verify: `openspec validate --strict`.* -- [ ] 8.3 **User run-verify** in a stable-signed build: "remember that…" writes a fact/subfile that survives the next session; the agent's CORE stays within the cap as it learns (eviction to subfiles is invisible to the user but the facts it pulls back stay correct); hand-editing `core.md` / a subfile is picked up on reload; "forget…" removes it; a parked session can still auto-write memory and it shows in the audit trail; a bulk forget escalates to needs-you. diff --git a/openspec/changes/ai-background-autonomy/.openspec.yaml b/openspec/changes/ai-background-autonomy/.openspec.yaml deleted file mode 100644 index 38f7628..0000000 --- a/openspec/changes/ai-background-autonomy/.openspec.yaml +++ /dev/null @@ -1,2 +0,0 @@ -schema: spec-driven -created: 2026-06-22 diff --git a/openspec/changes/ai-background-autonomy/design.md b/openspec/changes/ai-background-autonomy/design.md deleted file mode 100644 index 1f7515d..0000000 --- a/openspec/changes/ai-background-autonomy/design.md +++ /dev/null @@ -1,271 +0,0 @@ -## Context - -This slice is **Wave 4** of the V2 AI agent (see `docs/ai-agent-v2-blueprint.md` §3.7). It is the **policy layer**: it owns the user **whitelist**, **effective-tier resolution**, the append-only **audit log**, and the parked **auto-vs-escalate** decision. It consumes types from earlier waves verbatim and never redefines them. - -Read these before the design — they are the ground truth this slice plugs into, not forks: - -- **`ai-tool-routing` (Wave 2, on disk at `openspec/changes/ai-tool-routing/`)** — OWNS the bare `WritePolicyTier` enum (`auto`/`confirm`/`dangerous`) on every `ToolDescriptor`, integration fix **C1**. It already gates each step through an **injected `WritePolicyResolving`** seam with a stand-alone `DescriptorWritePolicy` default that returns `descriptor.writePolicy` unchanged. Its loop (`AgentLoop`, Decision 7) calls `registry.run(call, gate:)`; the contributor reads the effective tier and either runs `.auto` immediately or pauses `.confirm`/`.dangerous` for the `ApprovalGate` (DOWN=approve / RIGHT=skip). **This slice ships the real `WritePolicyResolving` conformer** that replaces `DescriptorWritePolicy` in production, plus the audit + escalation wiring around `registry.run`. No routing type is redefined here. -- **`ai-parked-sessions` (Wave 3, on disk)** — OWNS `ParkState` (`active`/`parked`/`needsYou`/`idle`), `ParkedSession`, and `ParkScheduler` with `escalate(_ id: AgentSessionID, reason: String)` (→ `.needsYou` + badge + ambient notch glow). It explicitly defers "what makes a step `needs-you`" to **this** slice. This slice never renders the badge or glow — it **calls `escalate`**; the rail renders. -- **`ai-conversation-runtime` (Wave 1)** — OWNS `AgentSessionID` (used to attribute every `AuditRecord` and every escalation). -- **`AI/Tasks/TaskSinks.swift` / `TaskDispatcher.swift` / `AICommandExecutor.swift`** — the existing side-effect machinery. The sinks (`CalendarSink`, `ReminderSink`, `ContactSink`, `ProjectStore`, `ToolOpener`, `DestinationSender`) are what the tiers map onto. `TaskError`/`AIError.message(for:)` already enforce "no raw error text in a headline; a side effect that did not land is `.failed`, never a false Done." This slice inherits that contract — an audited outcome is a `ToolStepStatus`, never a fabricated success. -- **`AI/AIError.swift`** — the SINGLE translator `AIError.message(for:) -> AIPresentedError`. Every headline this slice surfaces (a failed audit-store write, an escalation reason) routes through it. - -The blueprint §3.7 pins `AuditRecord`/`AuditLog` and the user decisions; this slice OWNS them. - -## Goals / Non-Goals - -**Goals:** -- A crisp **blast-radius tier model** (CONTAINED / WHITELISTED / DANGEROUS) with every existing and planned tool/sink mapped to a default `WritePolicyTier`, and the rule for lowering a tier to `.auto` (CONTAINED or whitelist match) — pure, `swift test`-able. -- A **user-visible, user-editable whitelist**: path prefixes + command-pattern globs, default-empty for arbitrary entries, memory + project stores **pre-trusted** as CONTAINED (not whitelist rows). Pure matching rules. -- The concrete **`WritePolicyResolving`** conformer the routing loop already injects, so resolution is a drop-in (no routing API change). -- An **append-only audit log** (`AuditRecord`/`AuditLog`) with a redacted args summary, persisted + capped, viewable in the notch rail drop-down and the Hub. -- A pure **`BackgroundGate`** auto-vs-escalate decision: `.auto` runs + audits even when parked; `.dangerous` while parked → `escalate` → `.needsYou`; `.confirm` while parked waits. -- One `AuditError` only where `RuntimeError`/`TaskError`/`ParkError` cannot carry a store failure; mapped at the boundary; surfaced bounded + non-blocking. - -**Non-Goals:** -- Defining `WritePolicyTier` (OWNED by `ai-tool-routing`, C1) — consumed verbatim. -- The per-step **approval gesture** + `ApprovalGate` (OWNED by `ai-tool-routing` / `ai-conversational-canvas`); this slice decides *which tier a parked step is*, not how the user resolves it. -- The needs-you **badge + notch glow rendering** and the `ParkState` machine (OWNED by `ai-parked-sessions`); this slice only calls `escalate`. -- The Claude-handoff **budget/rate cap** (OWNED by `ai-claude-handoff`); it consumes `AuditRecord` and the `.confirm`-default/`.auto`-per-skill resolution from here. -- **Rollback/undo** of a completed side effect — explicitly impossible (`ai-parked-sessions` Decision 7); the audit log records what happened, it does not reverse it. -- Any Intel/low-end fallback. Apple-Silicon M5/M4 only. - -## Decisions - -### 1. The blast-radius tier model — three tiers, mapped onto `WritePolicyTier` - -The three blast-radius tiers are the **conceptual** classification; they collapse onto the existing `WritePolicyTier` (`auto`/`confirm`/`dangerous`) so no new enum is born: - -| Blast radius | Meaning | Parked behavior | `WritePolicyTier` | -|---|---|---|---| -| **CONTAINED** | The app's own stores — agent memory + project notes. Bounded, reversible-in-spirit, never touches the user's wider filesystem. | **AUTO, even parked** (still audited). | descriptor ships `.auto` | -| **WHITELISTED** | A write/command whose **target matches a user whitelist entry** (a path under a trusted prefix, or a command matching a trusted pattern). | **AUTO if matched** (still audited). | descriptor ships `.confirm`; resolution **lowers to `.auto`** on a match | -| **DANGEROUS** | Delete, overwrite an existing file, arbitrary shell, anything **off-list**, and the Claude-handoff cost. | **FOREGROUND-ONLY**: park → `needsYou` badge + notch glow; resolved by pulling the session back and DOWN=approve. | descriptor ships `.dangerous` (or `.confirm` that did NOT match the whitelist) | - -The key asymmetry, encoded once: **the whitelist may only LOWER `.confirm` → `.auto`; it may NEVER lower `.dangerous`.** A delete or overwrite-existing is dangerous regardless of where it lands — a trusted folder does not make `rm` safe. CONTAINED is intrinsic to the descriptor (the memory/project store tools ship `.auto`), not a whitelist row, so it is auto even with an empty whitelist. - -**Tool/sink → default tier map** (each is the `writePolicy` the `ToolDescriptor` ships with, set by the owning slice; this slice asserts the mapping and classifies CONTAINED-ness): - -| Tool / sink (current or planned) | Owning slice | Blast radius | Default `WritePolicyTier` | Notes | -|---|---|---|---|---| -| `memory.read` | ai-agent-memory | CONTAINED (read) | `.auto` | read-only; always auto | -| `memory.write` (core / subfile) | ai-agent-memory | **CONTAINED** | `.auto` | the app's own store → auto even parked, audited | -| `save_to_project:` (`ProjectStore`) | ai-command-tasks | **CONTAINED** | `.auto` | append-only into the app's project note store | -| `retrieve` / `widen_candidates` (skills+memory index) | ai-tool-routing / skills | CONTAINED (read) | `.auto` | read-only retrieval | -| `add_to_calendar` (`CalendarSink`) | ai-command-tasks | external write | `.confirm` | creates an event in the user's calendar; whitelist does not apply (no path/command) → stays `.confirm` | -| `add_to_reminders` (`ReminderSink`) | ai-command-tasks | external write | `.confirm` | as above | -| `new_contact` (`ContactSink`) | ai-command-tasks | external write | `.confirm` | as above | -| `open_tool_with_payload:` (`ToolOpener`) | ai-command-tasks | launch / shell | `.confirm` → maybe `.auto` | a **command pattern** match (the tool/Shortcut name) can lower to `.auto`; a free-form shell tool that matches no pattern stays `.confirm` | -| `send_to:` (`DestinationSender`) | ai-command-tasks | shortcut / URL / **shell** | `.confirm` (shell variant **`.dangerous`**) | the `.shell(command)` destination is arbitrary shell → DANGEROUS unless its command matches a whitelist pattern; `.shortcut`/`.urlScheme` are `.confirm`, lowerable by a command-pattern match | -| `launch_claude` (`ClaudeHandoffContributor`) | ai-claude-handoff | spawns a Claude process (cost) | `.confirm` (per-skill `.auto` opt-in) | the handoff cost is DANGEROUS-by-spend; default `.confirm`, a skill may opt `.auto`, still budget-capped + audited (owned there) | -| a future **file delete / move / overwrite-existing** tool | future | **DANGEROUS** | `.dangerous` | never lowered by the whitelist; always foreground | - -This table is the **classification authority**: the slice ships a `BlastRadius` view over a descriptor (`contained` / `external` / `dangerous`) derived from the descriptor's name + tier, and unit-tests the whole map. - -### 2. `BlastRadius` + the CONTAINED predicate (pure Core) - -```swift -public enum BlastRadius: Equatable, Sendable { - case contained // app's own stores; descriptor ships .auto - case external // touches the user's world but bounded (calendar/contacts/launch/send) - case dangerous // delete/overwrite-existing/arbitrary-shell/off-list/handoff-cost -} -``` - -A pure classifier reads a `ToolDescriptor` (its stable `name` + `writePolicy`): -- `descriptor.writePolicy == .dangerous` → `.dangerous` (unconditional). -- the descriptor name is a CONTAINED tool (memory.* / save_to_project / read-only retrieval) → `.contained`. -- otherwise → `.external`. - -CONTAINED-ness is matched on a **stable name prefix set** owned here (`["memory.", "save_to_project", "retrieve", "widen_candidates"]`) so the policy layer recognizes the app's own stores without importing the memory slice. (A descriptor whose owning slice flags itself CONTAINED via its shipped `.auto` tier AND a name in this set is contained; a `.auto` descriptor NOT in the set is a defensive `.external` — a non-contained tool should never ship `.auto`, asserted in debug.) - -### 3. The whitelist — a pure, persisted, user-editable security boundary - -```swift -public struct Whitelist: Codable, Equatable, Sendable { - public var trustedPathPrefixes: [String] // standardized absolute path prefixes the user trusts for writes - public var trustedCommandPatterns: [String] // glob patterns matched against a command/tool/shortcut name - public static let empty = Whitelist(trustedPathPrefixes: [], trustedCommandPatterns: []) -} -``` - -- **Default-empty** for arbitrary entries — a fresh install trusts nothing on the wider filesystem. The memory + project stores are CONTAINED (Decision 2), so they are auto **without** any whitelist row; the user never has to whitelist the app's own stores. -- **Matching rules (pure, unit-tested):** - - A **path target** matches iff its standardized absolute path has one of `trustedPathPrefixes` as a path-component prefix (prefix is `/Users/me/Notes` ⇒ `/Users/me/Notes/x.md` matches, `/Users/me/Notes2` does NOT — component-boundary match, never a bare string prefix). Symlinks/`..` are resolved (`standardizedFileURL`) **before** matching so `/Users/me/Notes/../etc` cannot sneak past. - - A **command target** (the tool/Shortcut name, or a shell command's argv[0]) matches iff it matches one of `trustedCommandPatterns` as an `fnmatch`-style glob (`*`/`?`), anchored full-string. - - A target that is **both** (a shell command writing to a path) must match BOTH a command pattern AND, if it names a path, a path prefix — the stricter rule wins (a whitelisted command pointed at an un-trusted path stays `.confirm`). -- **The whitelist never lowers `.dangerous`** (Decision 1). It is consulted ONLY for an `external`/`.confirm` descriptor. -- Persisted in `AppSettings` (new keys `agentWhitelistPaths`, `agentWhitelistCommands`); included in the Hub AI page's reset semantics like other opt-ins. The model is pure (the *editing* UI is App-target, Decision 8). - -### 4. Effective-tier resolution — the concrete `WritePolicyResolving` (the drop-in) - -`ai-tool-routing` defines and injects: -```swift -public protocol WritePolicyResolving: Sendable { - func effectiveTier(for descriptor: ToolDescriptor) -> WritePolicyTier -} -``` -and ships `DescriptorWritePolicy` (identity) as the stand-alone default so its slice tests in isolation. **This slice ships the production conformer** — but it needs the *call's arguments* (the path/command target) to consult the whitelist, which `effectiveTier(for descriptor:)` alone does not carry. Two clean options: - -- **Adopted:** extend the resolution at the call site, not the protocol. This slice provides a `BackgroundPolicyResolver` that conforms to `WritePolicyResolving` for the descriptor-only fast path (CONTAINED→`.auto`, `.dangerous`→`.dangerous`, else the descriptor tier), **plus** a richer `effectiveTier(for descriptor:, target:)` overload that the contributor/loop calls when it has the routed call's target string. The protocol method stays satisfied (descriptor-only); the overload adds the whitelist lowering. No routing protocol change — the loop already holds the `RoutedCall` and can pass its target. - -```swift -public struct BackgroundPolicyResolver: WritePolicyResolving, Sendable { - public let whitelist: Whitelist - // Protocol requirement (descriptor-only): used when no target is available. - public func effectiveTier(for d: ToolDescriptor) -> WritePolicyTier - // Richer overload: lowers .confirm → .auto when target matches the whitelist; never lowers .dangerous. - public func effectiveTier(for d: ToolDescriptor, target: PolicyTarget?) -> WritePolicyTier -} - -public enum PolicyTarget: Equatable, Sendable { - case path(String) // a write destination - case command(String) // a tool / shortcut / shell command name - case both(command: String, path: String) - case none // calendar/contacts: no path/command → never whitelist-lowerable -} -``` - -Resolution table (pure, the unit-test heart of the slice): - -| descriptor.writePolicy | BlastRadius | target matches whitelist | → effective tier | -|---|---|---|---| -| `.auto` | contained | (n/a) | `.auto` | -| `.dangerous` | dangerous | (ignored) | **`.dangerous`** | -| `.confirm` | external | yes | **`.auto`** | -| `.confirm` | external | no / `.none` | `.confirm` | - -The contributor extracts the `PolicyTarget` from the routed call (a `save_to_project` has no user-path target → it is CONTAINED anyway; an `open_tool`/`send_to` exposes its tool/command; a future delete/move exposes its path). The extraction lives next to the descriptor that owns the args shape; this slice supplies the resolver + the matching, not the per-tool arg parsing. - -### 5. The audit log — append-only, redacted, capped, persisted - -```swift -public struct AuditRecord: Codable, Equatable, Identifiable, Sendable { - public let id: UUID - public let sessionID: AgentSessionID // attribution (3.1) - public let tool: String // the descriptor name - public let policy: WritePolicyTier // the EFFECTIVE tier this step ran at - public let argumentsSummary: String // redacted/short — NEVER raw secrets or full bodies - public let outcome: ToolStepStatus // reuses 3.3 (.done/.declined/.failed(headline)…) - public let wasBackground: Bool // true if applied while parked - public let timestamp: Date -} - -public protocol AuditLog: Sendable { - func record(_ r: AuditRecord) - func recent(limit: Int) -> [AuditRecord] -} -``` - -- **Every tool step writes one record** — auto, confirmed, declined, escalated, or failed — so the ledger is a complete "what did my agents do." A `.declined`/skipped step is recorded too (it is part of the story). An *escalated* step records `outcome = .awaitingApproval` at escalation time and a follow-up record on resolution (approve→`.done`/`.failed`, skip→`.declined`). -- **`argumentsSummary` redaction (pure, unit-tested):** a short, bounded, single-line summary built from the routed args — the path's last component(s), the command name (never the full shell line with embedded secrets), a truncated content preview (`.lineLimit`-style cap, `…` middle-truncation). The blueprint's "redacted/short, NOT raw secrets" — raw args go nowhere near the record. (Full raw args, if ever needed for debugging, live only in the os.Logger breadcrumb at the sink boundary, exactly as `TaskSinks` already logs.) -- **Outcome carries a clean headline only.** A `.failed(headline)` stores `AIPresentedError.headline` — never raw OS text. The audit viewer shows the headline; raw text (if any) stays behind the existing "Show details" disclosure pattern. -- **Append-only + capped:** `DiskAuditLog` appends to a JSON-lines file under Application Support (mirroring `ClipboardStore`/`ParkedSessionStore` on-disk patterns), retaining the most recent N (e.g. 500) with oldest-trimmed-on-write. `record(_:)` is non-blocking — it enqueues to a serialized off-main writer (the Files-band sync-model + async-cache pattern: a pure in-memory ring the viewers read synchronously, disk IO bridged off-main). A write failure maps to `AuditError` at the boundary and is itself surfaced bounded (a one-line "couldn't persist the audit log" banner on the Hub viewer) — it NEVER throws into the route loop (auditing must not break the agent), and the in-memory ring still has the record. -- **Two readers, one log:** the notch rail drop-down shows the recent slice for a parked session (or all sessions); the Hub AI page shows the full recent ledger. Both call `recent(limit:)`. `wasBackground` lets the UI distinguish "while you were away" from foreground actions. - -### 6. The gating decision — `BackgroundGate` (pure, the auto-vs-escalate brain) - -```swift -public enum BackgroundDecision: Equatable, Sendable { - case auto // run now, in the background, then audit - case waitParked // .confirm while parked: stay parked, resolve on restore (no escalation) - case escalate(reason: String) // .dangerous while parked: → ParkScheduler.escalate → .needsYou + glow - case foreground // session is active: the existing ApprovalGate handles it (this slice no-ops) -} - -public enum BackgroundGate { - static func decide(effectiveTier: WritePolicyTier, parkState: ParkState) -> BackgroundDecision -} -``` - -Decision table (pure; `ParkState` from `ai-parked-sessions`): - -| effectiveTier | parkState | → BackgroundDecision | -|---|---|---| -| `.auto` | `.parked` / `.idle` | `.auto` (run + audit) | -| `.auto` | `.active` | `.auto` (foreground auto still runs + audits) | -| `.confirm` | `.parked` / `.idle` | `.waitParked` (resolved on restore via the routing approval gate) | -| `.confirm` | `.active` | `.foreground` (the canvas approval gate owns it) | -| `.dangerous` | `.parked` / `.idle` | `.escalate(reason)` (→ `.needsYou`) | -| `.dangerous` | `.active` | `.foreground` (already in front; the approval gate owns it, no glow needed) | -| any | `.needsYou` | `.waitParked` (already escalated; do not double-escalate) | - -The **wiring** (App/integration, the routing loop's contributor host): for each routed call, compute `effectiveTier` (Decision 4), call `BackgroundGate.decide(...)`, then: -- `.auto` → run the step, write an `AuditRecord(wasBackground: parked, policy: .auto)`. -- `.escalate(reason)` → call `scheduler.escalate(sessionID, reason:)` (the `ai-parked-sessions` seam → `.needsYou` + badge + glow), write an `AuditRecord(outcome: .awaitingApproval, wasBackground: true)`, and **suspend** the step (it resumes when the user pulls the session back and approves, exactly the routing `ApprovalGate` flow — the step is now foreground). -- `.waitParked` → leave the step paused on the routing `ApprovalGate`; no escalation, no glow (a `.confirm` is not urgent). -- `.foreground` → no-op for this slice; the routing approval gate + canvas drive it as today. - -`reason` for escalation is a clean one-liner (`" needs your approval (\(blast))"`) — short, headline-grade, routed through `AIError.message(for:)` if it ever wraps an error. The glow itself is `ai-parked-sessions`'. - -### 7. Errors — one `AuditError`, mapped at the boundary, bounded + non-blocking - -`enum AuditError: Error, Equatable, LocalizedError` carries ONLY what `RuntimeError`/`TaskError`/`ParkError` cannot: the **audit store** persistence failure (`.persistFailed`, `.storeUnavailable`). `FileManager`/JSON errors map into `AuditError` at the `DiskAuditLog` boundary so Core stays MLX-free and no OS error text leaks. It routes through `AIError.message(for:)` → `AIPresentedError` for a clean headline. A store failure surfaces as a **bounded, non-blocking** banner on the Hub audit viewer (headline only; raw text behind "Show details"), **never** `NSAlert.runModal`, and crucially **never throws into the route loop** — `AuditLog.record(_:)` is infallible from the caller's view (it enqueues; a persistence failure is observed on the viewer, the in-memory ring is unaffected). The whitelist matching and the gate are pure value logic and cannot fail. - -No other taxonomy is introduced: policy resolution produces a `WritePolicyTier` (never an error); a denied side effect is already a `TaskError`→`.failed` recorded as the audit outcome. - -## Type & file touch list (all Core, MLX-free, verified by `swift test` unless noted) - -| File (new unless noted) | Target | Contents | Verification | -|---|---|---|---| -| `AI/Audit/BlastRadius.swift` | Core | `BlastRadius` enum + the pure descriptor→radius classifier; the CONTAINED name-prefix set | `swift test` (every mapped tool → expected radius; `.dangerous` is unconditional) | -| `AI/Audit/Whitelist.swift` | Core | `Whitelist` (Codable) + pure path-prefix (component-boundary, standardized) and command-glob (`fnmatch`) matching + the both-rule | `swift test` (prefix component boundary; `..`/symlink resolved before match; glob anchored; both-rule) | -| `AI/Audit/WritePolicyResolution.swift` | Core | `BackgroundPolicyResolver: WritePolicyResolving` (descriptor-only) + the `effectiveTier(for:target:)` overload + `PolicyTarget` | `swift test` (full resolution table; `.dangerous` never lowered; `.confirm`+match→`.auto`) | -| `AI/Audit/BackgroundGate.swift` | Core | `BackgroundDecision` + pure `BackgroundGate.decide(effectiveTier:parkState:)` | `swift test` (full decision table; `.needsYou` never double-escalates) | -| `AI/Audit/AuditRecord.swift` | Core | `AuditRecord` (Codable/Identifiable) + the pure `argumentsSummary` redaction builder | `swift test` (Codable round-trip; redaction truncates + strips raw secrets; headline-only failed outcome) | -| `AI/Audit/AuditLog.swift` | Core (IO bridged off-main) | `AuditLog` protocol + in-memory ring `InMemoryAuditLog` + persisted append-only `DiskAuditLog` (cap/trim, off-main writer) | `swift test` (in-memory: append/recent/cap; `DiskAuditLog` via temp dir: round-trip, trim, `AuditError` mapping) | -| `AI/Audit/AuditError.swift` | Core | `AuditError` (`LocalizedError`) + `AIError.message(for:)` routing case | `swift test` (each case → clean headline; no raw OS text) | -| `App/AppSettings.swift` (modify) | Core | persisted `agentWhitelistPaths` / `agentWhitelistCommands` keys + defaults + reset inclusion | `swift test` (default empty; persistence; reset preserves like other opt-ins) | -| `Hub/HubAIPage.swift` (modify) or `Hub/HubBackgroundAutonomySection.swift` | App | the whitelist editor (add/remove paths via `NSOpenPanel` folder pick; add/remove command patterns) + the audit log viewer (recent list, "while you were away") | `xcodebuild` compile-verify; **user run-verifies** the editor + viewer | -| `AppCoordinator`/route-loop host wiring (modify) | App | inject `BackgroundPolicyResolver` (replacing `DescriptorWritePolicy`), compute `PolicyTarget`, call `BackgroundGate.decide`, wire `escalate` + `AuditLog.record` around `registry.run` | `xcodebuild` compile-verify; **user run-verifies** the parked auto/escalate behavior | -| `Tests/.../BackgroundAutonomyTests.swift` etc. | Core (test) | resolution table, whitelist matching, gate table, redaction, audit ring/disk | `swift test` | - -No file in this slice links MLX. The resolver/whitelist/gate/audit are pure value types + protocols; tests drive them directly with fabricated `ToolDescriptor`s and `RoutedCall`s. **No `.app` build, no signing, no permission change** — the slice never touches the build/sign path. To compile-check in isolation without sibling slices' uncommitted files, use a throwaway `git worktree` + `swift build`. - -## Edge cases - -- **Empty whitelist (default):** every `external`/`.confirm` step stays `.confirm`; only CONTAINED tools are auto. A fresh install runs nothing dangerous in the background and confirms every external write — the safe default. The user opts into autonomy by adding entries. -- **Whitelisted path vs `.dangerous` delete inside it:** a delete/overwrite tool ships `.dangerous`; the whitelist match is **ignored** (Decision 1) — it still escalates. A trusted folder never makes destruction auto. -- **`..` / symlink escape in a path target:** standardized (`standardizedFileURL`/`resolvingSymlinksInPath`) BEFORE the prefix match, so `/trusted/../etc/passwd` does NOT match `/trusted`. Unit-tested. -- **Shell command writing to an un-trusted path:** the both-rule requires command-pattern AND path-prefix; a whitelisted command (`git`) aimed at an off-list path stays `.confirm`/escalates. -- **A `.auto` descriptor that is NOT in the CONTAINED set:** defensive — treated as `.external` for radius (so an accidental `.auto` on a non-contained tool is not silently trusted); a debug assertion flags the misconfiguration. The CONTAINED set is the authority. -- **Audit store write fails while parked:** the in-memory ring still records the step; the disk write failure surfaces as a bounded banner on the Hub viewer (next reveal), never throws into the loop, never loses the running step. Auditing failing must not stop the agent. -- **Session goes from parked → active mid-step (user pulls it back during an `.auto` background run):** the `.auto` step keeps running (it was already safe); `wasBackground` is set from the state **at decision time**, so the record honestly says it started in the background. -- **`.needsYou` already set, another dangerous step queues:** `BackgroundGate` returns `.waitParked` (no double-escalate); the badge count increments via the rail's own `badgeCount` (owned by `ai-parked-sessions`), the glow stays lit. The audit log still records each `.awaitingApproval`. -- **Escalation reason references an error:** routed through `AIError.message(for:)` so the `needsYou` reason is always a clean headline, never raw text. -- **Per-skill `.auto` Claude handoff:** `launch_claude` ships `.confirm`; a skill opting `.auto` makes its descriptor `.auto` (owned by `ai-claude-handoff`), which this slice then treats as CONTAINED-style auto for the gate **but still audits** and is budget-capped (the cap is the handoff slice's). The whitelist does not gate the handoff (no path); the skill opt-in is the trust mechanism there. - -## Rejected alternatives - -1. **Defining `WritePolicyTier` here.** Rejected per integration fix **C1**: the bare enum lives with `ToolDescriptor` in `ai-tool-routing` (the earlier wave) so every descriptor is self-describing with no DAG back-edge. This slice owns resolution/whitelist/audit/escalation only. -2. **Extending the `WritePolicyResolving` protocol to take a target.** Rejected: it would force a routing-slice API change for a Wave-4 need. Instead this slice satisfies the descriptor-only protocol AND adds a richer overload the loop calls when it has the `RoutedCall` — zero routing change. -3. **A fourth `WritePolicyTier` case for WHITELISTED.** Rejected: WHITELISTED is not a *descriptor* property, it is a *resolution outcome* (a `.confirm` lowered to `.auto` by a match). Adding a case would leak resolution state into the descriptor and every consumer's switch. The blast-radius model is a *view*, the tier stays three cases. -4. **The whitelist lowering `.dangerous`.** Rejected as a security hole: a trusted folder does not make `rm`/overwrite safe. The whitelist only lowers `.confirm` → `.auto`; danger is intrinsic to the operation. -5. **Auditing only background actions.** Rejected: the ledger is "what did my agents do" — foreground auto-runs and skips are part of the story. Every step writes one record; `wasBackground` distinguishes them. -6. **A throwing `AuditLog.record`.** Rejected: auditing must never break the agent. `record` enqueues to an in-memory ring + off-main writer; a persistence failure is observed bounded on the viewer, the step is unaffected. -7. **A separate `BackgroundError` for escalation.** Rejected: escalation produces an observable `.needsYou` + a clean reason string, never an error; the only failure this slice can have is the store write, carried by `AuditError`. Honors "at most one `Error`." -8. **Rendering the needs-you badge/glow here.** Rejected: `ai-parked-sessions` owns the rail/badge/glow; this slice calls `escalate` and renders nothing. One owner per surface. - -## Target split & verification (per component) - -| Component | Target | Verified by | -|---|---|---| -| `BlastRadius` + classifier | Core | `swift test` (tool→radius map; `.dangerous` unconditional; CONTAINED set) | -| `Whitelist` + matching | Core | `swift test` (component-boundary prefix; standardized `..`; glob; both-rule) | -| `BackgroundPolicyResolver` (`WritePolicyResolving`) | Core | `swift test` (full resolution table; `.dangerous` never lowered) | -| `BackgroundGate.decide` | Core | `swift test` (full decision table; `.needsYou` no double-escalate) | -| `AuditRecord` + redaction | Core | `swift test` (Codable; redaction truncates/strips; headline-only failed) | -| `AuditLog` (in-memory + disk) | Core (IO off-main) | `swift test` (ring cap/recent; disk round-trip/trim/`AuditError`) | -| `AuditError` + `AIError` routing | Core | `swift test` (each case → clean headline) | -| `AppSettings` whitelist keys | Core | `swift test` (defaults/persistence/reset) | -| Hub Background-autonomy section (editor + viewer) | App | `xcodebuild` compile-verify; **user run-verifies** | -| Route-loop host wiring (resolver/gate/escalate/record) | App | `xcodebuild` compile-verify; **user run-verifies** parked auto/escalate | - -Per the house rule, an agent **never** builds/signs/installs the `.app` (ad-hoc signing breaks TCC). Pure Core is `swift build`/`swift test`; the App-target wiring + Hub views are `xcodebuild` compile-verify only; behavior (a parked agent auto-running a whitelisted write, a dangerous step lighting the glow, the audit viewer) is the user's run-verify in a stable-signed build. - -## Open Questions - -- **Q1 (target extraction ownership):** the `PolicyTarget` for a routed call must be derived from the call's args. I propose the resolver supplies the matching and the gate, while the per-tool arg→target extraction lives next to each `ToolContributor` (it owns the args shape). Confirm `ai-tool-routing` is happy to pass the extracted `PolicyTarget` into `effectiveTier(for:target:)` at the call site (no protocol change, an additive overload). -- **Q2 (audit retention cap):** 500 recent records is a guess for the "while you were away" ledger. Confirm a count or a time window (e.g. last 7 days) — a small, tunable policy. -- **Q3 (whitelist command-pattern surface):** I scope command patterns to the tool/Shortcut name and a shell command's `argv[0]` (anchored glob). A full shell-command-line pattern is rejected as too sharp an edge to get right safely. Confirm `argv[0]`-only is the right grain for v1. -- **Q4 (escalation-resolution audit follow-up):** an escalated step writes `.awaitingApproval` then a follow-up record on resolution. Confirm two records is preferred over mutating one (append-only argues for two; a viewer collapses them by `id`-lineage). I default to two append-only records. diff --git a/openspec/changes/ai-background-autonomy/proposal.md b/openspec/changes/ai-background-autonomy/proposal.md deleted file mode 100644 index 4644a90..0000000 --- a/openspec/changes/ai-background-autonomy/proposal.md +++ /dev/null @@ -1,33 +0,0 @@ -## Why - -V2 lets an agent keep working while it is **parked** at the notch — running tool steps in the background, with no one watching the canvas. That is only safe if there is a crisp, user-visible **trust boundary**: a parked agent may quietly write to its own memory and to folders/commands the user explicitly trusts, but it must **never** silently delete a file, overwrite an existing one, shell out off-list, or spend a Claude-handoff call without the user's eyes on it. The user decision (adopted, not relitigated): background memory writes + whitelisted writes are **AUTO even when parked** (still audited); **dangerous** writes escalate to the foreground via the needs-you badge. - -The route loop already carries the raw signal for this. `ai-tool-routing` defines (integration fix **C1**) a bare `WritePolicyTier` enum **on every `ToolDescriptor`**, and gates each step through an injected `WritePolicyResolving` seam with a stand-alone `DescriptorWritePolicy` default. What is missing is the **policy layer**: the user-editable **whitelist** that turns a `.confirm` descriptor into an effective `.auto` for trusted paths/commands, the **effective-tier resolution** (`descriptor.writePolicy ∩ whitelist`) wired into that seam, the **append-only audit log** that records every background action ("what did my agents do while I was away"), and the **escalation** that turns a dangerous-while-parked step into a `ParkState.needsYou` badge + the ambient notch glow. - -This slice is the accountability that earns the autonomy. It owns no UI gesture and no recognizer state; it is a pure Core policy + ledger that the routing loop reads, the parked-sessions rail renders, and memory/handoff consume. - -## What Changes - -- **A blast-radius tier model, mapped onto every sink.** Three tiers — **CONTAINED** (the app's own memory / project stores → AUTO, even parked), **WHITELISTED** (user-listed paths + command patterns → AUTO if matched), **DANGEROUS** (delete, overwrite-existing, shell, anything off-list, the Claude-handoff cost → FOREGROUND-ONLY: park + needs-you badge + notch glow, resolved by pulling the session back and DOWN=approve). Every existing and planned tool/sink is mapped to a default tier (the `writePolicy` it ships with on its `ToolDescriptor`). -- **A user-visible, user-editable whitelist** (a new Hub page on the AI feature). Default-empty for arbitrary paths and commands; the app's **memory store and project-note store are pre-trusted** (CONTAINED, not whitelist entries). The whitelist is the security boundary: a path-prefix list + a command-pattern (glob) list, with explicit matching rules. -- **Effective-tier resolution wired into the routing seam.** This slice ships the concrete `WritePolicyResolving` conformer that the loop already injects: `effectiveTier(for:) = descriptor.writePolicy` lowered to `.auto` **only** when (a) the descriptor is CONTAINED, or (b) its argument target matches a whitelist entry; otherwise the descriptor's own tier stands (a `.dangerous` descriptor is never lowered by the whitelist). -- **An append-only audit log.** Every background tool step writes one `AuditRecord` (tool, redacted args summary, effective tier, outcome, `wasBackground`, timestamp). The log is viewable in the notch rail drop-down and on the Hub AI page: a reverse-chronological "while you were away" ledger. Persisted, capped, never blocks. -- **The gating decision.** A pure `BackgroundGate` decides, per step, **auto** vs **escalate**: a parked session running a step whose effective tier is `.auto` runs and is audited; `.confirm` while parked waits as `parked` (resolved on restore via DOWN=approve / RIGHT=skip, owned by the routing approval gate); `.dangerous` while parked calls `ParkScheduler.escalate(_:reason:)` → `.needsYou` + badge + glow (owned by `ai-parked-sessions`). When the session is **active** (foreground), the existing approval gate already handles it — this slice only changes the *parked* path. - -## Capabilities - -### New Capabilities - -- `ai-background-autonomy`: the write-policy **resolution** (whitelist ∩ descriptor → effective tier; CONTAINED/WHITELISTED/DANGEROUS blast-radius model), the user-editable **whitelist** (path prefixes + command patterns, default-empty, memory/project pre-trusted), the append-only **audit log** (`AuditRecord`/`AuditLog`), and the parked **auto-vs-escalate gate** (`.auto` runs + audits, `.dangerous` escalates to `.needsYou`). - -### Modified Capabilities - -- `configuration-hub`: the AI feature page gains a **Background autonomy** section — the whitelist editor (add/remove trusted folders and command patterns) and the **audit log viewer** ("what your agents did while you were away"), both persisted and using the shared Liquid Glass presentation. - -## Impact - -- **Code (all MLX-free Core unless noted):** new `AI/Audit/WritePolicyTier+Resolution.swift` (the `WritePolicyResolving` conformer + blast-radius classification — the **enum itself is owned by `ai-tool-routing`**, consumed here verbatim), `AI/Audit/Whitelist.swift` (the `Whitelist` model + matching rules), `AI/Audit/AuditLog.swift` (the `AuditRecord`/`AuditLog` protocol + an append-only persisted `DiskAuditLog`, off-main IO bridged like `ParkedSessionStore`), `AI/Audit/BackgroundGate.swift` (the pure auto-vs-escalate decision), `AI/Audit/AuditError.swift` (a single `AuditError` only where `RuntimeError`/`TaskError`/`ParkError` cannot carry a store failure). New `AppSettings` persisted keys for the whitelist. The Hub **Background autonomy** section (`Hub/`) is App-target view code (`xcodebuild` compile-verify; the user run-verifies). -- **Consumes verbatim (never redefines):** `WritePolicyTier`, `ToolDescriptor`, `ToolStepResult`/`ToolStepStatus`, `WritePolicyResolving` (the seam this slice fills) from `ai-tool-routing`; `AgentSessionID` from `ai-conversation-runtime`; `ParkScheduler.escalate` / `ParkState.needsYou` from `ai-parked-sessions`; `AIError.message(for:)`/`AIPresentedError` from the existing `AIError.swift`. -- **MLX-free Core:** the tier model, whitelist matching, gate, audit record, and the in-memory audit log are pure and `swift test`-able; the persisted store's disk IO is bridged off-main; the Hub views need the real app (compile-verify via `xcodebuild`, run-verify by the user). -- **No signing, no permission, no TCC interaction.** The whitelist reads no filesystem at edit time (it stores path strings); the audit log writes only under the app's existing Application Support dir. The slice never touches the build/sign path. -- **Out of scope:** the `WritePolicyTier` enum definition (owned by `ai-tool-routing`); the per-step approval gesture (DOWN=approve / RIGHT=skip, owned by `ai-tool-routing`/`ai-conversational-canvas`); the needs-you badge + notch glow **rendering** (owned by `ai-parked-sessions` — this slice only calls `escalate`); the Claude-handoff budget/rate cap (owned by `ai-claude-handoff` — it consumes `AuditRecord` and the `.confirm`/`.auto` resolution here); rollback/undo of a completed side effect (explicitly impossible, per `ai-parked-sessions`). diff --git a/openspec/changes/ai-background-autonomy/specs/ai-background-autonomy/spec.md b/openspec/changes/ai-background-autonomy/specs/ai-background-autonomy/spec.md deleted file mode 100644 index 3cdbfb4..0000000 --- a/openspec/changes/ai-background-autonomy/specs/ai-background-autonomy/spec.md +++ /dev/null @@ -1,100 +0,0 @@ -## ADDED Requirements - -### Requirement: Blast-radius tiers classify every tool's background autonomy -The system SHALL classify every agent tool/sink into one of three **blast-radius tiers**, collapsing onto the existing write-policy tiers (`auto`/`confirm`/`dangerous`, owned by the tool-routing capability): - -- **CONTAINED** — the app's own stores (agent memory, project notes) and read-only retrieval. These tools SHALL ship `auto` and SHALL run in the background **even when the session is parked** (still audited). -- **WHITELISTED** — a `confirm` tool whose argument **target** (a path under a trusted prefix, or a command matching a trusted pattern) matches the user whitelist. These SHALL resolve to `auto` **only on a match** (still audited). -- **DANGEROUS** — delete, overwrite an existing file, arbitrary shell, anything off-list, and the Claude-handoff cost. These SHALL ship `dangerous` and SHALL be **foreground-only**: a parked session encountering one SHALL escalate to the needs-you state rather than act in the background. - -The whitelist SHALL be able to **lower `confirm` to `auto`** on a match but SHALL **never** lower `dangerous` — a destructive operation is dangerous regardless of where it lands. CONTAINED-ness SHALL be intrinsic to the tool (its shipped `auto` tier), not a whitelist entry, so the app's own stores are auto with an empty whitelist. - -#### Scenario: A contained store write runs in the background while parked -- **WHEN** a parked agent runs an agent-memory write or a save-to-project append (CONTAINED) -- **THEN** it executes without confirmation in the background and is recorded in the audit log - -#### Scenario: A whitelisted write runs auto; an off-list one does not -- **WHEN** a parked agent runs a `confirm` write whose target matches a whitelist entry -- **THEN** the effective tier is lowered to `auto` and it runs in the background (audited) -- **AND WHEN** the target matches no whitelist entry, the effective tier stays `confirm` and the step does not act in the background - -#### Scenario: A dangerous operation is never lowered by the whitelist -- **WHEN** a delete / overwrite-existing / arbitrary-shell tool would write inside a whitelisted folder -- **THEN** its tier remains `dangerous` and it is treated as foreground-only, not auto - -### Requirement: A user-visible, user-editable whitelist is the security boundary -The system SHALL provide a **whitelist** the user can view and edit: a list of **trusted folder path prefixes** and a list of **trusted command patterns**. The whitelist SHALL **default empty** for arbitrary entries — a fresh install SHALL trust nothing on the wider filesystem — while the app's **memory store and project-note store SHALL be pre-trusted as CONTAINED** (auto without any whitelist row). Whitelist edits SHALL persist with the same keys/defaults/reset semantics as the other AI opt-ins. - -A **path target** SHALL match the whitelist iff its standardized absolute path (with `..`/symlinks resolved before matching) has a trusted prefix at a **path-component boundary**. A **command target** SHALL match iff it matches a trusted pattern as an anchored glob. A target that is both a command and a path SHALL require **both** a command-pattern match and a path-prefix match (the stricter rule wins). - -#### Scenario: Default-empty trusts nothing arbitrary -- **WHEN** the whitelist is at its default -- **THEN** no arbitrary path or command is trusted, and only the CONTAINED memory/project stores run auto - -#### Scenario: Path matching respects component boundaries and resolves escapes -- **WHEN** a write targets a path that, once standardized, lies under a trusted prefix at a component boundary -- **THEN** it matches the whitelist; a path that only shares a string prefix (e.g. `/Notes2` vs trusted `/Notes`) or that escapes via `..`/symlinks does NOT match - -#### Scenario: A command and a path must both be trusted -- **WHEN** a whitelisted command is aimed at a path outside every trusted prefix -- **THEN** the target does not match the whitelist and the step stays `confirm` - -#### Scenario: Whitelist edits persist -- **WHEN** the user adds or removes a trusted folder or command pattern -- **THEN** the change persists across launches and is preserved by a reset-to-defaults like the other AI opt-ins - -### Requirement: Effective-tier resolution wired into the tool-routing seam -The system SHALL provide the concrete write-policy resolver that the tool-routing loop already injects (the `WritePolicyResolving` seam), replacing the stand-alone descriptor-identity default. The resolver SHALL compute a step's **effective tier** as: the descriptor's own tier, lowered to `auto` **only** when the tool is CONTAINED or its target matches the whitelist, and **never** lowered when the descriptor is `dangerous`. The resolver SHALL satisfy the existing descriptor-only seam method AND offer a target-aware resolution for the routed call's argument target, **without changing the routing protocol**. - -#### Scenario: Resolution lowers a matched confirm to auto -- **WHEN** the resolver evaluates a `confirm` external tool whose routed target matches the whitelist -- **THEN** the effective tier is `auto` - -#### Scenario: Resolution leaves an unmatched confirm and never lowers dangerous -- **WHEN** the resolver evaluates a `confirm` tool with no matching target, or any `dangerous` tool -- **THEN** the effective tier is `confirm` (unmatched) or `dangerous` (unconditional), respectively - -### Requirement: Append-only audit log of background actions -The system SHALL keep an **append-only audit log** in which **every** agent tool step writes one record — auto, confirmed, declined, escalated, or failed. Each record SHALL carry the session id, the tool name, the **effective tier** the step ran at, a **redacted, short arguments summary** (never raw secrets or full bodies), the outcome (reusing the tool-step status, a failure carrying a clean headline only), a `wasBackground` flag, and a timestamp. The log SHALL be **viewable** both in the notch rail drop-down and on the Hub AI page as a reverse-chronological "what did my agents do while I was away" ledger. Writing a record SHALL be **non-blocking** and SHALL NOT throw into the agent loop; a persistence failure SHALL be surfaced bounded and non-blocking, never as an app-modal alert, and SHALL NOT lose the in-memory record or stop the running step. - -#### Scenario: Every background action is recorded -- **WHEN** a parked agent runs, declines, escalates, or fails a tool step -- **THEN** one record is appended with the tool, effective tier, redacted args, outcome, `wasBackground=true`, and a timestamp - -#### Scenario: Arguments are redacted, failures carry a clean headline -- **WHEN** a step's arguments contain a long body or an embedded secret, or the step fails -- **THEN** the record's arguments summary is short and redacted (no raw secret), and a failure stores only the clean headline, not raw OS error text - -#### Scenario: The ledger is viewable in the notch and the Hub -- **WHEN** the user opens the notch rail drop-down or the Hub AI page after the agent worked in the background -- **THEN** the recent records appear in reverse-chronological order, distinguishing background actions from foreground ones - -#### Scenario: An audit persistence failure does not break the agent -- **WHEN** persisting the audit log to disk fails -- **THEN** the running step is unaffected, the record stays in the in-memory log, and the failure is shown as a bounded non-blocking banner (never an app-modal alert) - -### Requirement: A parked agent decides auto vs escalate per step -The system SHALL decide, per tool step, whether a **parked** session acts automatically or escalates, from the step's **effective tier** and the session's park state: - -- an `auto` effective tier SHALL **run in the background** and be audited; -- a `confirm` effective tier on a parked session SHALL **wait parked** (resolved later when the session is brought back, via the routing approval gesture DOWN=approve / RIGHT=skip) without escalating; -- a `dangerous` effective tier on a parked session SHALL **escalate to the foreground** by raising the needs-you state (via the parked-sessions scheduler), which surfaces the needs-you badge and the ambient notch glow (rendered by the parked-sessions capability), with a clean one-line reason; -- when the session is **active** (foreground), this decision SHALL defer to the existing routing approval gate (no escalation, no glow). - -An escalation SHALL be resolved by the user pulling the session back and approving (DOWN), at which point the step proceeds as a foreground step; a skip (RIGHT) SHALL decline it. A session already in needs-you SHALL NOT be double-escalated. A completed side effect SHALL NOT be rolled back — the audit log records it; it is not reversed. - -#### Scenario: Auto step runs in the background -- **WHEN** a parked session reaches a step whose effective tier is `auto` -- **THEN** the step runs in the background and is audited, without user attention - -#### Scenario: Confirm step waits parked without escalating -- **WHEN** a parked session reaches a step whose effective tier is `confirm` -- **THEN** the step waits in the parked session and is resolved when the user brings the session back, without lighting the needs-you glow - -#### Scenario: Dangerous step escalates to needs-you -- **WHEN** a parked session reaches a step whose effective tier is `dangerous` -- **THEN** the session escalates to the needs-you state with a clean reason, the needs-you badge and ambient notch glow appear, and the step waits until the user pulls the session back and approves it - -#### Scenario: Already-escalated session is not double-escalated -- **WHEN** a session is already in needs-you and another dangerous step arrives -- **THEN** the session is not escalated again (the badge count and glow already reflect the pending attention) and the step is recorded as awaiting approval diff --git a/openspec/changes/ai-background-autonomy/specs/configuration-hub/spec.md b/openspec/changes/ai-background-autonomy/specs/configuration-hub/spec.md deleted file mode 100644 index 982617b..0000000 --- a/openspec/changes/ai-background-autonomy/specs/configuration-hub/spec.md +++ /dev/null @@ -1,29 +0,0 @@ -## ADDED Requirements - -### Requirement: The AI page hosts the Background autonomy whitelist and audit log -The Hub's **AI** feature page SHALL provide a **Background autonomy** section with two surfaces: - -- a **whitelist editor** — add, remove, and review the trusted **folder path prefixes** (picked as **local folders only**) and the trusted **command patterns** that let a parked agent run a `confirm` write automatically. The editor SHALL default empty (a fresh install trusts nothing arbitrary), SHALL **persist** its values with the same keys/defaults/reset semantics as the other AI opt-ins, and SHALL state plainly that whitelisting a folder/command makes matching writes run in the background, and that dangerous operations (delete, overwrite-existing, arbitrary shell) are never made automatic by the whitelist. -- an **audit log viewer** — a reverse-chronological "what your agents did while you were away" ledger of recent background and foreground tool steps, each showing the tool, a redacted arguments summary, the effective tier, the outcome (a failure shown as a clean headline with an opt-in details disclosure), and a timestamp, distinguishing background actions from foreground ones. - -Both surfaces SHALL use the shared Liquid Glass presentation consistent with the rest of the Hub. A failure to persist or load the audit log SHALL surface as a **bounded, non-blocking** banner (clean headline, opt-in details), **never** an app-modal alert. - -#### Scenario: The Background autonomy section is reachable on the AI page -- **WHEN** the user opens the Hub and selects the AI feature page -- **THEN** a Background autonomy section shows the whitelist editor (trusted folders + command patterns) and the audit log viewer - -#### Scenario: Editing the whitelist persists and live-applies -- **WHEN** the user adds a trusted folder or command pattern and removes another -- **THEN** the change persists across launches, is preserved by a reset-to-defaults like the other AI opt-ins, and the agent's effective-tier resolution reflects it on the next step - -#### Scenario: Only local folders can be added as trusted prefixes -- **WHEN** the user adds a trusted folder -- **THEN** only a local folder is accepted (network / iCloud-placeholder locations are rejected) - -#### Scenario: The audit viewer reads the ledger -- **WHEN** the user opens the audit log viewer after the agent has worked -- **THEN** the recent tool steps appear in reverse-chronological order with their tool, redacted args, effective tier, outcome, and timestamp, marking which ran in the background - -#### Scenario: An audit store failure is non-blocking -- **WHEN** loading or persisting the audit log fails -- **THEN** the viewer shows a bounded banner with a clean headline (details behind an opt-in disclosure) and the rest of the page stays usable, with no app-modal alert diff --git a/openspec/changes/ai-background-autonomy/tasks.md b/openspec/changes/ai-background-autonomy/tasks.md deleted file mode 100644 index 68eb995..0000000 --- a/openspec/changes/ai-background-autonomy/tasks.md +++ /dev/null @@ -1,52 +0,0 @@ -> Decomposed for a workflow fan-out: §1 is the pure type substrate (do first), §2–§5 are the policy / whitelist / gate / audit (each independent once §1 lands), §6 is the cross-slice wiring, §7 verifies. Every item is MLX-free Core verified by `swift test` unless noted; this slice OWNS no UI gesture and **never builds/signs the `.app`**. `WritePolicyTier`/`ToolDescriptor`/`ToolStepStatus`/`WritePolicyResolving` are CONSUMED verbatim from `ai-tool-routing`; `AgentSessionID` from `ai-conversation-runtime`; `ParkScheduler.escalate`/`ParkState` from `ai-parked-sessions`. - -## 1. Blast-radius classification (pure Core type substrate) - -- [x] 1.1 Add `AI/Audit/BlastRadius.swift`: `BlastRadius` enum (`contained`/`external`/`dangerous`) + a pure classifier `BlastRadius.of(_ descriptor: ToolDescriptor) -> BlastRadius` reading the descriptor's `name` + `writePolicy`; the CONTAINED name-prefix set (`["memory.", "save_to_project", "retrieve", "widen_candidates"]`). `.dangerous` is unconditional; a `.auto` descriptor NOT in the set is defensively `.external` (debug-asserted). DO NOT redefine `WritePolicyTier`/`ToolDescriptor` (consume from `ai-tool-routing`). *Verify: `swift test` — every tool in the design's tool→tier map classifies to its expected radius; `.dangerous` ignores the name.* -- [x] 1.2 Doc-comment that `WritePolicyTier` is OWNED by `ai-tool-routing` (C1) and this slice owns only resolution/whitelist/audit/escalation. *Verify: review.* - -## 2. The user whitelist (pure, persisted) - -- [x] 2.1 Add `AI/Audit/Whitelist.swift`: `Whitelist{trustedPathPrefixes, trustedCommandPatterns}` (Codable/Equatable/Sendable) + `.empty`. *Verify: `swift test` — Codable round-trip; `.empty` is two empty arrays.* -- [x] 2.2 Implement the pure **path matching**: standardize (`standardizedFileURL`/`resolvingSymlinksInPath`) the target THEN component-boundary prefix match against `trustedPathPrefixes` (`/a/b` matches `/a/b/x`, not `/a/b2`). *Verify: `swift test` — component boundary; `..`/symlink resolved before match cannot escape a trusted prefix.* -- [x] 2.3 Implement the pure **command matching**: anchored `fnmatch`-style glob (`*`/`?`) against `trustedCommandPatterns`, matched on the tool/Shortcut name or a shell `argv[0]`. *Verify: `swift test` — glob anchored full-string; `git*` matches `git`, not `forgit`.* -- [x] 2.4 Implement the **both-rule** (a command writing to a path must match BOTH a command pattern and a path prefix; the stricter wins). *Verify: `swift test` — whitelisted command at an off-list path does NOT match.* -- [x] 2.5 Persist `agentWhitelistPaths` / `agentWhitelistCommands` in `App/AppSettings.swift` (default empty; included in the Hub AI reset semantics like other opt-ins). *Verify: `swift test` — defaults empty; persists; reset preserves like other opt-ins (no new default leak).* - -## 3. Effective-tier resolution (the `WritePolicyResolving` drop-in) - -- [x] 3.1 Add `AI/Audit/WritePolicyResolution.swift`: `PolicyTarget` (`.path`/`.command`/`.both`/`.none`) + `BackgroundPolicyResolver` conforming to the CONSUMED `WritePolicyResolving` (descriptor-only fast path: CONTAINED→`.auto`, `.dangerous`→`.dangerous`, else descriptor tier). *Verify: `swift test` — descriptor-only path returns expected tier without a target.* -- [x] 3.2 Add the richer overload `effectiveTier(for:target:)`: lower `.confirm`→`.auto` ONLY when `BlastRadius == .external` AND the target matches the whitelist; NEVER lower `.dangerous`; `.none` target is never lowerable. *Verify: `swift test` — the full resolution table from design Decision 4, including `.dangerous`+whitelisted-path stays `.dangerous`.* - -## 4. The background gate (auto-vs-escalate brain) - -- [x] 4.1 Add `AI/Audit/BackgroundGate.swift`: `BackgroundDecision` (`auto`/`waitParked`/`escalate(reason)`/`foreground`) + pure `BackgroundGate.decide(effectiveTier:parkState:)` consuming `ParkState` (from `ai-parked-sessions`). *Verify: `swift test` — the full decision table; `.needsYou` never double-escalates; `.dangerous`+`.active`→`.foreground` (no glow when already front).* - -## 5. The audit log (append-only, redacted, capped) - -- [x] 5.1 Add `AI/Audit/AuditRecord.swift`: `AuditRecord{id, sessionID, tool, policy, argumentsSummary, outcome, wasBackground, timestamp}` (Codable/Identifiable) consuming `AgentSessionID` + `ToolStepStatus` verbatim. *Verify: `swift test` — Codable round-trip; a `.failed` outcome stores a headline string only.* -- [x] 5.2 Implement the pure `argumentsSummary` **redaction** builder: bounded single-line summary (last path component(s), command name, middle-truncated content preview); strips raw secrets / full bodies. *Verify: `swift test` — long args truncate with `…`; a shell line with an embedded token does not appear raw; bounded length.* -- [x] 5.3 Add `AI/Audit/AuditLog.swift`: `AuditLog` protocol (`record`, `recent(limit:)`) + `InMemoryAuditLog` (a bounded ring). *Verify: `swift test` — append; `recent(limit:)` reverse-chronological; cap trims oldest.* -- [x] 5.4 Add `DiskAuditLog` (append-only JSON-lines under Application Support, mirroring `ClipboardStore`/`ParkedSessionStore`; cap/trim on write; non-blocking `record` via a serialized off-main writer + in-memory ring the viewers read synchronously). *Verify: `swift test` via a temp dir — round-trip; trim to cap; `record` never throws to the caller.* -- [x] 5.5 Add `AI/Audit/AuditError.swift`: `AuditError` (`LocalizedError`: `.persistFailed`/`.storeUnavailable`) mapped at the `DiskAuditLog` boundary from `FileManager`/JSON errors; add the `AIError.message(for:)` routing case. *Verify: `swift test` — each case → clean headline via `AIError.message(for:)`; no raw OS text; a store failure is observable, not thrown into a caller.* - -## 6. Cross-slice wiring (App / integration target — compile-verify + user run-verify) - -> WIRED (`wire-full-potential-gates`): the App-target host now exists. `AICommandExecutor` takes an optional `backgroundRunner: BackgroundToolRunner?` (default nil = the plain foreground path / every existing caller) threaded into the live `AgentLoop`, so each routed tool step runs under `BackgroundPolicyResolver` + `BackgroundGate.decide` + audit when present. `AppCoordinator` constructs the runner — `resolver: BackgroundPolicyResolver(whitelist: settings.agentWhitelist)`, `audit: auditLog`, `scheduler: parkController.parkScheduler`, `parkStateOf:` reading `parkController.parkState(of:)` (new accessor over the live `SerialParkScheduler` snapshot) — ONLY when `FullPotentialGate.isUnlocked(.backgroundAutonomy)` (master ∧ sub-flag ∧ ai-commands). LOCKED (default master OFF) → nil → calmly inert (no auto-run/escalation, no error, never a false "Done"). `swift test` (1444) + `xcodebuild` GREEN. - -- [x] 6.1 Replace the routing loop's injected `DescriptorWritePolicy` with `BackgroundPolicyResolver(whitelist:)` at the route-loop host; thread the user's `Whitelist` from `AppSettings`. *Verify: `xcodebuild` compile; **user run-verifies** a whitelisted write auto-runs while parked.* -- [x] 6.2 At the contributor/loop call site, extract the routed call's `PolicyTarget` (per-tool, owned next to each contributor) and call `effectiveTier(for:target:)`; then `BackgroundGate.decide(effectiveTier:parkState:)`. *Verify: `xcodebuild` compile; **user run-verifies** an off-list shell escalates.* -- [x] 6.3 Wire the decision outcomes around `registry.run`: `.auto`→run + `AuditLog.record(wasBackground: parked)`; `.escalate(reason)`→`scheduler.escalate(sessionID, reason:)` + record `.awaitingApproval` + suspend (resumes foreground via the routing `ApprovalGate`); `.waitParked`→leave paused (no glow); `.foreground`→no-op (existing gate). *Verify: `xcodebuild` compile; **user run-verifies** a dangerous parked step lights the needs-you badge/glow (rendered by `ai-parked-sessions`).* -- [x] 6.4 Write an `AuditRecord` for EVERY tool step (auto/confirmed/declined/escalated/failed), and a follow-up record on escalation resolution (approve→`.done`/`.failed`, skip→`.declined`). *Verify: `xcodebuild` compile; **user run-verifies** the ledger shows each action.* - -## 7. The Hub Background-autonomy section (App target — compile-verify + user run-verify) - -> DONE (implementation + `xcodebuild` compile-verify): App-target SwiftUI Hub views (whitelist editor + audit viewer) consuming the §1–§5 primitives + `AppSettings.agentWhitelist*` keys (already shipped). Shipped in `Hub/HubBackgroundAutonomy.swift`, sectioned onto `AIPage`, with the audit seam wired through `HubContext` to a shared `DiskAuditLog` in `AppCoordinator`. No `swift test` surface (pure UI); only the live feel (edits persist/apply, ledger reads, banner non-blocking) is the user's install-and-observe run-verify. - -- [x] 7.1 Add the **whitelist editor** to the Hub AI page (`Hub/`): add/remove trusted folders (folder pick via `NSOpenPanel`, local folders only) and add/remove command patterns; persists to `AppSettings`; shared Liquid Glass presentation. *Verify: `xcodebuild` compile; **user run-verifies** edits persist and live-apply.* — DONE: `Hub/HubBackgroundAutonomy.swift` `HubWhitelistEditor` (binds `$settings.agentWhitelistPaths`/`agentWhitelistCommands`; `NSOpenPanel` local-folder-only via `isLocalFolder` rejecting iCloud-placeholder + non-local volumes; glob command patterns), added as a section on `AIPage`. `xcodebuild` BUILD SUCCEEDED; only the live persist/apply feel is the user's install-and-observe. -- [x] 7.2 Add the **audit log viewer** to the Hub AI page + the notch rail drop-down: reverse-chronological recent records (tool, redacted args, effective tier, outcome, timestamp), distinguishing `wasBackground` ("while you were away"); a bounded non-blocking banner on a store-persist failure (headline only, "Show details" disclosure). NEVER `NSAlert.runModal`. *Verify: `xcodebuild` compile; **user run-verifies** the ledger reads correctly and the failure banner is non-blocking.* — DONE: `Hub/HubBackgroundAutonomy.swift` `HubAuditLogViewer`/`AuditRow`/`AuditPersistBanner` reading `HubContext.recentAuditRecords` (wired to a shared `DiskAuditLog` in `AppCoordinator`) reverse-chronological with tier badge + outcome + relative timestamp + "while you were away" tag + opt-in failure detail; the store-persist failure banner reads `DiskAuditLog.lastPersistError` routed through `AIError.message(for:)` (never `NSAlert`). The notch-rail drop-down is owned/rendered by `ai-parked-sessions` (same `recent(limit:)` seam); this slice ships the Hub surface. `xcodebuild` BUILD SUCCEEDED; the live read + non-blocking banner is the user's install-and-observe. - -## 8. Spec sync + cross-slice confirmation - -- [x] 8.1 Confirm the consumed contracts match the committed sibling definitions: `WritePolicyTier`/`ToolDescriptor`/`ToolStepStatus`/`WritePolicyResolving` (`ai-tool-routing`), `AgentSessionID` (`ai-conversation-runtime`), `ParkState`/`ParkScheduler.escalate` (`ai-parked-sessions`). *Verify: review — no redefinition; the additive `effectiveTier(for:target:)` overload needs no routing protocol change (Open Q1).* -- [x] 8.2 Update `openspec/specs/ai-background-autonomy/spec.md` (new capability) and the `configuration-hub` delta from this change's `specs/`. *Verify: `openspec` validate; review the delta is ADDED-only for the new capability and a true MODIFIED for `configuration-hub`.* — DEFERRED to archive: the change's `specs/` deltas are authored and pass `openspec validate ai-background-autonomy --strict`; publishing into the un-versioned `openspec/specs/` is the archive-time `opsx:sync` step (siblings keep their capability spec in the change until archive), not done while the change is open. diff --git a/openspec/changes/ai-batched-runtime-and-context/.openspec.yaml b/openspec/changes/ai-batched-runtime-and-context/.openspec.yaml deleted file mode 100644 index 38f7628..0000000 --- a/openspec/changes/ai-batched-runtime-and-context/.openspec.yaml +++ /dev/null @@ -1,2 +0,0 @@ -schema: spec-driven -created: 2026-06-22 diff --git a/openspec/changes/ai-batched-runtime-and-context/design.md b/openspec/changes/ai-batched-runtime-and-context/design.md deleted file mode 100644 index 35c5fc6..0000000 --- a/openspec/changes/ai-batched-runtime-and-context/design.md +++ /dev/null @@ -1,176 +0,0 @@ -## Context - -The on-device runtime today is a single `LLMRuntime` conformer (`GemmaMLXRuntime`, GemmaRuntime target) that serves **one** request at a time: `generate(_:)` runs a per-token decode loop on one resident graph. `ModelManager` (Core, `@MainActor`) owns weights/residency and injects the real runtime through an existing `ModelProvisioner` seam — `GemmaRuntime.makeModelManager` builds the provisioner that creates and `prepare`s a `GemmaMLXRuntime`. Feature code only ever sees `LLMRuntime` (design D1). - -V2 needs **concurrency** (foreground session + parked background sessions advancing) and **long context** (skills + memory TOC + a long thread). Two facts drive the architecture: - -- **Decode is memory-bandwidth-bound.** One decode step reads the whole ~17 GB weight set from unified memory to emit *one* token per stream. The 16 GPU cores are mostly idle waiting on that read. K independent `generate()` calls each re-read the weights → K× bandwidth, no win. **Continuous batching** reads the weights **once** and advances K streams together — the cores do K× the arithmetic against the one weight read. This is the only correct concurrency primitive here. -- **RAM is the concurrency ceiling.** 48 GB unified = weights (once, ~17 GB at 4-bit) + **K** KV caches. The KV cache is per-stream and grows linearly with context length. So K is **derived from free RAM at a given context length**, not a constant — and growing context shrinks K. This is why context tuning and concurrency are one slice. - -`ai-conversation-runtime` already landed the seam shapes this slice plugs into: `LLMChatRequest` + a default-flattened `LLMRuntime.chat()` (blueprint §3.2), the `AgentMessage`/`AgentSessionID`/`AgentConversation`/`AgentTurn` types, and — crucially — its compaction reads the token budget through an **injected `ContextBudgetProviding`** (integration fix C3), explicitly so it does not depend on this slice landing first. This slice ships the **concrete** provider (backed by `ModelDescriptor.maxContextTokens` ∩ the user's `agentContextTokens`) and the batched conformer that **overrides** `chat()` for true KV-reuse multi-turn. `ai-parked-sessions` owns `ParkScheduler`; this slice **consumes** `runnableSessions(now:maxSlots:)` to fill batch slots and reports back via `didAdvance`. - -## Goals / Non-Goals - -**Goals:** -- A `BatchedLLMRuntime` conformer that multiplexes K sessions over one weight read per decode step, de-multiplexing tokens by `AgentSessionID`, with the foreground session always slotted and the rest fed by `ParkScheduler`. -- A pure, testable **memory budget** that derives K from weights + free RAM + per-token KV cost (a function of context length and KV-quant bits), accounting for Gemma's interleaved sliding-window/global attention. -- KV-cache quantization (8/4-bit), prefix/prompt caching of the shared system+skills prefix, and a rotating fixed-window KV option — all internal to the GemmaRuntime conformer. -- User-adjustable context (presets Balanced/Long/Max + compact-KV toggle), clamped to the model max, with the RAM/speed cost surfaced; a global default + per-skill override; the effective budget feeding conversation-runtime compaction through the injected provider. -- A `Subagent` fixed-pattern primitive: run a sub-task in a fresh context, return only a summary. - -**Non-Goals:** -- The conversation state machine / compaction *logic* (owned by `ai-conversation-runtime`; this slice only supplies the budget number through the provider). -- The parked-session durable store, the notch rail, and park/restore lifecycle (`ai-parked-sessions`); this slice only *consumes* `ParkScheduler`. -- The route→execute→continue loop and tool execution (`ai-tool-routing`). -- The skill file format (`ai-skills-as-files`); this slice reads only an optional per-skill context-override value off the resolved skill. -- Open-ended, model-driven recursive subagent spawning — the primitive is fixed-pattern and bounded on purpose (a small model orchestrates dynamic spawning poorly). -- Any Intel / low-end / non-Metal fallback or defensive degraded decode path. The target is M5/M4 only. -- A cloud/remote runtime, or video/audio batching. - -## Decisions - -### D1. `BatchedLLMRuntime` is a Core protocol; the conformer is GemmaRuntime. - -The **protocol** (blueprint §3.6) lives in MLX-free Core alongside `LLMRuntime` so feature code and the scheduler can reference `maxConcurrentStreams` and the batched entry point without linking MLX: - -``` -public protocol BatchedLLMRuntime: LLMRuntime { - func batchStep(_ requests: [AgentSessionID: LLMChatRequest]) - -> AsyncThrowingStream<(AgentSessionID, Token), Error> - var maxConcurrentStreams: Int { get } // K — derived from RAM (D4), not a constant -} -``` - -The **conformer** `BatchedGemmaMLXRuntime` (GemmaRuntime target, MLX-linked) implements it plus `chat()` (the single-session entry — it just runs a one-element batch) and the inherited `generate`/`structured`. It is the new resident runtime the provisioner returns; it subsumes `GemmaMLXRuntime`'s single path (vision, reasoning channels, the manual loop) and adds the batched decode loop. Verified by `xcodebuild` compile-only — an agent never builds/signs/installs the `.app`. - -- **`maxConcurrentStreams` is RAM-derived, not constant.** It reads `ConcurrencyBudget` (D4) against the current `agentContextTokens` and KV-quant bits, so growing context lowers K honestly. It is recomputed when the context setting changes or memory pressure is reported. - -### D2. Continuous batching: one weight read, K streams, per-stream de-mux. - -The decode loop (inside the conformer) is the classic continuous-batching loop: - -1. **Slot assignment.** The foreground active session (if any) takes slot 0 unconditionally. Remaining `maxConcurrentStreams - 1` slots are filled from `ParkScheduler.runnableSessions(now:maxSlots:)`. New requests can join **mid-flight** (continuous, not static batching): a stream that finishes (`isFinal`) frees its slot for the next runnable session on the very next step — the weights stay resident, no reload. -2. **Per-step forward pass.** Each active stream contributes its next-token query; the K queries are stacked into one batched forward pass that reads the weights **once**. Padding/masking handles unequal sequence lengths; a per-stream KV cache (D3) holds each stream's keys/values. -3. **De-mux.** The pass yields one logit row per stream → one sampled `Token` per stream, emitted as `(AgentSessionID, Token)` on the merged stream. Channel classification (`.thinking`/`.response`, the existing `ChannelClassifier` idiom) is per-stream. -4. **Prefill vs decode.** A newly admitted stream first runs a **prefill** (its full prompt) to populate its KV cache; thereafter it joins the single-token decode batch. Prefill of a new stream and decode of running streams are interleaved so admitting a session never stalls the others for long (chunked prefill, bounded chunk size). -5. **Feedback.** On each stream's `isFinal`, the conformer calls `ParkScheduler.didAdvance(id, result:)` so the scheduler can re-rank; a stream error maps to `RuntimeError` and is emitted as that stream's terminal token-stream error (the other streams are unaffected — failure is per-stream, never a batch-wide abort). - -`chat()` for a single foreground turn is just `batchStep([id: request])` with K=1 — the same code path, so the foreground multi-turn case reuses the KV cache across turns (the win conversation-runtime wants). - -### D3. KV cache: per-stream, quantized, prefix-cached, rotating-window option. - -The KV cache is the per-stream memory cost and the long-context lever. Three mechanisms, all internal to the conformer: - -- **KV quantization (8/4-bit).** Keys/values are stored quantized (MLX `quantized_kv` / quantized-cache idiom). 8-bit ≈ halves KV bytes vs bf16 with negligible quality loss; 4-bit quarters it for the longest threads. The **compact-KV toggle** (D6) selects 8-bit; "Max" context may force it. Per-token KV bytes feed `ConcurrencyBudget` (D4). -- **Prefix / prompt caching.** The shared **system + skills prefix** (the combined TOC of `allSummaries()` from `ai-skills-as-files`/`ai-agent-memory`, plus the system preamble) is identical across turns and often across *sessions*. Its KV is computed **once** and reused: a turn's prefill skips the cached prefix and only prefills the turn-specific suffix. This is the single biggest per-turn latency win for long static prefixes; it is keyed by a hash of the prefix text so a changed prefix invalidates cleanly. -- **Rotating fixed-window KV (optional).** For unbounded background threads, a fixed-window (ring-buffer) KV caps the cache at the last W tokens — bounded memory regardless of how long a parked session runs. This is a *runtime* cap distinct from conversation-runtime's *compaction* (which rewrites history): the window bounds the GPU cache; compaction bounds the re-fed token count. They compose — compaction keeps the assembled request small; the rotating window is the backstop so a runaway background thread can never blow the KV budget. -- **Gemma interleaved attention accounting.** Gemma 4 interleaves **local sliding-window** layers (most layers; their KV is naturally bounded by the window) with a few **global** layers (full-context KV). So per-token KV bytes are **not** uniform across layers: the budget math (D4) sums `(num_sliding_layers × min(ctx, window) + num_global_layers × ctx) × kvBytesPerTokenPerLayer`. Treating every layer as global would massively over-estimate KV and under-provision K; treating every layer as local would under-estimate and risk OOM. The conformer exposes the layer split to the budget model. - -### D4. `ConcurrencyBudget` — pure, RAM-is-the-ceiling math (Core, testable). - -A pure value model (no MLX) that the conformer queries for K and the Hub queries for the cost surface: - -``` -struct KVCacheCost { // pure - var slidingLayers: Int - var globalLayers: Int - var slidingWindow: Int // tokens - var kvBytesPerTokenPerLayer: Double // a function of head dim × kv-quant bits - func kvBytes(forContext ctx: Int) -> Int64 // the interleaved-attention sum (D3) -} - -struct ConcurrencyBudget { // pure - var unifiedMemoryBytes: Int64 // total (probed at the boundary; injected here) - var weightBytes: Int64 // resident weights, read once - var reservedBytes: Int64 // OS + app + graph activations headroom - var kv: KVCacheCost - /// How many concurrent streams fit at this context length, given KV-quant bits. - func maxStreams(contextTokens: Int) -> Int // floor((free - weights) / kvBytes(ctx)), clamped ≥1 - /// Estimated resident RAM at a given (streams, context) for the Hub cost surface. - func estimatedRAM(streams: Int, contextTokens: Int) -> Int64 -} -``` - -- **Foreground guarantee.** `maxStreams` is clamped to **≥ 1** so the foreground session always fits even if a chosen context is so large only one stream is affordable (then K=1 — background sessions wait, no OOM). This is the "RAM is the ceiling" honesty: growing context trades concurrency for length, visibly. -- Real memory probing (free unified bytes) happens at the GemmaRuntime boundary and is **injected** into the pure budget, so the math is unit-testable with fixed inputs (no Metal in `swift test`). - -### D5. Context tuning: `maxContextTokens` + user-adjustable `agentContextTokens`, cost surfaced (integration fix C3). - -- **`ModelDescriptor.maxContextTokens: Int`** is added (Core) — the model's architectural max (Gemma 4's large context). It is the clamp ceiling. -- **`agentContextTokens`** (persisted, Core `AppSettings`) is the user's chosen budget, **clamped to `maxContextTokens`**. It is set via three **presets** — **Balanced** (a comfortable mid value, the default), **Long**, **Max** (= the model max) — plus a **"compact long contexts (8-bit KV)"** toggle (`agentCompactKV`) that selects 8-bit KV so a longer context fits in the same RAM. A `agentContextPreset` enum persists the chosen preset (Balanced/Long/Max/custom). -- **Cost surfaced (house requirement — never silent OOM).** The Hub's AI page shows, derived from `ConcurrencyBudget.estimatedRAM` + `maxStreams`: estimated **RAM** at the chosen context and the resulting **concurrent-stream count** (e.g. "Max context · ~38 GB · 1 background session" vs "Balanced · ~24 GB · 3 background sessions"), and a relative **speed** note (longer context = slower per token). This is the explicit RAM/speed-cost-surfaced decision (cross-cutting decision: context growable AND user-adjustable with cost surfaced). -- **Global default + per-skill override.** A heavy skill (e.g. a long-document summarizer) may declare a larger context need; the effective budget for a session is `max(globalDefault, skillOverride)` clamped to the model max. The per-skill override value rides on the skill file (`ai-skills-as-files`); this slice only **reads** it through a small `SkillContextOverriding` seam, never owning the skill format. -- **The budget feeds compaction (the tie-in).** This slice ships the concrete `ContextBudgetProviding` (the protocol is owned by `ai-conversation-runtime`, C3): `maxContextTokens` resolves to the effective `agentContextTokens` (∩ model max ∩ per-skill override). conversation-runtime's `needsCompaction`/`plan` read this number — so growing the context slider directly raises the compaction trigger, and the two never disagree about "the budget." - -### D6. The compact-KV toggle and the KV-quant selection. - -`agentCompactKV` ON → the conformer uses **8-bit** KV (D3); OFF → bf16 (or the model's native KV precision). "Max" preset may **force** 8-bit (and surface that it did) because the model-max context at bf16 KV would not leave room for even one stream + headroom. 4-bit KV is reserved for the rotating-window background path where the longest threads run; it is not exposed as a user toggle (a feel-only internal choice for background streams), keeping the user surface to one comprehensible toggle. - -### D7. `Subagent` — a fixed-pattern context-hygiene primitive (NOT concurrency, NOT dynamic spawning). - -A subagent is **context hygiene**, not parallelism: it runs a bounded sub-task in a **fresh** `AgentConversation` (empty history, its own skill/prompt) and returns only a **summary** `AgentMessage` to the orchestrator, so the orchestrator's context stays lean (it never absorbs the sub-task's intermediate turns). - -``` -struct Subagent { // pure orchestration, Core - let name: String // a named, fixed template (not model-invented) - let systemPrompt: String - let maxTurns: Int // bounded -} -struct SubagentResult { let summary: String; let sessionID: AgentSessionID } -``` - -- **Fixed-pattern, bounded.** The orchestrator invokes a **named** subagent (from a small registered set) with an input; the subagent runs ≤ `maxTurns` turns in its fresh session and returns a summary. There is **no** open-ended, model-decided recursive spawning (a small model handles that poorly — cross-cutting note). A subagent may itself be **a routed tool step** (`ai-tool-routing`), so "run subagent X" is a `ToolDescriptor`; the loop appends only the returned summary as a `.tool` message. -- **Runs on the batched runtime as just another session.** A subagent's fresh `AgentSessionID` can occupy a batch slot like any other stream — so subagents are *concurrency-cheap* (they ride the one weight read) while being *context-cheap* (their turns never pollute the orchestrator). It does not get its own weight read. -- Subagent results are summaries, never raw turn dumps — this is the whole point (orchestrator stays lean). - -### D8. Errors map at the GemmaRuntime boundary; failure is per-stream and observable. - -Vendor/MLX/OS errors (graph build, OOM at admit, a Metal abort) are mapped into `RuntimeError` **at the conformer boundary** — Core stays MLX-free, so only `RuntimeError` crosses out. Surfacing is bounded + non-blocking via `AIError.message(for:)`: a stream that fails emits a terminal error on **its** sub-stream only (its turn becomes `.failed` with a clean headline in conversation-runtime), and the **other batched streams keep running** (never a batch-wide abort, never a false "done" for the failed stream). An admit that cannot fit (OOM headroom) is **not** a failure of a running stream — the new session simply isn't admitted this step (it waits); `maxStreams ≥ 1` guarantees the foreground always fits. No `NSAlert`; raw MLX text goes only to logs / `AIPresentedError.details`. - -### D9. Plug-in via the existing `ModelProvisioner` — no `ModelManager` API change. - -`GemmaRuntime.makeModelManager`'s provisioner currently returns `GemmaMLXRuntime()`. It changes to return `BatchedGemmaMLXRuntime()` (which conforms to `LLMRuntime` *and* `BatchedLLMRuntime`). `ModelManager` is untouched — it stores the returned `LLMRuntime` resident exactly as today. Callers that need the batched entry point downcast `currentRuntime as? BatchedLLMRuntime` (the scheduler-driven background advancer does; the foreground `chat()` path does not need to — `chat()` is on the base protocol). This keeps Core's `ModelManager` MLX-free and the residency/lifecycle/registry behavior identical. **The metallib bundle landmine is unchanged** — `build-app.sh` still copies `*.bundle` into `Contents/Resources/`; the batched conformer hits the same Metal path, so no GPU use without the bundle (no regression introduced, none required). - -### Rejected alternatives - -- **Spawn K independent `generate()` tasks for concurrency.** Rejected — each re-reads the full weights from unified memory, so K tasks = K× the bandwidth on the *single* bottleneck; on a bandwidth-bound decode this is *slower* per token, not faster. Continuous batching (one read, K streams) is the only win. -- **A constant `maxConcurrentStreams`.** Rejected — KV grows with context and per stream, so a constant K either OOMs at long context or under-uses RAM at short context. K must be RAM-derived (`ConcurrencyBudget`). -- **Treat all attention layers as global for KV math (simpler).** Rejected — Gemma interleaves sliding-window (most) and global (few) layers; uniform-global over-estimates KV ~N× and needlessly throttles K. Per-layer-split math is required. -- **Expose raw KV-quant bits (bf16/8/4) as a user control.** Rejected — three precisions × context presets is an incomprehensible surface; one "compact long contexts (8-bit)" toggle is the user lever, 4-bit is an internal background choice. -- **Open-ended dynamic subagent spawning (model decides when/what to spawn, recursively).** Rejected — a small model orchestrates unbounded recursion poorly (runaway spawns, confused summaries); the fixed-pattern named-subagent + bounded-turns primitive is reliable and still gives the context-hygiene win. -- **Read `agentContextTokens` directly in conversation-runtime's compaction.** Rejected by integration fix C3 (would create a DAG back-edge to this slice). The injected `ContextBudgetProviding` (owned there) is the contract; this slice supplies the concrete provider. -- **A whole-new background runtime separate from the foreground one.** Rejected — two resident copies of the weights would blow the 48 GB budget. One batched runtime serves foreground slot 0 and background slots 1…K-1 over the same single weight read. - -## Target split & verification (per component) - -| Component | Target | Verification | -|---|---|---| -| `BatchedLLMRuntime` protocol (blueprint §3.6) | Core (MLX-free) | `swift build` / `swift test` (protocol + a stub conformer for tests) | -| `ConcurrencyBudget` + `KVCacheCost` (D4) | Core | `swift test` (K monotonic ↓ as context ↑; interleaved-layer KV sum; clamp ≥1; estimatedRAM) | -| `ModelDescriptor.maxContextTokens` (D5) | Core | `swift test` (registry decodes; clamp) | -| `ContextBudgetProviding` concrete provider (D5, C3) | Core | `swift test` (resolves effective budget = global ∩ model-max ∩ per-skill; injected into a conversation-runtime compaction test) | -| `agentContextTokens` / `agentContextPreset` / `agentCompactKV` persistence (D5/D6) | Core (`AppSettings`) | `swift test` (defaults Balanced, clamp, reset-to-defaults, legacy decode) | -| `Subagent` fixed-pattern primitive (D7) | Core | `swift test` (fresh-context isolation; only summary returned; bounded maxTurns) | -| `BatchedGemmaMLXRuntime` conformer: batched decode loop, KV-quant, prefix cache, rotating window (D2/D3) | GemmaRuntime (MLX-linked) | `xcodebuild` COMPILE-VERIFY ONLY; **real validation needs the user's stable-signed build** (TCC + Metal + live batching) | -| Provisioner returns the batched conformer (D9) | GemmaRuntime | `xcodebuild` compile-verify (the full product links Core + GemmaRuntime) | -| Hub AI-page cost surface (RAM/stream-count/speed) (D5) | Core/app UI | `xcodebuild` compile-verify; **user run-verify** the displayed numbers track the slider | - -To compile-check this slice in isolation from sibling uncommitted files, use a throwaway `git worktree` + `swift build` — never the shared working tree's `.app` (ad-hoc signing breaks TCC; the user does real builds). - -## Edge cases - -- **Context so large only the foreground fits (K=1).** `maxStreams` clamps to ≥1; background sessions simply don't get a slot until context/RAM allows. The cost surface shows "0 background sessions" honestly. -- **A new session can't be admitted this step (OOM headroom).** It waits (stays in `runnableSessions`); running streams are untouched. Not a failure. -- **Prefix changes (a skill is enabled mid-session).** The prefix hash changes → the prefix KV cache invalidates and re-prefills once; subsequent turns reuse the new cached prefix. -- **A parked session is restored to foreground while batched.** Identity is `AgentSessionID` (stable across park/restore); it just moves to slot 0 — its KV cache is preserved if still resident, else re-prefilled. -- **A stream errors mid-batch.** Per-stream terminal error → that turn `.failed`; the rest of the batch continues (D8). -- **`agentContextTokens` lowered below the current thread's assembled size.** conversation-runtime compaction (reading the new lower budget via the provider) compacts on the next turn; the rotating-window KV is the GPU-side backstop. -- **Compact-KV toggled mid-session.** KV precision change applies to *new* prefill; existing quantized/unquantized cache for live streams is honored until they free their slot (no mid-stream re-quantize). -- **Subagent itself wants a heavy context.** Its per-skill override resolves independently in its fresh session; it occupies a batch slot under the same RAM ceiling. - -## Risks / Trade-offs - -- **Batched decode is the hardest MLX code in the project and is `xcodebuild`-compile-verify only.** Real correctness (no cross-stream KV bleed, correct masking/padding, prefill interleave) can only be validated in the user's stable-signed build. Mitigation: keep all *schedulable* and *budget* logic pure in Core (tested), and keep the conformer a thin, well-documented decode loop; the spec's scenarios are written so the user run-verify is unambiguous. -- **KV-quant quality loss.** 8-bit KV is near-lossless; 4-bit (background only) may degrade long-thread coherence. Mitigation: 4-bit is reserved for the rotating-window background path; foreground uses bf16 or 8-bit. -- **Memory probing accuracy.** Free-unified-bytes is an estimate; activations spike during prefill. Mitigation: `reservedBytes` headroom + clamp ≥1 + admit-or-wait (never admit into OOM). -- **Prefix-cache invalidation bugs leak stale context.** Mitigation: key strictly by prefix-text hash; a mismatch always re-prefills. diff --git a/openspec/changes/ai-batched-runtime-and-context/proposal.md b/openspec/changes/ai-batched-runtime-and-context/proposal.md deleted file mode 100644 index 5de6c18..0000000 --- a/openspec/changes/ai-batched-runtime-and-context/proposal.md +++ /dev/null @@ -1,32 +0,0 @@ -## Why - -V2 turns the one-shot AI command into a conversational, tool-using, **background-capable** agent: parked sessions advance in the background, the foreground canvas streams a multi-turn thread, and long contexts (skills + memory TOC + a long conversation) become the norm. Two physics problems block that on the current runtime. - -1. **The idle GPU.** Decode is **memory-bandwidth-bound**, not compute-bound: each token step reads the entire ~17 GB weight set from unified memory to produce *one* token per stream. On the target machine (M5 Pro, 16-core GPU, 48 GB unified) a single `generate()` leaves most of the 16 GPU cores idle — the bottleneck is the one weight read, not the arithmetic. Spawning K independent `generate()` calls does **not** help: each one re-reads the full weights, so K parallel generations read the weights K times and contend for the same bandwidth. The right answer is **continuous batching** — fold K queued streams into **one** forward pass so the weights are read **once** and K tokens are produced together, harvesting the otherwise-idle cores. This is the v2 concurrency primitive that lets the foreground session and several parked sessions advance without N× the bandwidth cost. - -2. **Long context is expensive — and the cost is invisible.** The KV cache grows linearly with context length and *per concurrent stream*; on 48 GB unified the real concurrency ceiling is **RAM, not cores**. Growing context toward the model maximum is only affordable with **KV-cache quantization** (8/4-bit), **prefix/prompt caching** (the shared system + skills prefix computed once, not per turn), and a **rotating fixed-window** KV option for unbounded background threads. And because the user can grow context, the **RAM/speed cost must be surfaced** (presets + a "compact long contexts" toggle) rather than silently OOM-ing or thrashing. - -The existing seams already make this additive. The `LLMRuntime` protocol is the single model seam; `ModelManager` owns residency and injects the real runtime via the existing `ModelProvisioner` (no `ModelManager` API change). `ai-conversation-runtime` already added `LLMChatRequest` + a default-flattened `chat()` and reads its compaction budget through an **injected `ContextBudgetProviding`** (integration fix C3) — this slice supplies the concrete provider and the batched conformer that overrides `chat()` for true KV-reuse. - -## What Changes - -- **Batched / continuous-batching MLX runtime** behind the `LLMRuntime` seam (`BatchedLLMRuntime`, blueprint §3.6): a new MLX-linked conformer in the `GemmaRuntime` target that multiplexes up to **K** sessions over **one** weight read per decode step (`batchStep(...) -> AsyncThrowingStream<(AgentSessionID, Token), Error>`), de-multiplexing per-stream tokens keyed by `AgentSessionID`. The **foreground active session always gets a slot**; remaining slots are filled by consuming `ParkScheduler.runnableSessions(now:maxSlots:)` (blueprint §3.5). This is the concurrency answer — **NOT** spawning independent `generate()` calls. -- **Memory budget model (Core, pure).** A `ConcurrencyBudget` value type that, given weight size, free unified memory, per-token KV bytes (a function of context length and KV-quant bits), computes how many concurrent streams fit at a given context length — so `maxConcurrentStreams` (K) is **derived from RAM**, not a constant. Unit-tested with fixed inputs (no real Metal). -- **KV-cache quantization (8/4-bit), prefix/prompt caching, and a rotating fixed-window KV option** inside the batched conformer, accounting for Gemma's **interleaved sliding-window / global attention** (most layers are local sliding-window; a few are global) so the per-token KV math is per-layer-correct. -- **Context tuning, user-adjustable with the cost surfaced.** Add `ModelDescriptor.maxContextTokens`; add a persisted, user-adjustable `agentContextTokens` clamped to the model max, exposed via **presets** (Balanced / Long / Max) plus a **"compact long contexts (8-bit KV)"** toggle, with the estimated **RAM + speed cost surfaced** in the Hub. A **global default + per-skill override** for heavy skills. The effective budget feeds `ai-conversation-runtime`'s compaction trigger through the `ContextBudgetProviding` provider this slice ships. -- **Subagents as context hygiene (a fixed-pattern primitive, NOT open-ended spawning).** A `Subagent` primitive runs one sub-task in a **fresh** conversation context and returns only a **summary** message to the orchestrator, keeping the orchestrator lean. It is a fixed, bounded pattern (the orchestrator invokes a named subagent template with an input and gets back a summary) — explicitly **not** dynamic, model-driven recursive spawning (a small model handles that poorly). - -## Capabilities - -### Modified Capabilities - -- `on-device-ai-runtime`: ADD a batched/continuous-batching runtime requirement (K streams, one weight read, per-stream de-mux, foreground slot, scheduler-fed); ADD KV-cache quantization + prefix caching + rotating fixed-window KV with Gemma interleaved-attention accounting; ADD a memory-budget / concurrency-ceiling requirement (RAM is the ceiling); ADD growable + user-adjustable context with `maxContextTokens`/`agentContextTokens` and the injected budget provider; ADD the subagent context-hygiene primitive. The existing single-prompt `generate`, `structured`, lifecycle, registry, and `chat()` requirements are unchanged. -- `tunable-settings`: ADD the persisted `agentContextTokens` setting (presets + compact-KV toggle + per-skill override) with the RAM/speed cost surfaced, default Balanced, included in reset-to-defaults. - -## Impact - -- **Code (GemmaRuntime target — `xcodebuild` compile-verify only):** a new `BatchedGemmaMLXRuntime` conformer (continuous-batching decode loop, per-stream KV caches, KV-quant, prefix cache, rotating-window option); wiring it in via `GemmaRuntime.makeModelManager`'s existing `ModelProvisioner` (no `ModelManager` API change). The metallib bundle landmine stands — no change to `build-app.sh`'s `*.bundle` copy. -- **Code (MLX-free Core — `swift build` / `swift test`):** `ModelDescriptor.maxContextTokens`; `BatchedLLMRuntime` protocol (Core, MLX-free — the conformer is GemmaRuntime); a pure `ConcurrencyBudget` + `KVCacheCost` value model; a `Subagent` fixed-pattern primitive (pure orchestration); the concrete `ContextBudgetProviding` provider backed by `maxContextTokens` ∩ `agentContextTokens`; persisted `agentContextTokens` / `agentContextPreset` / `agentCompactKV` in `AppSettings`; a per-skill context-override read seam. -- **Consumes (verbatim, never redefines):** `LLMChatRequest` + `LLMRuntime.chat()` and `AgentMessage`/`AgentSessionID`/`AgentConversation`/`AgentTurn` (owned by `ai-conversation-runtime`); the `ContextBudgetProviding` protocol seam (owned by `ai-conversation-runtime`, integration fix C3 — this slice supplies the concrete provider); `ParkScheduler.runnableSessions`/`didAdvance` (owned by `ai-parked-sessions`, blueprint §3.5); `RuntimeError` + `AIError.message(for:)` for all failure surfacing. -- **House rules:** Apple-Silicon M5/M4 only — no Intel/low-end degraded path (the batched runtime and growable context exist *because* the hardware can serve them). One error taxonomy (`RuntimeError`), one translator, mapped at the GemmaRuntime boundary, surfaced bounded + non-blocking; a batch step that fails is an observable `.failed` for that stream's turn, never a false "done," and never an `NSAlert`. -- **Out of scope:** the conversation state machine / compaction logic itself (`ai-conversation-runtime`); the parked-session store + notch rail + lifecycle (`ai-parked-sessions`); the route loop / tool execution (`ai-tool-routing`); per-skill files (`ai-skills-as-files` owns the skill file; this slice only reads an optional context-override off it); any cloud/remote runtime. diff --git a/openspec/changes/ai-batched-runtime-and-context/specs/on-device-ai-runtime/spec.md b/openspec/changes/ai-batched-runtime-and-context/specs/on-device-ai-runtime/spec.md deleted file mode 100644 index 7924240..0000000 --- a/openspec/changes/ai-batched-runtime-and-context/specs/on-device-ai-runtime/spec.md +++ /dev/null @@ -1,100 +0,0 @@ -## ADDED Requirements - -### Requirement: Batched, continuous-batching concurrency over one weight read -The runtime SHALL provide a **batched** entry point that advances multiple conversation sessions concurrently by folding their queued streams into a **single forward pass per decode step**, so the model weights are read **once** per step and one token is produced for **each** active stream — rather than spawning independent single-session generations that would each re-read the weights. The batched entry point SHALL accept a set of per-session requests keyed by session identity and SHALL emit produced tokens **tagged with the session identity they belong to**, so a consumer can de-multiplex per-stream output. The **foreground active session SHALL always be granted a slot**; remaining slots SHALL be filled from the parked-session scheduler's runnable set. A stream that finishes SHALL free its slot for the next runnable session on the following step **without reloading the weights** (continuous, not static, batching). This batched runtime SHALL be the v2 concurrency mechanism; the system SHALL NOT achieve concurrency by running independent single-session generation calls in parallel. - -#### Scenario: K streams advance over a single weight read -- **WHEN** several sessions are runnable and the runtime advances a decode step -- **THEN** the weights are read once and one token is produced for each active stream in that single step, rather than once per stream - -#### Scenario: Per-stream tokens are de-multiplexed by session identity -- **WHEN** the batched entry point emits a token -- **THEN** the token carries the session identity it belongs to so the consumer routes it to the correct conversation - -#### Scenario: The foreground session always gets a slot -- **WHEN** the batch is full and the foreground session needs to advance -- **THEN** the foreground session is granted a slot and a background session yields, rather than the foreground waiting behind background work - -#### Scenario: A finished stream frees its slot without reloading weights -- **WHEN** one batched stream completes -- **THEN** its slot is filled by the next runnable session on the next step while the weights remain resident - -#### Scenario: A single stream's failure does not abort the batch -- **WHEN** one batched stream errors mid-flight -- **THEN** that stream's turn becomes an observable failed state with a clean headline while the other batched streams keep producing tokens, and no stream is reported as a false success - -### Requirement: Concurrency ceiling derived from unified memory -The number of concurrent streams the runtime serves SHALL be **derived from available unified memory**, not a fixed constant: the budget SHALL account for the resident weights (read once) plus one key/value cache **per concurrent stream**, where the per-stream cache size is a function of the context length and the key/value quantization. As the chosen context length grows, the affordable number of concurrent streams SHALL decrease accordingly. The budget SHALL always permit **at least the foreground stream** to fit, even when a chosen context length is so large that only one stream is affordable; in that case background sessions SHALL wait rather than the system over-committing memory. A session that cannot be admitted in a given step because memory headroom is insufficient SHALL **wait** (remain runnable) rather than be treated as a failure. - -#### Scenario: Growing context lowers the affordable stream count -- **WHEN** the user increases the context length -- **THEN** the runtime serves fewer concurrent streams because each stream's key/value cache is larger - -#### Scenario: The foreground always fits -- **WHEN** the chosen context length is large enough that only one stream's memory fits -- **THEN** the foreground session is served and background sessions wait, rather than the runtime over-committing memory - -#### Scenario: An unadmittable session waits, not fails -- **WHEN** a runnable background session cannot fit in the current step's memory headroom -- **THEN** it waits for a later step and is not reported as a failure - -### Requirement: KV-cache quantization, prefix caching, and a rotating fixed-window option -To make long context affordable, the runtime SHALL support **key/value cache quantization** (8-bit and 4-bit), **prefix/prompt caching** of the shared system + skills prefix (computed once and reused across turns and sessions rather than recomputed each turn), and an optional **rotating fixed-window** key/value cache that caps a stream's cache at the most recent window of tokens for unbounded background threads. The per-token cache-size accounting SHALL respect the model's **interleaved sliding-window / global attention**: most layers use a bounded sliding-window cache and a few layers use a full-context (global) cache, and the memory math SHALL sum these per-layer rather than treating every layer as global. The prefix cache SHALL be invalidated when the shared prefix changes. - -#### Scenario: The shared prefix is computed once, not per turn -- **WHEN** successive turns of a session share the same system + skills prefix -- **THEN** the prefix's key/value cache is reused and only the turn-specific suffix is prefilled, rather than recomputing the prefix each turn - -#### Scenario: Quantized key/value cache reduces per-stream memory -- **WHEN** the compact-context option selects 8-bit key/value caching -- **THEN** each stream's key/value cache occupies less memory, allowing more streams or more context at the same memory budget - -#### Scenario: A background thread's cache stays bounded by the rotating window -- **WHEN** a background session runs for many tokens with the rotating fixed-window option -- **THEN** its key/value cache stays bounded to the most recent window rather than growing without limit - -#### Scenario: Memory math respects interleaved attention -- **WHEN** the runtime computes a stream's key/value cache size at a context length -- **THEN** it sums the bounded sliding-window layers and the full-context global layers separately rather than treating every layer as full-context - -#### Scenario: A changed prefix invalidates the prefix cache -- **WHEN** the shared system + skills prefix changes (for example a skill is enabled mid-session) -- **THEN** the cached prefix is invalidated and re-prefilled once, and subsequent turns reuse the new cached prefix - -### Requirement: Growable, user-adjustable context with the cost surfaced -The system SHALL expose the model's maximum context length on the model descriptor and SHALL let the user grow the agent's context length toward that maximum, **clamped to it**. The chosen context SHALL be adjustable via **presets** (a balanced default, a long option, and a maximum option) plus a **"compact long contexts" (8-bit key/value)** toggle, and the system SHALL **surface the cost** of the choice — the estimated memory footprint, the resulting concurrent-stream count, and a relative speed note — so the user sees the RAM/speed trade-off rather than silently running out of memory or thrashing. The system SHALL support a **global default** context length and a **per-skill override** for heavy skills (the effective budget being the larger of the two, clamped to the model maximum). The effective context budget SHALL be supplied to the conversation runtime's compaction trigger through the injected budget provider, so the compaction threshold and the user-chosen context length never disagree. - -#### Scenario: Context is clamped to the model maximum -- **WHEN** the user selects the maximum context preset -- **THEN** the effective context length equals the model's architectural maximum and never exceeds it - -#### Scenario: The cost of a context choice is shown -- **WHEN** the user changes the context preset or the compact-context toggle -- **THEN** the surface shows the estimated memory footprint, the resulting concurrent-stream count, and a relative speed note for that choice - -#### Scenario: A heavy skill raises the effective context -- **WHEN** a session is driven by a skill that declares a larger context need than the global default -- **THEN** the effective context budget for that session is the larger of the two, clamped to the model maximum - -#### Scenario: The context choice feeds the compaction trigger -- **WHEN** the user grows the context length -- **THEN** the conversation runtime's compaction trigger reads the larger budget through the injected provider, so old turns are kept verbatim longer before being summarized - -### Requirement: Subagents as a fixed-pattern context-hygiene primitive -The system SHALL provide a **subagent** primitive that runs a bounded sub-task in a **fresh** conversation context (its own system prompt, an empty history, and a bounded turn count) and returns **only a summary** to the orchestrating session, so the orchestrator's context stays lean and never absorbs the sub-task's intermediate turns. The subagent primitive SHALL be a **fixed pattern** invoked by name from a registered set with an input, and SHALL NOT be an open-ended, model-driven, recursively self-spawning mechanism. A subagent SHALL be invokable as a routed tool step, and SHALL run as an ordinary batched stream (sharing the single weight read) rather than as its own independent weight read. - -#### Scenario: A subagent returns only a summary -- **WHEN** a subagent completes its sub-task -- **THEN** the orchestrating session receives only the subagent's summary, and the subagent's intermediate turns are not added to the orchestrator's context - -#### Scenario: A subagent runs in a fresh, bounded context -- **WHEN** a subagent is invoked -- **THEN** it runs in a fresh conversation with an empty history and a bounded turn count, isolated from the orchestrator's history - -#### Scenario: Subagents are fixed-pattern, not open-ended spawning -- **WHEN** the orchestrator needs a sub-task run -- **THEN** it invokes a named subagent from the registered set with an input, rather than the model dynamically deciding to recursively spawn arbitrary subagents - -#### Scenario: A subagent shares the single weight read -- **WHEN** a subagent runs alongside other sessions -- **THEN** it occupies a batch slot served by the same single weight read, rather than triggering an independent weight read diff --git a/openspec/changes/ai-batched-runtime-and-context/specs/tunable-settings/spec.md b/openspec/changes/ai-batched-runtime-and-context/specs/tunable-settings/spec.md deleted file mode 100644 index 65a7fe9..0000000 --- a/openspec/changes/ai-batched-runtime-and-context/specs/tunable-settings/spec.md +++ /dev/null @@ -1,28 +0,0 @@ -## ADDED Requirements - -### Requirement: Agent context-length setting with cost surfaced -With the AI commands opt-in on, the settings SHALL expose the agent's **context length** as a user-adjustable choice: a **preset** (a balanced default, a long option, and a maximum option equal to the model's architectural maximum) and a **"compact long contexts" (8-bit key/value)** toggle. The chosen context length SHALL be persisted, SHALL be **clamped to the selected model's maximum context**, and SHALL default to the **balanced** preset with the compact-context toggle **off**. The settings SHALL **surface the cost** of the choice — the estimated memory footprint, the resulting number of concurrent background sessions, and a relative speed note — so the user sees the RAM/speed trade-off when growing context, rather than the system silently running out of memory. These settings SHALL be included in **reset-to-defaults**, SHALL apply immediately when changed, and settings saved before this feature SHALL load with the balanced default and the toggle off. - -#### Scenario: Defaults are balanced with compaction off -- **WHEN** the app loads with no prior agent-context settings -- **THEN** the context preset is the balanced default and the compact-context toggle is off - -#### Scenario: Context choice shows its RAM and concurrency cost -- **WHEN** the user changes the context preset or the compact-context toggle with the AI opt-in on -- **THEN** the settings show the estimated memory footprint, the resulting concurrent-session count, and a relative speed note for that choice - -#### Scenario: Context is clamped to the selected model's maximum -- **WHEN** the user selects the maximum preset -- **THEN** the persisted context length equals the selected model's maximum and never exceeds it - -#### Scenario: Context setting persists and applies immediately -- **WHEN** the user changes the context preset and relaunches -- **THEN** the chosen preset is retained, and changing it while running takes effect on the next turn without a restart - -#### Scenario: Reset returns context to the balanced default -- **WHEN** the user resets settings to defaults -- **THEN** the context preset returns to balanced and the compact-context toggle returns to off - -#### Scenario: Older settings load with the balanced default -- **WHEN** settings saved before this feature are loaded -- **THEN** they decode successfully with the balanced context default and the compact-context toggle off, and existing settings are not reset diff --git a/openspec/changes/ai-batched-runtime-and-context/tasks.md b/openspec/changes/ai-batched-runtime-and-context/tasks.md deleted file mode 100644 index f2f67a4..0000000 --- a/openspec/changes/ai-batched-runtime-and-context/tasks.md +++ /dev/null @@ -1,55 +0,0 @@ -> Decomposed for a workflow fan-out: §1–§3 are the pure-Core substrate (do first, all `swift test`), §4 is the MLX-linked conformer (`xcodebuild` compile-verify only), §5 is the user-facing context tuning, §6 is the subagent primitive, §7 wires the provisioner, §8 verifies. The batched conformer (§4) requires the user's stable-signed build for real validation — the agent compile-verifies only. - -## 1. `BatchedLLMRuntime` protocol + memory budget (pure Core) - -- [x] 1.1 Add the `BatchedLLMRuntime` protocol (blueprint §3.6) to `AI/LLMRuntime.swift` (Core, MLX-free): `batchStep(_:[AgentSessionID:LLMChatRequest]) -> AsyncThrowingStream<(AgentSessionID, Token), Error>` + `var maxConcurrentStreams: Int`. It refines `LLMRuntime`; consumes `LLMChatRequest` + `AgentSessionID` verbatim from `ai-conversation-runtime` (do NOT redefine). *Verify: `swift build`.* -- [x] 1.2 Add `KVCacheCost` (Core): `slidingLayers`/`globalLayers`/`slidingWindow`/`kvBytesPerTokenPerLayer` + `kvBytes(forContext:)` implementing the Gemma interleaved sliding-window/global sum (D3). *Verify: `swift test` — uniform-global vs interleaved differ; sliding KV clamps at the window.* -- [x] 1.3 Add `ConcurrencyBudget` (Core): `maxStreams(contextTokens:)` = floor((free − weights − reserved) / kvBytes(ctx)), **clamped ≥ 1**; `estimatedRAM(streams:contextTokens:)`. Memory probe is injected (no Metal). *Verify: `swift test` — K decreases monotonically as context grows; K ≥ 1 even at model-max; estimatedRAM tracks streams×KV + weights.* -- [x] 1.4 Add a deterministic `StubBatchedRuntime` (Core, test-only) conforming to `BatchedLLMRuntime` that scripts per-stream token sequences. *Verify: `swift test` — K streams de-mux to the right `AgentSessionID`; a finished stream frees its slot for a queued one.* - -## 2. `ModelDescriptor.maxContextTokens` (pure Core) - -- [x] 2.1 Add `maxContextTokens: Int` to `ModelDescriptor` (AI/ModelRegistry.swift) and set real values on the `.standard` registry entries (Gemma 4's architectural max). *Verify: `swift test` — registry constructs; the field round-trips.* -- [x] 2.2 Confirm additive: existing `ModelDescriptor` construction sites still compile (it has an explicit init). *Verify: `swift build` + `xcodebuild` compile (GemmaRuntime's `pipelineModel(for:)` unaffected).* - -## 3. Concrete context-budget provider + per-skill override seam (pure Core) - -- [x] 3.1 Add `AgentContextBudgetProvider` conforming to `ai-conversation-runtime`'s `ContextBudgetProviding` (integration fix C3 — consume that protocol, do NOT redefine it): `maxContextTokens` resolves the effective budget = `agentContextTokens` ∩ `ModelDescriptor.maxContextTokens` ∩ per-skill override. *Verify: `swift test` — effective budget is the clamped min; an oversized slider clamps to model max.* -- [x] 3.2 Add a small `SkillContextOverriding` read seam (Core) — reads an optional context-override off the active skill (the value rides on the skill file owned by `ai-skills-as-files`; this slice only reads it). Default provider returns nil. *Verify: `swift test` — `max(globalDefault, skillOverride)` clamped to model max; nil override → globalDefault.* -- [x] 3.3 Wire the provider into a `ai-conversation-runtime` compaction test (inject this provider instead of the fixed stub) to prove growing the budget raises the compaction trigger. *Verify: `swift test` — a larger budget defers compaction; a smaller one triggers it sooner.* - -## 4. `BatchedGemmaMLXRuntime` conformer (GemmaRuntime — `xcodebuild` compile-verify ONLY) - -> MLX-linked. The agent NEVER builds/signs/installs the `.app` (ad-hoc signing breaks TCC grants). Real validation (live batching, KV-quant quality, no cross-stream bleed) requires the **user's stable-signed build** — see §8.3. The metallib bundle landmine stands: no change to `build-app.sh`'s `*.bundle` → `Contents/Resources/` copy. - -- [x] 4.1 Add `BatchedGemmaMLXRuntime` (GemmaRuntime target) conforming to `LLMRuntime` + `BatchedLLMRuntime`; subsume `GemmaMLXRuntime`'s single-session paths (text fast path, vision manual loop, reasoning channel classifier). *Verify: `xcodebuild` compile.* -- [x] 4.2 Implement the continuous-batching decode loop (D2): slot assignment (foreground slot 0, rest from `ParkScheduler.runnableSessions`), one batched forward pass per step (weights read once), per-stream sampling + de-mux to `(AgentSessionID, Token)`, mid-flight admit/free of slots, chunked prefill interleaved with decode. *Verify: `xcodebuild` compile; **user run-verify** (§8.3).* -- [x] 4.3 Implement per-stream **quantized** KV caches (8/4-bit, D3), selected by the compact-KV setting (8-bit) with 4-bit reserved for the rotating-window background path. Expose the per-layer sliding/global split to `ConcurrencyBudget`. *Verify: `xcodebuild` compile; **user run-verify** quality + RAM.* -- [x] 4.4 Implement **prefix/prompt caching** of the shared system+skills prefix (D3): compute prefix KV once, key by prefix-text hash, reuse across turns/sessions, invalidate on hash change. *Verify: `xcodebuild` compile; **user run-verify** repeated turns skip prefix prefill.* -- [x] 4.5 Implement the optional **rotating fixed-window KV** for unbounded background threads (D3) — bounded GPU cache independent of compaction. *Verify: `xcodebuild` compile; **user run-verify** a long background thread holds bounded KV.* -- [x] 4.6 Compute `maxConcurrentStreams` from `ConcurrencyBudget` against the live free-memory probe + current context setting + KV-quant bits (D1/D4); recompute on context-setting change / memory pressure. *Verify: `xcodebuild` compile; **user run-verify** K drops as context grows.* -- [x] 4.7 Map MLX/OS/OOM failures to `RuntimeError` at the conformer boundary; a stream error is per-stream terminal (the rest of the batch continues), surfaced via `AIError.message(for:)` — never a batch-wide abort, never a false "done," never `NSAlert` (D8). *Verify: `xcodebuild` compile; `swift test` on the Core de-mux/feedback path with the stub.* - -## 5. Context tuning settings + Hub cost surface (Core + app UI) - -- [x] 5.1 Persist `agentContextTokens` (Int), `agentContextPreset` (Balanced/Long/Max/custom; default **Balanced**), `agentCompactKV` (Bool; default off) in `AppSettings` — clamp to model max; included in **reset-to-defaults**; legacy settings decode with defaults. *Verify: `swift test` — defaults, clamp, reset, legacy decode.* -- [x] 5.2 Hub AI-page controls: the preset segmented control + the "compact long contexts (8-bit KV)" toggle, writing the persisted keys. Secondary model-management controls unchanged. *Verify: `xcodebuild` compile.* -- [x] 5.3 Hub **cost surface** (house requirement — never silent OOM): show estimated RAM + concurrent-stream count (from `ConcurrencyBudget`) + a relative speed note for the chosen context, updating live with the preset/toggle (D5). *Verify: `xcodebuild` compile; **user run-verify** the numbers track the chosen preset.* -- [x] 5.4 Per-skill context override: surface that a heavy skill may raise the effective context (read-only display via `SkillContextOverriding`); the value is authored on the skill file (`ai-skills-as-files`). *Verify: `xcodebuild` compile; `swift test` covers the resolution in §3.2.* - -## 6. `Subagent` fixed-pattern primitive (pure Core) - -- [x] 6.1 Add `Subagent` (name + systemPrompt + bounded `maxTurns`) and `SubagentResult` (summary + sessionID) (D7) — runs a sub-task in a **fresh** `AgentConversation`, returns only a summary `AgentMessage`. Fixed named set; **no** dynamic/recursive spawning. *Verify: `swift test` — orchestrator context excludes the subagent's intermediate turns; only the summary returns; `maxTurns` bounds it.* -- [x] 6.2 Expose "run subagent X" as a `ToolDescriptor`-shaped capability so it is invokable as a routed tool step (`ai-tool-routing` consumes it); a subagent rides a batch slot like any other session (concurrency-cheap, context-cheap). *Verify: `swift test` — the tool returns a `ToolStepResult` carrying the summary; `xcodebuild` compile for the GemmaRuntime slot usage.* - -## 7. Provisioner wiring (GemmaRuntime — `xcodebuild` compile-verify) - -- [x] 7.1 Change `GemmaRuntime.makeModelManager`'s `ModelProvisioner` to construct and `prepare` a `BatchedGemmaMLXRuntime` instead of `GemmaMLXRuntime` (D9) — **no `ModelManager` API change**; residency/lifecycle/registry behavior identical. *Verify: `xcodebuild` compile; the full product links Core + GemmaRuntime.* **(wire-full-potential-gates: now GATED — the provisioner constructs `BatchedGemmaMLXRuntime` ONLY when `batchedRuntimeUnlocked` (the already-resolved `FullPotentialGate.isUnlocked(.batchedRuntime)`, threaded via `AIRuntimeInjection.modelManagerFactory(optedIn:cpuLaneUnlocked:batchedRuntimeUnlocked:)` + `main.swift`); LOCKED (default master OFF) → the proven single-session `GemmaMLXRuntime` is resident, so the K-stream / growable-context surface is never constructed and the build behaves exactly as today's single-session one — the calm panic-off, no error. `xcodebuild` GREEN.)** -- [x] 7.2 The scheduler-driven background advancer downcasts `ModelManager.currentRuntime as? BatchedLLMRuntime` to drive `batchStep`; the foreground `chat()` path uses the base `LLMRuntime`. *Verify: `xcodebuild` compile; `swift test` for the Core-side downcast guard logic with the stub.* - -## 8. Verify - -- [x] 8.1 `swift build` + `swift test` green: `ConcurrencyBudget`/`KVCacheCost` (monotonic K, interleaved KV, clamp ≥1), the context-budget provider (effective-budget resolution + the conversation-runtime compaction tie-in), the `Subagent` isolation, the `StubBatchedRuntime` de-mux/slot-free, settings defaults/clamp/reset/legacy. *Verify: `swift test`.* -- [x] 8.2 `xcodebuild` compile-verify only: the full `ThreeFingerSwitcher` product builds + links (Core + GemmaRuntime/MLX) with `BatchedGemmaMLXRuntime` and the provisioner change. The agent does NOT build/sign/install the `.app`. *Verify: `xcodebuild` (compile).* -- [ ] 8.3 **User run-verify in a stable-signed build** (`INSTALL=1 ./scripts/build-app.sh`): the foreground session + ≥1 parked session advance together (one weight read, not N×); KV-quant + prefix caching hold; growing the context preset visibly lowers the background-stream count and the Hub cost surface tracks it; a heavy skill's per-skill override raises the effective context; a single stream's failure does not abort the batch. *Verify: user-build (required — Metal + TCC + live batching cannot be agent-verified).* -- [x] 8.4 `openspec validate --strict` passes; the `on-device-ai-runtime` ADDED requirements and the `tunable-settings` ADDED requirement match the implementation. *Verify: `openspec validate --strict`.* diff --git a/openspec/changes/ai-claude-handoff/.openspec.yaml b/openspec/changes/ai-claude-handoff/.openspec.yaml deleted file mode 100644 index 38f7628..0000000 --- a/openspec/changes/ai-claude-handoff/.openspec.yaml +++ /dev/null @@ -1,2 +0,0 @@ -schema: spec-driven -created: 2026-06-22 diff --git a/openspec/changes/ai-claude-handoff/design.md b/openspec/changes/ai-claude-handoff/design.md deleted file mode 100644 index 8676636..0000000 --- a/openspec/changes/ai-claude-handoff/design.md +++ /dev/null @@ -1,274 +0,0 @@ -## Context - -This slice is the **escalation valve**: when the small local agent (Gemma 4, batched, via MLX) judges a task beyond it, it reaches for real Claude Code. Read these before the design: - -- **`Launcher/ClaudeLaunch.swift`** — the open-claude-here launch path this slice REUSES wholesale. `ClaudeLauncher.commandScript(folder:command:claudePath:)` builds a self-deleting `.command` script that `cd`s into a folder and `exec`s an interactive **login** shell (so PATH from nvm/fnm/homebrew applies); `writeCommandFile(...)` writes it executable to a temp file; `resolveClaudePath()` finds the `claude` binary (login shell + known-install backstop). The handoff opens via `NSWorkspace.shared.open(url)` — **no Apple Events, no new permission**. `ClaudeLaunchError` is the existing clean `LocalizedError` taxonomy (`claudeNotFound`/`terminalOpenFailed`/`scriptWriteFailed`) with opt-in `copyableDetails`. -- **`Launcher/LaunchService.swift` → `launchClaude(folder:command:claudePath:title:)`** — the fire-and-forget production flow: resolution + write **off the main thread**, the `NSWorkspace` open + any failure notification on the main actor, **success needs no notification** (the terminal window is its own feedback), failure is bounded + non-blocking (never an alert). This is the exact shape the v1 handoff wants. -- **`openspec/specs/open-claude-here/spec.md`** — the capability spec: folder-bound launch, configurable command, no-new-permission handoff, robust `claude` resolution, bounded failure surfacing. The handoff tool is "open-claude-here, but the *model* fires it, the *folder+prompt* come from the conversation, and a *cost gate* sits in front." -- **`openspec/changes/ai-tool-routing/design.md`** — OWNS `ToolDescriptor`/`ToolRoute`/`ToolStepResult`/`ToolStepStatus`/`WritePolicyTier`/`ToolContributor`/`ToolRegistry`/`ApprovalGate`/`WritePolicyResolving`. This slice registers ONE more `ToolContributor`; the route→execute→continue loop is **untouched** (the tool-routing design explicitly notes "`launch_claude` arrives as one more `ToolContributor` with no loop change"). -- **`openspec/changes/ai-skills-as-files/design.md`** — `SkillManifest` carries an optional `claudeHandoff: ClaudeHandoffConfig?` block parsed from the skill file's front-matter; skills-as-files OWNS the carrier (the file format + manifest field) and CONSUMES the type, which THIS slice OWNS. - -The shared blueprint (`docs/ai-agent-v2-blueprint.md` §3.8) pins `ClaudeHandoffConfig`/`HandoffConfirmMode`; this slice OWNS those types. - -**Dependency note (honest):** `ai-background-autonomy` (which OWNS `AuditRecord`/`AuditLog`/`WritePolicyResolving`-as-whitelist-resolver/`ParkScheduler.escalate`) is NOT yet on disk (it is Wave 4, the same wave as this slice). `ai-tool-routing` already defines the bare `WritePolicyTier` enum WITH `ToolDescriptor` (integration fix C1) and a stand-alone `WritePolicyResolving`/`DescriptorWritePolicy` default, so this slice compiles + `swift test`-passes against those today and binds to autonomy's whitelist resolver + audit log when they land. Where a consumed autonomy type is not yet committed, this design declares a narrow Core seam this slice owns (`HandoffAuditing`, `HandoffEscalating`) with a no-op default, re-pointed at the real `AuditLog`/`ParkScheduler` when autonomy lands. See Open Questions. - -## Goals / Non-Goals - -**Goals:** -- A **model-callable `launch_claude` tool** the router can pick like any capability — `ToolDescriptor{name: "launch_claude", argsSchema: {folder, prompt}, writePolicy: .dangerous}` — that opens Claude Code in the right folder with a starting prompt, reusing the open-claude-here `.command` handoff (NO new launch mechanism, NO new permission). -- **Cost gating per skill:** `confirmMode` defaults to **`confirm`** (per call, money); a skill opts into **`auto`** explicitly in its `claudeHandoff:` block. The effective gate is `auto` ONLY when the skill says so AND the budget allows. -- A **budget/rate cap** (`maxCallsPerDay`, `maxConcurrent`) over an append-only spend ledger, `now:`-injected (deterministic), so an autonomous loop physically cannot rack up real spend. **Every** handoff (auto or confirmed) is audited. -- **DANGEROUS-tier behavior:** foreground unless the skill is `auto` AND under budget; a `confirm`/over-budget call escalates to the foreground approval step (active session) or the **needs-you** badge (parked session) — never runs silently. -- **V1 fire-and-forget:** open Claude, return `.done`, the local agent continues. The **round-trip** (consume Claude's output, resume the conversation) is documented as a future with a `HandoffOutcome` seam left. -- Entire slice MLX-free Core, verified by `swift test` with a fake `HandoffLauncher` + scripted `ApprovalGate` + fake `HandoffAuditing`. - -**Non-Goals:** -- The route→execute→continue loop, `ToolRegistry` aggregation, the `ApprovalGate` itself — OWNED by `ai-tool-routing`; this slice registers a contributor and reads the gate. -- The skill-file format + parsing of the `claudeHandoff:` block — OWNED by `ai-skills-as-files`; this slice OWNS only the `ClaudeHandoffConfig` type it parses into and the manifest field's semantics. -- The user whitelist, the append-only `AuditLog` storage, and `ParkState.needsYou` escalation plumbing — OWNED by `ai-background-autonomy`; consumed here via narrow seams. -- Capturing / parsing Claude Code's output, and resuming the local conversation from it (the **round-trip** — explicitly a documented future; the seam is left, the behavior is not built). -- Choosing the Claude model, passing arbitrary Claude CLI flags, or any non-Claude escalation backend. -- A full Hub UI for the global budget (the persisted `claudeHandoffBudgetPerDay` key is defined; surfacing it is a small follow-up, noted not built). - -## Decisions - -### 1. Type contracts (blueprint §3.8, OWNED here) - -```swift -public enum HandoffConfirmMode: String, Codable, Equatable, Sendable { - case confirm // DEFAULT — foreground approval per call (money is on the line) - case auto // per-skill opt-in: runs without per-call approval, STILL budget-capped + audited -} - -public struct ClaudeHandoffConfig: Codable, Equatable, Sendable { - public var enabled: Bool // a skill may carry handoff but keep it off - public var confirmMode: HandoffConfirmMode // defaults to .confirm - public var maxCallsPerDay: Int // per-skill rate cap (0 = use the global default) - public var maxConcurrent: Int // at-most-N in-flight (v1: in-flight = "opened, not yet reaped"; see §6) - public var defaultWorkingDirectory: URL? // fallback folder when the route omits one - - public static let `default` = ClaudeHandoffConfig( - enabled: true, confirmMode: .confirm, maxCallsPerDay: 0, maxConcurrent: 1, - defaultWorkingDirectory: nil) -} -``` - -- `confirmMode` defaults to **`.confirm`** — the cost decision. A skill file opts into `auto` only by writing `confirmMode: auto` (or a `claudeHandoff: { auto: true }` shorthand) in its front-matter; `ai-skills-as-files` parses it into this struct, so the default-confirm rule holds for any skill that omits the block. -- `maxCallsPerDay == 0` means "fall back to the **global** persisted default" (`claudeHandoffBudgetPerDay`, a camelCase agent-scoped settings key per the naming convention). A skill may pin a tighter per-skill cap. -- This struct is the SAME type carried on `SkillManifest.claudeHandoff` (§ ai-skills-as-files); skills-as-files owns the carrier, this slice owns the type. There is exactly one definition. - -### 2. `launch_claude` is a `ToolDescriptor` (writePolicy `.dangerous`) - -```swift -// argsSchema (a StructuredSchema, the same shape ai-command-tasks uses for ParsedActions): -{ - "type": "object", - "required": ["prompt"], - "properties": { - "folder": { "type": "string", "description": "absolute path to open Claude in; omit to use the skill's default working directory" }, - "prompt": { "type": "string", "description": "the starting prompt Claude opens with" } - } -} -``` - -`ClaudeHandoffContributor.descriptors()` returns a single `ToolDescriptor{name: "launch_claude", summary: "Hand the task to Claude Code in a folder with a starting prompt — use when the task exceeds the local model.", argsSchema: aboveSchema, writePolicy: .dangerous, keywords: ["claude","handoff","escalate","code","refactor","big task"]}`. `writePolicy` is **always `.dangerous`** on the descriptor — the `auto` opt-in is a *per-skill effective-tier downgrade* (Decision 4), NOT a descriptor change, so the registry/router never sees a non-dangerous handoff descriptor and a skill cannot accidentally publish a globally-auto handoff. A skill that wants the handoff also lists `launch_claude` in its `tools:` allow-list (§ ai-tool-routing candidate inclusion). - -### 3. The contributor — `run` reuses the open-claude-here launch path - -```swift -public protocol HandoffLauncher: Sendable { // the side-effecting spawn seam (headless-fakeable) - // Fire-and-forget: write the .command + NSWorkspace.open; throws a mapped failure if it can't. - func launch(folder: URL, prompt: String) async throws -} - -public struct ClaudeHandoffContributor: ToolContributor { - let config: ClaudeHandoffConfig // the active skill's config (or .default for a free handoff) - let budget: HandoffBudget // pure rate/budget gate (Decision 5) - let launcher: HandoffLauncher // production = OpenClaudeHandoffLauncher (Decision 7) - let audit: HandoffAuditing // → ai-background-autonomy AuditLog (Decision 6) - let resolver: WritePolicyResolving // effective-tier resolution (Decision 4) - let escalation: HandoffEscalating // parked → needs-you (Decision 8) - - public func descriptors() -> [ToolDescriptor] // [launch_claude] - public func canHandle(_ tool: String) -> Bool // tool == "launch_claude" - public func run(_ call: RoutedCall, gate: ApprovalGate) async -> ToolStepResult -} -``` - -`run` is the heart (Decision 4's state machine). The production `HandoffLauncher` (`OpenClaudeHandoffLauncher`) is a thin adapter over the existing `ClaudeLauncher`: it builds the inner command as **`claude `** (the starting prompt is passed as Claude's argument, exactly how a custom `command` is passed through `ClaudeLauncher.commandScript`'s inner-command slot — so a non-empty prompt runs `claude ""`, an empty prompt opens a bare `claude` session), resolves the path via `resolveClaudePath()`, writes the `.command` via `writeCommandFile`, and opens it with `NSWorkspace.shared.open(url)` — **byte-for-byte the open-claude-here handoff**, off-main, success-needs-no-notification. No new launch mechanism, no new permission. - -### 4. Effective-tier resolution — the cost gate (the heart of the slice) - -`launch_claude`'s descriptor is always `.dangerous`. The **effective** gate for a given call is resolved at `run` time: - -``` -effectiveGate(config, budget, now) -> HandoffGate - if !config.enabled -> .disabled // .declined("Claude handoff is off for this skill") - if !budget.allows(now) -> .overBudget // degrade per below - switch config.confirmMode: - .auto -> .autoRun // run now, no per-call approval (still audited) - .confirm -> .needsApproval // foreground DOWN=approve / RIGHT=skip -``` - -The mapping of `effectiveGate` → behavior in `run`: -- **`.autoRun`** (skill opted `auto` AND under budget): record the spend, emit an `AuditRecord(wasBackground: session-parked)`, `launcher.launch(folder, prompt)`, return `ToolStepResult(.done, "Opened Claude Code in .")`. If `launch` throws → `.failed(headline)` (never a false "Done") AND the spend is **refunded** to the ledger (a launch that didn't land didn't spend). -- **`.needsApproval`** (default-confirm, OR an `auto` skill the user chose to keep confirming): emit `ToolStepResult(.awaitingApproval, "Hand this to Claude Code in ?")` and surface the call as observable approval state; await the `ApprovalGate`: - - **approve (DOWN)** → record + audit + `launch` → `.done` (or `.failed` + refund). - - **skip (RIGHT)** → `ToolStepResult(.declined(reason: "skipped"))`, audited as declined, **no spend**. - - **cancel** (whole canvas discarded) → ends the loop quietly (`RuntimeError.cancelled` semantics), no spend, no launch. -- **`.overBudget`**: this is the runaway-spend guard. An **`auto` skill over budget DEGRADES to `.needsApproval`** (a foreground confirm) rather than running unprompted OR silently dropping — the user can still choose to approve the one extra call; an autonomous loop cannot self-approve. A **`confirm` skill over budget** is already foreground; it shows the budget state in the approval card so the user knows they are over the daily cap. Either way the call is **never silently dropped** — over-budget without a user present (a parked session) escalates to needs-you (Decision 8). -- **`.disabled`**: `ToolStepResult(.declined(reason))` with a clean headline; no spend, no launch. - -`resolver` (`WritePolicyResolving`, owned by `ai-tool-routing` / refined by `ai-background-autonomy`) intersects the descriptor's `.dangerous` with the user whitelist FIRST — a user who has not whitelisted handoff keeps it `.dangerous` → always `.needsApproval`/needs-you regardless of the skill's `auto`. The skill `auto` can only *downgrade within* what the whitelist permits; it can never *override* a user who declined to trust handoff. This is the autonomy contract: **effective tier = descriptor default ∩ user whitelist**, then the per-skill `confirmMode` chooses within the permitted range. - -### 5. The budget/rate cap — pure, `now:`-injected, refundable - -```swift -public struct HandoffSpend: Codable, Equatable, Sendable { public let at: Date; public let skillID: String? } - -public struct HandoffBudget: Equatable, Sendable { - public let maxCallsPerDay: Int // resolved (skill cap or global default) - public let maxConcurrent: Int - public private(set) var ledger: [HandoffSpend] // append-only spend records; in-flight tracked separately - public private(set) var inFlight: Int - - // Pure predicate — `now` is an INPUT (deterministic, DockHoverModel-style). - public func allows(now: Date) -> Bool { - callsInLast24h(now) < maxCallsPerDay && inFlight < maxConcurrent - } - public mutating func record(at: Date, skillID: String?) // spend a call (append + inFlight += 1) - public mutating func reap() // a fire-and-forget launch is "done" → inFlight -= 1 (v1: immediately after open, see §6) - public mutating func refund(at: Date) // a launch that threw didn't spend → remove + inFlight -= 1 - func callsInLast24h(_ now: Date) -> Bool // rolling window, deterministic -} -``` - -- The **rolling-24h** call count (`maxCallsPerDay`) is a sliding window over `now`, not a calendar-day reset — so a loop cannot dump N calls at 23:59 and N more at 00:01. `now:` is injected so the window is unit-testable without a clock. -- The ledger is **append-only** and persisted (mirroring the audit log's durability) so the cap survives a relaunch within the window — a process restart cannot reset the daily budget. (v1 persists the ledger alongside settings under Application Support, like `ClipboardStore`.) -- `maxConcurrent` bounds simultaneous in-flight handoffs (v1 default 1). Because v1 is fire-and-forget with no reaping signal from Claude, **`reap()` runs immediately after a successful `open`** — concurrency is effectively "don't fire two handoffs from the same loop step." The seam is left so the future round-trip reaps on *Claude exit* instead (Decision 9). This is an explicit v1 simplification, documented, not a bug. - -### 6. Audit — every handoff is recorded (append-only, consumed from autonomy) - -```swift -public protocol HandoffAuditing: Sendable { // → ai-background-autonomy AuditLog.record(_:) - func record(_ record: AuditRecord) async -} -``` - -Every `run` outcome emits exactly one `AuditRecord{sessionID, tool: "launch_claude", policy: .dangerous, argumentsSummary: " · ", outcome: ToolStepStatus, wasBackground: session-parked, timestamp}` (blueprint §3.7). `argumentsSummary` is **redacted/short** — the folder path + a truncated prompt, NEVER the full prompt verbatim (a prompt can carry secrets; raw text rides only in logs / opt-in details, never the audit headline). The audit fires for `.done`, `.declined`, `.failed`, AND `.overBudget`-degraded — so the spend ledger and the audit log agree, and a user can see "the agent wanted to escalate 4 times today, 1 ran." Until `ai-background-autonomy` lands, `HandoffAuditing` has a no-op default (`NoopHandoffAudit`) and a recording test double; it re-points at the real `AuditLog` when autonomy commits. - -### 7. Production launcher — the open-claude-here adapter (Core, side-effecting) - -```swift -public struct OpenClaudeHandoffLauncher: HandoffLauncher { - public func launch(folder: URL, prompt: String) async throws { - // Off-main resolution + write, NSWorkspace open on the main actor — the launchClaude() shape. - let inner = prompt.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty - ? nil // empty prompt → bare claude session - : "claude \(ClaudeLauncher.shellQuote(prompt))" // starting prompt as claude's argument - let claudePath = inner == nil ? ClaudeLauncher.resolveClaudePath() : nil - let url = try ClaudeLauncher.writeCommandFile(folder: folder, command: inner, claudePath: claudePath) - // open via NSWorkspace.shared.open(url) on the main actor; throw HandoffError on failure - } -} -``` - -This is the ONLY side-effecting code in the slice, and it adds **no new launch primitive** — it composes `ClaudeLauncher`'s existing pure script builder + write + the `.command`/`NSWorkspace` handoff. It lives in Core (the open-claude-here code already does) but its real side effect is exercised by the **user's** stable-signed build; `swift test` drives the contributor against a **fake** `HandoffLauncher` that records `(folder, prompt)` without spawning anything. Failures map at this boundary into `HandoffError` (Decision 10). - -### 8. Parked-session escalation — dangerous always escalates, even when parked - -Adopting the autonomy rule (do not relitigate): a **dangerous** handoff in a **parked** session does NOT auto-run and does NOT silently wait — it raises `ParkState.needsYou` + a badge via `ParkScheduler.escalate`, so the user is pulled back to approve the spend. This applies to BOTH a `.needsApproval` handoff (default-confirm) AND an `.overBudget`-degraded `auto` handoff that has no foreground user to confirm it. An `.autoRun` handoff (skill `auto`, under budget, in a parked session) DOES run in the background (it is the explicitly-trusted, budget-capped, audited path the user opted into) — mirroring "whitelisted writes are auto even when parked, still audited." The contributor reads whether the session is parked from the `RoutedCall`/context (`wasBackground`) and routes to `escalation.escalate(...)` instead of the synchronous `ApprovalGate` when parked-and-needs-approval. - -```swift -public protocol HandoffEscalating: Sendable { // → ai-parked-sessions ParkScheduler.escalate - func escalate(_ sessionID: AgentSessionID, reason: String) async -} -``` - -Until `ai-parked-sessions`/`ai-background-autonomy` land, a no-op default + a recording double; re-pointed at `ParkScheduler.escalate` when they commit. When parked-and-escalated, `run` returns `ToolStepResult(.awaitingApproval, ...)` so the parked-session UI shows the pending handoff behind the needs-you badge; no spend until the user returns and approves. - -### 9. V1 fire-and-forget — and the round-trip seam left for the future - -V1 contract: `launch` opens Claude with the prompt; the contributor returns `.done` and the loop continues. The local agent does **not** wait for Claude, does **not** read Claude's output. This is deliberate — a synchronous round-trip would block the local loop on a long Claude session and tangle two agent loops. The future round-trip is documented and **seamed**: - -```swift -// FUTURE (not built in v1): the launcher returns a handle a later slice can poll/await. -public struct HandoffOutcome: Sendable { public let handle: UUID /* ; future: stdout, exitCode, resultText */ } -// v1 HandoffLauncher.launch returns Void (fire-and-forget). A future variant returns HandoffOutcome, -// reaps inFlight on Claude EXIT (not on open), and feeds resultText back as a .tool AgentMessage so -// the local agent continues from Claude's result. The contributor's run() already appends a .tool -// result; swapping .done for a consumed HandoffOutcome is additive. -``` - -The v1 `ToolStepResult.summary` ("Opened Claude Code in .") is what the model sees as the tool result — enough for it to know the handoff happened and to wind down its own loop (it should NOT keep planning around a result it won't get). No round-trip code is written; only the shape is left so the future slice is additive. - -### 10. Errors — one taxonomy, mapped at the boundary, observable + bounded - -A new `enum HandoffError: Error, Equatable, LocalizedError` ONLY for handoff cases `RuntimeError`/`TaskError`/`ClaudeLaunchError` cannot carry: - -```swift -public enum HandoffError: Error, Equatable { - case disabled // the skill carries handoff but enabled == false - case overBudgetNoUser // over budget AND parked with no one to confirm (escalated) - case missingFolder // no folder in the route AND no defaultWorkingDirectory - case launchFailed(headline: String, details: String?) // wraps a mapped ClaudeLaunchError -} -``` - -- Clean `errorDescription` per case; raw OS/vendor text rides ONLY in `details` / logs, never the headline. `AIError.message(for:)` is extended to translate `HandoffError` → `AIPresentedError` (THE one translator) so a handoff failure reads identically in the canvas, the audit, and any settings row. -- The production launcher maps `ClaudeLaunchError` (`claudeNotFound`/`terminalOpenFailed`/`scriptWriteFailed`) into `HandoffError.launchFailed` at the launch boundary — Core stays consistent, the existing clean headline flows through (e.g. "Couldn't find the 'claude' command. Install Claude Code, then try again."). -- A failure becomes a `ToolStepResult(.failed(headline:))` (a side effect that didn't land is `.failed`, never a false "Done") and the spend is refunded. No `NSAlert`; the canvas / parked UI render it bounded + non-blocking with Retry/Dismiss. -- `RuntimeError.cancelled` (a discard during approval) is NOT a failure — the loop ends quietly, no spend, no launch (the autonomy/routing cancel path). - -## Type & file touch list (all Core, MLX-free; verified by `swift test` unless noted) - -| File (new unless noted) | Target | Contents | Verification | -|---|---|---|---| -| `AI/Handoff/ClaudeHandoffConfig.swift` | Core | `HandoffConfirmMode`, `ClaudeHandoffConfig` (+ `.default`, confirm-default) | `swift test` (Codable round-trip, default == confirm) | -| `AI/Handoff/HandoffBudget.swift` | Core | `HandoffSpend`, `HandoffBudget` (`allows(now:)`, `record`/`reap`/`refund`, rolling-24h) | `swift test` (cap, rolling window, concurrency, refund) | -| `AI/Handoff/HandoffLauncher.swift` | Core | `HandoffLauncher` protocol + `OpenClaudeHandoffLauncher` (the open-claude-here adapter) | `swift test` (protocol + prompt→inner-command mapping); side effect = **user build** | -| `AI/Handoff/ClaudeHandoffContributor.swift` | Core | `ToolContributor` for `launch_claude`; `descriptors()`; the `effectiveGate` resolution; `run` state machine; audit emission; parked escalation | `swift test` w/ fake launcher + scripted gate + recording audit/escalation | -| `AI/Handoff/HandoffSeams.swift` | Core | `HandoffAuditing`/`HandoffEscalating` narrow seams + no-op defaults (re-pointed at `AuditLog`/`ParkScheduler` when autonomy/parked land) | `swift test` | -| `AI/Handoff/HandoffError.swift` | Core | `HandoffError` (`LocalizedError`) + `AIError.message(for:)` extension | `swift test` (clean headline, no raw interpolation) | -| `Tests/.../ClaudeHandoffTests.swift` etc. | Core (test) | fake `HandoffLauncher`, scripted `ApprovalGate`, recording audit/escalation; every `run` branch | `swift test` | - -No file in this slice links MLX. The production `OpenClaudeHandoffLauncher` composes the existing `Launcher/ClaudeLaunch.swift` (already Core) — **no change** to that file is required; the contributor only *calls* its pure builders. **No `.app` build, no signing, no permission change — the user's stable-signed build is unaffected;** the real spawn is exercised by the user end-to-end. - -## Edge cases - -- **Route gives no `folder`:** fall back to `config.defaultWorkingDirectory`; if that is also nil → `ToolStepResult(.failed(HandoffError.missingFolder headline))`, no spend. The model can re-route with a folder. -- **Route gives an empty `prompt`:** allowed — opens a **bare `claude` session** in the folder (the open-claude-here default-command behavior). Audited with an empty-prompt summary. -- **`claude` not found at launch time:** the production launcher's `ClaudeLauncher.resolveClaudePath()` returns nil → the script falls back to PATH-`claude` (existing behavior); if the open itself fails, `ClaudeLaunchError` maps to `HandoffError.launchFailed` → `.failed` + refund. (Unlike open-claude-here's *setup-time* resolution, the handoff resolves at fire time, so a clean failure surfaces in the canvas, never a silent no-op.) -- **`auto` skill, over the daily cap:** degrades to a foreground `.needsApproval` (active session) or needs-you (parked) — never silently dropped, never auto-run over budget. The approval card states "daily Claude limit reached" so the user knows this is the budget speaking. -- **`confirm` skill, over the cap:** already foreground; the approval card shows the over-budget state; approving still spends (the user is the cap override, the loop is not). -- **User has NOT whitelisted handoff:** the resolver keeps the effective tier `.dangerous` → always foreground/needs-you regardless of the skill's `auto`. A skill cannot self-grant background spend. -- **`launch` throws after the spend was recorded:** refund the ledger entry + `inFlight -= 1` and return `.failed` — a handoff that didn't land didn't spend, so the cap stays honest. -- **Whole canvas discarded mid-approval:** gate resolves cancelled → loop ends quietly, no spend, no launch (`RuntimeError.cancelled` is not a failure). -- **Two `launch_claude` steps in one loop:** `maxConcurrent` (default 1) blocks the second within the same step window (it reads `inFlight`); since v1 reaps on open, a *later* step is allowed once the prior opened — bounded by the daily cap. Documented v1 concurrency semantics. -- **Skill carries `claudeHandoff` but `enabled: false`:** `.declined("Claude handoff is off for this skill")` — a skill can ship the config disabled and a user/Hub can flip it on later. -- **`maxCallsPerDay == 0` (use global):** resolves to the persisted `claudeHandoffBudgetPerDay`; if that is also 0 (a user who disabled handoff globally) → treat as disabled → `.declined`. -- **Ledger persisted across relaunch:** a process restart does NOT reset the rolling-24h window — the cap survives, so an agent cannot restart itself to dodge the budget. - -## Rejected alternatives - -- **`launch_claude` as a `.confirm` (not `.dangerous`) descriptor.** Rejected: money is on the line; the blueprint adopted decision pins handoff as dangerous-tier (foreground unless the skill is auto AND under budget). A `.confirm` default would let the whitelist resolver treat it like a benign write. -- **Per-skill `auto` flips the descriptor's `writePolicy` to `.auto`.** Rejected: that would publish a globally-auto handoff descriptor the router could pick from ANY context, bypassing the cost gate. The descriptor stays `.dangerous`; `auto` is a per-skill *effective-tier downgrade* resolved at `run` time, intersected with the user whitelist. -- **Synchronous round-trip in v1 (await Claude's output, feed it back).** Rejected for v1: it blocks the local loop on a long Claude session and tangles two agent loops; it also needs a way to capture Claude Code's structured output that does not exist in the `.command` handoff. Fire-and-forget is the bounded v1; the round-trip is seamed for a future slice. -- **A calendar-day budget reset (midnight).** Rejected: a loop could dump N calls at 23:59 and N more at 00:01. A rolling-24h window over an injected `now:` is the runaway-spend-safe cap. -- **A new launch mechanism / scripting Terminal.app for the handoff.** Rejected: the open-claude-here `.command` + `NSWorkspace.open` handoff already opens the user's default terminal with NO new permission (no Apple Events). Reuse it byte-for-byte. -- **Silently dropping an over-budget call.** Rejected: a side effect that didn't happen must be observable. Over-budget degrades to a foreground confirm / needs-you and is audited — never a silent no-op, never a false "Done." -- **The full prompt verbatim in the audit record.** Rejected: a prompt can carry secrets. `argumentsSummary` is redacted/short (folder + truncated prompt); raw text rides only in logs / opt-in details. -- **A dedicated `HandoffAuditLog` separate from autonomy's `AuditLog`.** Rejected: one append-only audit log (blueprint §3.7); this slice records INTO it via a narrow seam, it does not fork the log. - -## Target-split & verification summary - -- **Everything in this slice is MLX-free Core**, verified by `swift build` + `swift test`. The config/budget/error types are pure value types; `effectiveGate`/`HandoffBudget.allows(now:)` are pure functions (`now:`-injected, deterministic); the contributor's `run` is driven by an injected `HandoffLauncher` (fake in tests, recording `(folder, prompt)`), a scripted `ApprovalGate`, and a recording `HandoffAuditing`/`HandoffEscalating`. -- **The production spawn (`OpenClaudeHandoffLauncher`) lives in Core** (it composes the already-Core `ClaudeLauncher`), but its real side effect (writing the `.command`, `NSWorkspace.open`) is exercised by the **user's stable-signed build** end-to-end — an agent never builds/signs/installs the `.app` (ad-hoc signing breaks TCC grants). -- **No `xcodebuild`/`.app`/signing in this slice.** The MLX-linked batched runtime that ultimately answers route turns is owned downstream; this slice is agnostic to which `LLMRuntime` conformer drives the loop. To compile-check in isolation without other in-flight slices' uncommitted files, use a throwaway `git worktree` + `swift build`. - -## Open Questions - -- **Q1 (autonomy seams):** `ai-background-autonomy` (`AuditRecord`/`AuditLog`/whitelist `WritePolicyResolving`) and `ai-parked-sessions` (`ParkScheduler.escalate`/`AgentSessionID`) are not yet on disk (same Wave 4). I plan narrow `HandoffAuditing`/`HandoffEscalating` seams I own with no-op defaults, re-pointed at the real types when they land. Confirm the blueprint §3.7/§3.5 sketches are final enough to bind, and confirm `AuditRecord`'s `argumentsSummary` redaction contract. -- **Q2 (global budget surfacing):** I define the persisted `claudeHandoffBudgetPerDay` settings key + the ledger persistence, but a Hub UI to view/edit the global budget + the spend ledger ("the agent escalated N times today") is a small follow-up. Confirm whether it ships in this slice or as a Hub-page follow-up alongside the autonomy whitelist UI. -- **Q3 (v1 concurrency reaping):** v1 reaps `inFlight` immediately on `open` (no exit signal from a fire-and-forget `.command`). `maxConcurrent` therefore really means "one handoff per loop step." Confirm that is acceptable for v1, with true on-exit reaping deferred to the round-trip future. -- **Q4 (skill `auto` shorthand):** the skill front-matter block — `claudeHandoff: { auto: true, maxPerDay: 3, dir: "~/proj" }` vs the full `ClaudeHandoffConfig` field names. `ai-skills-as-files` owns the parse; confirm the front-matter spelling so the two slices agree. diff --git a/openspec/changes/ai-claude-handoff/proposal.md b/openspec/changes/ai-claude-handoff/proposal.md deleted file mode 100644 index 3ea4835..0000000 --- a/openspec/changes/ai-claude-handoff/proposal.md +++ /dev/null @@ -1,28 +0,0 @@ -## Why - -The V2 agent runs a **small local model** (Gemma 4 via MLX, batched). It is fast, private, and free — but it is small. Some requests exceed it: a multi-file refactor, a real coding task, a deep research synthesis, anything that wants a frontier model with tools and filesystem reach. The right move there is not to make the small model fail or hallucinate a half-answer — it is to **hand the job to real Claude Code**, in the right folder, with a starting prompt the local agent has already shaped from the conversation. - -The escalation surface already exists in the codebase, just not as a tool the model can reach: the **open-claude-here** capability resolves the `claude` binary and opens it in a chosen folder through a self-deleting `.command` handoff that needs **no new permission** (no Apple Events). V2 turns that launch path into a **model-callable tool** — `launch_claude(folder, prompt)` — so the router can pick it like any other capability when the task is beyond the local model. - -But Claude Code costs real money. An autonomous agent loop that could spawn Claude freely is a runaway-spend risk. So the handoff is a **DANGEROUS-tier action by default**: it requires foreground confirmation per call, and even the explicitly-trusted "auto" path is **rate/budget-capped** and **fully audited**, so a spinning loop physically cannot rack up real spend. V1 is deliberately **fire-and-forget** — open Claude with the prompt, consider the handoff complete; a structured round-trip (the local agent consuming Claude's output and continuing) is a documented future with the seam left for it. - -## What Changes - -- **The `launch_claude` tool.** A `ToolDescriptor{name: "launch_claude", argsSchema: {folder, prompt}, writePolicy: .dangerous}` aggregated into `ai-tool-routing`'s `ToolRegistry` via a `ClaudeHandoffContributor` (`ToolContributor`). When the router picks it and the call is approved, the contributor reuses the **existing open-claude-here launch path** (`ClaudeLauncher` script builder + the `.command` handoff) to spawn `claude` in `folder` with `prompt` as its starting input — **no new permission, no new launch mechanism.** -- **Per-skill cost gating (`auto | confirm`).** A `ClaudeHandoffConfig{enabled, confirmMode, maxCallsPerDay, maxConcurrent, defaultWorkingDirectory}` (blueprint §3.8). `confirmMode` defaults to **`confirm`** (money is on the line); a skill opts into **`auto`** explicitly in its skill-file `claudeHandoff:` block (the per-skill block already carried on the `SkillManifest` by `ai-skills-as-files`). The effective tier resolves to `.confirm` unless a skill opts `auto` AND the budget allows. -- **Budget/rate cap with audit.** A pure `HandoffBudget` enforces `maxCallsPerDay` and `maxConcurrent` over an append-only spend ledger; `now:` is an input (deterministic). Every handoff — auto or confirmed — emits an `AuditRecord` to `ai-background-autonomy`'s `AuditLog`. An over-budget call is **never silently dropped**: it surfaces a bounded, clean `.declined`/`.failed` step ("daily Claude handoff limit reached"), and an `auto` skill over budget **degrades to a foreground confirm** rather than running unprompted. -- **DANGEROUS-tier gating.** `launch_claude` is foreground unless the skill is `auto` AND under budget. A `confirm`-mode or over-budget call escalates to the foreground approval step (DOWN=approve / RIGHT=skip) when the session is active, or to the **needs-you** badge (`ParkState.needsYou`) when the session is parked — adopting the autonomy rule that dangerous writes escalate even when parked. -- **V1 fire-and-forget.** The contributor's `run` opens Claude and returns `ToolStepResult(.done, "Opened Claude Code in .")` — the local agent considers the handoff complete and continues its loop. There is **no** capture of Claude's output in v1; the **round-trip is a documented future** (a `HandoffOutcome` seam is left so a later slice can consume Claude's result and resume the conversation). - -## Capabilities - -### New Capabilities - -- `ai-claude-handoff`: the escalation valve — the `launch_claude` model-callable tool, the per-skill `ClaudeHandoffConfig` (`auto | confirm`, default confirm), the rate/budget cap with append-only audit, DANGEROUS-tier foreground/needs-you gating, and the v1 fire-and-forget launch reusing the open-claude-here `.command` handoff (with the round-trip seam left for a future). - -## Impact - -- **Code (MLX-free Core, `AI/Handoff/`):** new pure `ClaudeHandoffConfig`/`HandoffConfirmMode` value types; `HandoffBudget` (pure rate/budget gate over a ledger, `now:`-injected); `HandoffLauncher` seam (the side-effecting spawn, behind a protocol so the contributor is headless-testable); `ClaudeHandoffContributor` (the `ToolContributor` projecting `launch_claude` + resolving the effective tier + budget check + audit emission). A new `enum HandoffError: Error, Equatable, LocalizedError` ONLY for handoff-specific failures `RuntimeError`/`TaskError` cannot carry (over-budget, handoff disabled, missing folder). -- **Reuse, not rebuild:** `ClaudeLauncher.commandScript`/`writeCommandFile`/`resolveClaudePath` + the `NSWorkspace.shared.open(url)` `.command` handoff (the open-claude-here launch path) — the production `HandoffLauncher` is a thin adapter over these, unchanged. `ToolDescriptor`/`ToolRoute`/`ToolStepResult`/`WritePolicyTier`/`ToolContributor`/`ApprovalGate` (consumed, owned by `ai-tool-routing`). `AuditRecord`/`AuditLog`/`WritePolicyResolving` (consumed, owned by `ai-background-autonomy`). `ClaudeHandoffConfig` rides the `SkillManifest`'s optional `claudeHandoff` block (the carrier is owned by `ai-skills-as-files`; this slice OWNS the type's definition and semantics). `AIError.message(for:)` extended to translate `HandoffError` (the one translator). `ParkScheduler.escalate` (consumed) for the parked needs-you path. -- **MLX-free Core, `swift test`-verified:** the config types, the budget/rate cap, the effective-tier resolution, the audit emission, and the contributor's `run` branches (auto-under-budget → launch; confirm → await gate; over-budget → degrade/decline) are pure or stub-driven and verify under `swift build`/`swift test` with a fake `HandoffLauncher` + scripted gate + fake `AuditLog`. The production `HandoffLauncher` (the real `NSWorkspace`/`Process` spawn) is in Core too (the open-claude-here launch code already lives in Core) but its side effect is exercised by the **user's** stable-signed build end-to-end. No GemmaRuntime/MLX code in this slice; the full app links via `xcodebuild` compile-verify only. -- **Out of scope (v1):** capturing or parsing Claude Code's output; resuming the local conversation from Claude's result (the round-trip — documented future, seam left); a Hub UI for the global budget (a persisted `claudeHandoffBudgetPerDay` key is defined; surfacing it is a small Hub follow-up); choosing *which* Claude model / passing Claude CLI flags beyond the starting prompt; any non-Claude escalation backend. diff --git a/openspec/changes/ai-claude-handoff/specs/ai-claude-handoff/spec.md b/openspec/changes/ai-claude-handoff/specs/ai-claude-handoff/spec.md deleted file mode 100644 index df97a7d..0000000 --- a/openspec/changes/ai-claude-handoff/specs/ai-claude-handoff/spec.md +++ /dev/null @@ -1,113 +0,0 @@ -## ADDED Requirements - -### Requirement: Model-callable Claude Code handoff tool -The system SHALL provide a `launch_claude` capability the router can select like any other tool, described by a tool descriptor whose arguments are a starting **prompt** and an optional **folder**. When selected and permitted, the handoff SHALL open Claude Code in that folder with that starting prompt, reusing the existing no-new-permission terminal handoff (the self-deleting `.command` file opened through the system default handler) and SHALL NOT introduce a new launch mechanism or request any new permission (in particular, no Apple Events / Automation grant). A non-empty prompt SHALL start Claude with that prompt as its input; an empty prompt SHALL open a bare Claude session in the folder. When the route omits a folder, the handoff SHALL use the skill's configured default working directory; when neither is available, it SHALL surface a clean, bounded failure and SHALL NOT spawn anything. - -#### Scenario: Router selects the handoff and it opens Claude with the prompt -- **WHEN** the router selects `launch_claude` with a folder and a prompt, and the call is permitted -- **THEN** Claude Code opens in that folder started with that prompt, via the existing self-deleting `.command` handoff, requiring no new permission - -#### Scenario: Empty prompt opens a bare session -- **WHEN** `launch_claude` is selected with a folder and an empty prompt -- **THEN** a bare Claude session opens in that folder - -#### Scenario: Missing folder fails cleanly without spawning -- **WHEN** `launch_claude` is selected with no folder and the skill has no default working directory -- **THEN** nothing is spawned and a clean, bounded failure is surfaced, and no Claude budget is spent - -### Requirement: Claude handoff is a dangerous-tier action defaulting to per-call confirmation -The `launch_claude` tool descriptor SHALL always carry the **dangerous** write-policy tier, and the effective gate for a call SHALL be the descriptor tier intersected with the user's whitelist, then narrowed by the skill's confirm mode. The handoff SHALL default to **confirm** (foreground approval per call), because real money is spent. A skill MAY opt into **auto** explicitly in its handoff configuration; the auto opt-in SHALL only downgrade within what the user whitelist permits and SHALL NEVER override a user who has not trusted handoff. Confirmation SHALL reuse the canonical approval gesture (DOWN = approve, RIGHT = skip) when the session is active. - -#### Scenario: Handoff defaults to confirm -- **WHEN** a skill triggers `launch_claude` without an explicit confirm mode -- **THEN** the handoff requires a foreground per-call approval before Claude is opened - -#### Scenario: A skill opts into auto -- **WHEN** a skill explicitly sets its handoff confirm mode to auto and the user has whitelisted handoff -- **THEN** the handoff runs without a per-call approval (subject to the budget cap and still audited) - -#### Scenario: User whitelist overrides a skill's auto -- **WHEN** a skill sets auto but the user has NOT whitelisted Claude handoff -- **THEN** the effective tier stays dangerous and the handoff requires foreground approval (or escalation), regardless of the skill's auto - -#### Scenario: Approve and skip use the canonical gesture -- **WHEN** a confirm-mode handoff awaits the user in an active session -- **THEN** a DOWN gesture approves and opens Claude, and a RIGHT gesture skips the handoff with no spend - -### Requirement: Budget and rate cap on Claude handoffs -Even an auto handoff SHALL be capped so an autonomous agent loop cannot rack up real spend: the system SHALL enforce a maximum number of handoffs per rolling 24-hour window and a maximum number of concurrent in-flight handoffs, over an append-only spend ledger that survives a relaunch within the window. The cap SHALL use a rolling window keyed off an injected current time, NOT a calendar-day reset. A per-skill cap MAY tighten the cap; a per-skill cap of zero SHALL fall back to the global default. A handoff whose launch fails after a spend was recorded SHALL refund that spend so the cap stays honest. An over-budget call SHALL NEVER be silently dropped. - -#### Scenario: Under the cap the handoff is allowed -- **WHEN** the number of handoffs in the last 24 hours is below the cap and concurrency is below its limit -- **THEN** the handoff is allowed to proceed (subject to its confirm/auto gate) - -#### Scenario: Rolling window cannot be gamed across midnight -- **WHEN** handoffs are spread across a calendar-day boundary but fall within the same rolling 24-hour window -- **THEN** they are counted together against the cap, with no midnight reset - -#### Scenario: Cap survives a relaunch -- **WHEN** the process restarts within the rolling window -- **THEN** prior handoffs inside the window still count against the cap and the budget is not reset - -#### Scenario: A failed launch refunds its spend -- **WHEN** a handoff records a spend but its launch then fails -- **THEN** the spend is refunded and the in-flight count is decremented, leaving the cap unchanged - -### Requirement: Over-budget auto handoffs degrade, never run unprompted -When an auto handoff is over the budget cap, it SHALL degrade to a foreground per-call confirmation (in an active session) rather than running unprompted, and SHALL escalate to the needs-you badge (in a parked session) — it SHALL NOT auto-run over budget and SHALL NOT be silently dropped. The user, never the loop, SHALL be the only authority that can spend over the cap. The approval surface SHALL indicate that the budget cap has been reached. - -#### Scenario: Auto over budget degrades to confirm -- **WHEN** an auto handoff is requested but the daily cap has been reached and the session is active -- **THEN** the handoff degrades to a foreground confirmation that states the budget cap was reached, and it does not auto-run - -#### Scenario: Auto over budget in a parked session escalates -- **WHEN** an auto handoff is over budget and the session is parked -- **THEN** the handoff escalates to a needs-you badge for the user to decide, and no spend occurs until the user returns and approves - -### Requirement: Every Claude handoff is audited -Every handoff attempt — auto, confirmed, declined, failed, or over-budget — SHALL emit exactly one append-only audit record naming the tool, the dangerous policy tier, a redacted/short argument summary (the folder and a truncated prompt, NEVER the full prompt verbatim), the outcome, whether it ran in the background, and a timestamp. The audit SHALL route into the single shared audit log rather than a separate handoff-only log. Raw prompt text SHALL appear only in logs or behind an opt-in details disclosure, never in the audit summary or any headline. - -#### Scenario: A handoff records one audit entry -- **WHEN** any `launch_claude` attempt resolves (done, declined, failed, or over-budget) -- **THEN** exactly one audit record is appended naming the tool, the dangerous tier, the redacted argument summary, the outcome, the background flag, and a timestamp - -#### Scenario: The audit summary never carries the full prompt -- **WHEN** an audit record for a handoff is inspected -- **THEN** its argument summary contains the folder and at most a truncated prompt, and the full prompt text is not present - -### Requirement: Dangerous handoff escalates from a parked session -A dangerous handoff that needs approval SHALL NOT auto-run and SHALL NOT silently wait inside a parked session — it SHALL raise the needs-you state with a badge so the user is pulled back to approve the spend. An auto handoff that is under budget and explicitly trusted MAY run in the background while parked (still audited), mirroring the autonomy rule that whitelisted/auto writes run when parked while dangerous writes escalate. - -#### Scenario: Confirm-mode handoff while parked escalates -- **WHEN** a confirm-mode (or whitelist-dangerous) handoff is requested while the session is parked -- **THEN** the session raises a needs-you badge for the pending handoff and no Claude is opened and no spend occurs until the user returns and approves - -#### Scenario: Auto under-budget handoff runs while parked -- **WHEN** an auto handoff that is under budget and within the user whitelist is requested while the session is parked -- **THEN** it opens Claude in the background and is audited, without pulling the user back - -### Requirement: V1 handoff is fire-and-forget with a documented round-trip future -Version 1 of the handoff SHALL be fire-and-forget: it SHALL open Claude with the prompt and consider the handoff complete, returning a done result the local agent uses to wind down its own loop, and SHALL NOT capture or consume Claude Code's output. The system SHALL leave a seam so a future version can perform the structured round-trip (consume Claude's result and resume the local conversation) as an additive change, without reworking the handoff tool or the cost gate. - -#### Scenario: V1 opens Claude and completes the step -- **WHEN** a permitted handoff launches in v1 -- **THEN** the step result is a done outcome summarizing that Claude opened in the folder, and the local agent does not wait for or read Claude's output - -#### Scenario: The round-trip is a documented future, not built in v1 -- **WHEN** the handoff design is reviewed -- **THEN** v1 does not consume Claude's output, and the seam for a future round-trip (consume the result and resume the conversation) is documented as an additive future - -### Requirement: Bounded, non-blocking handoff failure surfacing -Handoff failures SHALL map at the boundary into the handoff error taxonomy and surface bounded and non-blocking — never via an app-modal alert, and never with raw OS or vendor error text in a headline (raw text is allowed only in logs or behind an opt-in details disclosure). A failed launch SHALL become an observable failed step carrying a clean headline (never a false "Done"), and the corresponding spend SHALL be refunded. A handoff disabled for a skill SHALL surface as a clean declined result. A user discard during approval SHALL end the loop quietly with no spend and SHALL NOT be reported as a failure. - -#### Scenario: A failed launch surfaces a clean failed step -- **WHEN** opening Claude fails (e.g. the terminal could not be opened or the command file could not be written) -- **THEN** the step is reported failed with a clean headline, the spend is refunded, and no app-modal alert appears - -#### Scenario: No raw error text in a headline -- **WHEN** any handoff failure is presented -- **THEN** the headline is a clean, human-readable message and any raw OS/vendor error text appears only in logs or behind an opt-in details disclosure - -#### Scenario: Discard during approval is quiet, not a failure -- **WHEN** the user discards the canvas while a handoff awaits approval -- **THEN** the loop ends quietly, no Claude is opened, no spend occurs, and it is not reported as a failure diff --git a/openspec/changes/ai-claude-handoff/tasks.md b/openspec/changes/ai-claude-handoff/tasks.md deleted file mode 100644 index 74e1bc5..0000000 --- a/openspec/changes/ai-claude-handoff/tasks.md +++ /dev/null @@ -1,44 +0,0 @@ -> Decomposed for a workflow fan-out: §1 is the type substrate (do first), §2 the budget gate, §3 the launcher adapter, §4 the contributor state machine (depends on §1–§3), §5 the cross-slice seams, §6 errors, §7 verifies. Every item is MLX-free Core verified by `swift test` unless noted; NO `.app` build, NO signing. - -## 1. Handoff config types (pure Core type substrate) - -- [x] 1.1 `HandoffConfirmMode` (`confirm`/`auto`) + `ClaudeHandoffConfig` already on disk at `AI/Skills/ClaudeHandoffConfig.swift` (defined by skills-as-files, OWNED by this slice). CONSUMED verbatim per SCOPE (do not redefine). Its shape is `{confirmMode, folder, startingPrompt, maxPerDay}`; `confirmMode` defaults to `.confirm`. *Verify: `swift test` `ClaudeHandoffTests.testConfigDefaultsToConfirm` — default == confirm + Codable round-trip.* -- [x] 1.2 Doc-comment in the existing type already states it is the SAME type carried on `SkillManifest.claudeHandoff` (single definition in Core). *Verified against the on-disk file header.* - -## 2. Budget / rate cap (pure, `now:`-injected) - -- [x] 2.1 Added `AI/Handoff/HandoffBudget.swift`: `HandoffSpend{at, skillID}`, `HandoffBudget{maxCallsPerDay, maxConcurrent, ledger, inFlight}` with PURE `allows(now:)`, `record(at:skillID:)`, `reap()`, `refund(at:)`. `now` is an INPUT. *Verify: `swift test` — `testBudgetAllowsUnderCapBlocksAtCap`, `testBudgetConcurrencyBlocks`, `testBudgetRefundRestoresSlot`, `testBudgetRollingWindowNotCalendarDay`.* -- [x] 2.2 Rolling-24h, not calendar-day. *Verify: `swift test` `testBudgetMidnightCannotBeGamed` — two `now`s straddling midnight count together, no reset.* -- [x] 2.3 Ledger persistence seam: `HandoffLedgerStore` (+ `InMemoryHandoffLedgerStore` / `DiskHandoffLedgerStore` under Application Support); `HandoffBudgetBox` seeds the ledger from the store at init + re-saves on record/refund. *Verify: `swift test` `testBudgetPersistsAcrossReload` — a fresh box over the same store still counts prior spends in the window.* - -## 3. Production launcher (the open-claude-here adapter) - -- [x] 3.1 Added `AI/Handoff/HandoffLauncher.swift`: the `HandoffLauncher` protocol + `OpenClaudeHandoffLauncher` composing `ClaudeLauncher.shellQuote`/`writeCommandFile`/`resolveClaudePath` + `NSWorkspace.shared.open` (off-main write, main-actor open, success-needs-no-notification). Non-empty prompt → `claude ''`; empty → bare `claude`. `Launcher/ClaudeLaunch.swift` UNCHANGED. *Verify: `swift test` `testInnerCommandEmptyVsNonEmpty`; the real spawn is **user-build** only.* -- [x] 3.2 `ClaudeLaunchError` → `HandoffError.launchFailed` mapped at the launch boundary (`OpenClaudeHandoffLauncher.map`). *Verify: `swift test` `testLaunchErrorMapping` — clean headline through, raw text only in details.* - -## 4. The contributor — `launch_claude` tool + cost-gate state machine - -- [x] 4.1 Added `AI/Handoff/ClaudeHandoffContributor.swift` `descriptors()`: a single `ToolDescriptor{name:"launch_claude", argsSchema:{folder?, prompt}, writePolicy:.dangerous, keywords}` — ALWAYS `.dangerous`. `canHandle("launch_claude") == true`. *Verify: `swift test` `testDescriptorIsAlwaysDangerous`, `testCanHandle`.* -- [x] 4.2 `effectiveGate()`: 0 resolved cap → `.disabled`; resolver intersects `.dangerous` ∩ whitelist FIRST (non-whitelisted stays foreground regardless of skill auto); `!budget.allows(now)` → `.overBudget`; else `.auto`→`.autoRun`, `.confirm`→`.needsApproval`. (NB: the on-disk `ClaudeHandoffConfig` has no `enabled` field; "disabled" is modeled as a resolved cap of 0 — global default 0 → declined.) *Verify: `swift test` `testAutoSkillUnderBudgetRunsWithoutGate`, `testAutoSkillRequiresWhitelistToRunUnprompted`, `testDisabledWhenCapZero`.* -- [x] 4.3 `run(_:gate:)` state machine: `.autoRun`→record+audit+launch→`.done` (throw→`.failed`+refund); `.needsApproval`→await gate (approve→launch, skip→`.declined("skipped")` no spend, cancel→quiet loop end); `.overBudget`→degrade to confirm (active) / escalate needs-you (parked); `.disabled`→`.declined`. *Verify: `swift test` `testConfirmSkillGatesAndApprovesLaunchesOnce`, `testConfirmSkillSkipDoesNotSpendOrLaunch`, `testAutoOverBudgetEscalatesToConfirm`, `testCancelEndsQuietlyNotAFailure`.* -- [x] 4.4 Folder resolution: route `folder` → else `config.folder` → else `.failed(HandoffError.missingFolder)`, no spend. Empty prompt allowed (bare session). *Verify: `swift test` `testMissingFolderFailsCleanlyNoSpend`, `testEmptyPromptLaunchesBareSession`.* -- [x] 4.5 Refund-on-throw invariant. *Verify: `swift test` `testFailedLaunchRefundsSpend` — failed launch leaves the budget unchanged + `inFlight == 0`.* - -## 5. Cross-slice seams (compile-in-isolation today, bind later) - -- [x] 5.1 Added `AI/Handoff/HandoffSeams.swift`: `HandoffAuditing` (+ `NoopHandoffAudit` / `AuditLogHandoffAuditing` bridging the real `AuditLog`) + `HandoffEscalating` (+ `NoopHandoffEscalation` / `ParkSchedulerHandoffEscalation` bridging the real `ParkScheduler.escalate`). NB: `ai-background-autonomy` + `ai-parked-sessions` ARE on disk (the design's "not yet on disk" note is stale), so the production bridges are wired to the real types now, not deferred. *Verify: `swift test` with recording doubles.* -- [x] 5.2 One `AuditRecord{sessionID, tool:"launch_claude", policy:.dangerous, argumentsSummary: folder + truncated prompt (via `AuditRedaction`), outcome, wasBackground, timestamp}` per `run`, all branches. *Verify: `swift test` `testAuditNeverCarriesFullPrompt`, `testEveryBranchAuditsExactlyOnce` — exactly one record per run, no full prompt/secret in the summary.* -- [x] 5.3 Parked escalation: a `.needsApproval`/`.overBudget` handoff while parked routes to `escalation.escalate(...)` (needs-you, `.awaitingApproval`, no spend); `.autoRun` under budget runs in the background, still audited. *Verify: `swift test` `testConfirmWhileParkedEscalates`, `testAutoOverBudgetInParkedSessionEscalatesNeedsYou`, `testAutoUnderBudgetRunsWhileParked`.* - -## 6. Errors — one taxonomy, one translator - -- [x] 6.1 Added `AI/Handoff/HandoffError.swift`: `HandoffError` (`disabled`/`overBudgetNoUser`/`missingFolder`/`launchFailed(headline, details)`), `LocalizedError` with clean per-case `errorDescription`; raw text only in `copyableDetails`/logs. Extended `AIError.message(for:)` with a `HandoffError` branch (THE one translator). *Verify: `swift test` `testHandoffErrorTranslatorCleanHeadlines`, `testLaunchErrorMapping`.* -- [x] 6.2 A failed/over-budget handoff is an observable `ToolStepResult(.failed/.declined)` — never a false Done, never silence, never `NSAlert`; cancel is not a failure. *Verify: `swift test` `testFailedLaunchRefundsSpend`, `testCancelEndsQuietlyNotAFailure`.* - -## 7. Verify - -- [x] 7.1 `swift build --target ThreeFingerSwitcherCore` + `swift test` green (1230 tests, 0 failures; 27 in `ClaudeHandoffTests`) — config, budget, launcher mapping, contributor state machine, audit redaction, parked escalation all covered with fake launcher + scripted gate + recording audit/escalation. *Verify: `swift test`.* -- [x] 7.2 `Launcher/ClaudeLaunch.swift` / `Launcher/LaunchService.swift` UNMODIFIED. *Verify: `git diff --stat` shows no changes to those files.* -- [x] 7.3 `openspec validate ai-claude-handoff --strict` passes. *Verify below.* -- [x] 7.4 No `.app` build, no signing, no permission change — `swift test` only; the real `claude` spawn is exercised by the user's stable-signed build. *Verified.* -- [ ] 7.5 **User run-verify** (later, after the router + canvas + autonomy land, in a stable-signed build): a conversational ask that exceeds the local model routes to `launch_claude`; a default-confirm handoff waits for a DOWN approve / RIGHT skip and opens Claude in the folder with the prompt; an `auto` skill under budget opens Claude without a prompt; over the daily cap an `auto` handoff degrades to a foreground confirm (never silent); a parked session shows a needs-you badge for a pending dangerous handoff; the audit log shows every handoff attempt. *(Deferred — depends on downstream slices.)* diff --git a/openspec/changes/ai-compute-tiers/.openspec.yaml b/openspec/changes/ai-compute-tiers/.openspec.yaml deleted file mode 100644 index a4ac4d7..0000000 --- a/openspec/changes/ai-compute-tiers/.openspec.yaml +++ /dev/null @@ -1,2 +0,0 @@ -schema: spec-driven -created: 2026-06-23 diff --git a/openspec/changes/ai-compute-tiers/design.md b/openspec/changes/ai-compute-tiers/design.md deleted file mode 100644 index 562cf10..0000000 --- a/openspec/changes/ai-compute-tiers/design.md +++ /dev/null @@ -1,73 +0,0 @@ -# Design — ai-compute-tiers - -## Context - -The batched GPU runtime (`ai-batched-runtime-and-context`, blueprint §3.6) is the v2 concurrency primitive: K conversation streams folded into ONE GPU forward pass per decode step so the weights are read once. It is correct and stays as-is — but it is single-lane. Every unit of agent work (foreground reply, router `structured()` turn, classify, memory retrieval, parked-subagent advance) competes for the same GPU decode loop and the same ~153 GB/s unified-memory bus. - -This slice introduces a **second compute lane** so the light, frequent, structured work can run **off** the GPU, **concurrently** with the heavy reply, on a small ternary/BitNet-class model whose weights are ~32× smaller and therefore barely touch the shared bus. It owns the addendum §A1 contracts (`ComputeLane`, `AgentWorkRole`, `LaneRouting`) and amends the implicit single-GPU assumption on `on-device-ai-runtime` — **via MODIFIED requirements only**, never by editing the batched or parked slice files. The CPU ternary model is **another `LLMRuntime` conformer** (reuse the seam, do not invent a protocol), selected by lane through the fleet registry (addendum §C1). - -### Measured M5 facts this slice rests on (cited honestly) - -- **Prefill ≈ ~4× on the GPU neural accelerators.** M5's matmul accelerators give up to ~4× prompt-processing/prefill over the prior generation → the GPU is where prefill-heavy heavy generation and media diffusion belong. -- **Token-gen is bandwidth-bound at ~153 GB/s.** Decode reads the weight set per token; the bus, not the cores, is the ceiling. Two heavy generations on one lane contend for that one bus. -- **Ternary weights ≈ ~32× smaller.** A ternary/BitNet-class model's weights read per token cost a fraction of the bandwidth → a CPU-lane ternary decode runs concurrently with a GPU generation with low bus contention. -- **CPU per-token is slower (the honest limit).** A CPU ternary decode is slower per token than a GPU batched token, so the CPU lane is for short/frequent/structured bursts ONLY — never the long foreground reply. - -## Goals - -- Own `ComputeLane` / `AgentWorkRole` / `LaneRouting` (§A1) verbatim, and a pure role→lane policy that is `swift test`-verified. -- Add a second `LLMRuntime` conformer (`TernaryCPURuntime`) on the CPU lane — a conformer, not a protocol — with a deterministic stub for tests. -- Let the GPU lane and the CPU ternary lane run **concurrently** under a pure residency/co-residency budget, without the CPU lane starving the foreground GPU reply or borrowing GPU batch slots. -- Make a `parkedSubagent` session advance on the CPU lane **at the same time** as a foreground GPU generation, via an **additive** lane-affinity hint consumed by `ParkScheduler` (§3.5) and the batched runtime (§3.6) through their pinned shapes. -- AMEND the single-GPU assumption as MODIFIED requirements on `on-device-ai-runtime`. - -## Non-Goals - -- The GPU batched decode loop, KV-quant, prefix caching, context tuning — `ai-batched-runtime-and-context` (the GPU lane) owns these; this slice does not touch them. -- The parked store, notch rail, park/sleep/discard lifecycle, the `ParkScheduler` policy itself — `ai-parked-sessions` owns these; this slice only attaches an additive hint to its runnable set. -- The model registry, residency **eviction policy**, and cloud members — `ai-model-fleet` (§C1) owns these; this slice supplies the lane concept it consumes and the ternary co-residency MATH. -- The master toggle UI / cost-disclosure Hub page — `ai-full-potential-toggle` (§D1) owns these; this slice only reads `cpuLaneEnabled`/`fullPotentialEnabled`. -- Putting the long foreground reply on the CPU lane (it is slower per token — explicitly rejected). -- Any low-end/degraded fallback — the CPU lane is a bandwidth optimization for capable hardware, NOT a weak-hardware path. - -## Decisions - -### D1. `ComputeLane` + `AgentWorkRole` + `LaneRouting` are Core, verbatim from §A1; this slice OWNS them. -Define `ComputeLane { .gpu, .cpuTernary }`, `AgentWorkRole { foregroundGeneration, mediaDiffusion, toolRoute, classify, memoryRetrieval, parkedSubagent }`, and `protocol LaneRouting { func lane(for: AgentWorkRole) -> ComputeLane }` exactly as the addendum pins them. The concrete `DefaultLaneRouting` is a pure total function: `foregroundGeneration`/`mediaDiffusion` → `.gpu`; `toolRoute`/`classify`/`memoryRetrieval`/`parkedSubagent` → `.cpuTernary`. No state, time as no input needed — a switch. **Rationale:** a pure total map is trivially `swift test`-exhaustive (one assertion per case) and is the single source of truth every consumer (scheduler, batched runtime, executor) reads, so lane assignment can never drift between call sites. **Alternatives rejected:** (a) a `lane` field hand-set at each call site — drifts, untestable as a whole; (b) deriving the lane from the model descriptor alone — the SAME ternary model could in principle serve a GPU role, so role, not model, must drive the lane. - -### D2. The CPU ternary model is ANOTHER `LLMRuntime` conformer, never a new protocol. -`TernaryCPURuntime` conforms to the existing `LLMRuntime` (`generate`, `structured`, `capabilities`, `chat()` default). It carries a SMALL ternary/BitNet-class model on the CPU lane. Feature code stays model-agnostic (the band, executor, router already depend only on `LLMRuntime`); selection is by **lane**, resolved through the fleet registry (§C1: `role: .ternaryChat`, `lane: .cpuTernary`). **Rationale:** blueprint house rule — reuse the model seam; a second protocol would fork the abstraction the whole feature depends on and force every consumer to branch on runtime type. **Alternatives rejected:** (a) a `CPURuntime`/`TernaryRuntime` protocol — forks `LLMRuntime`, violates §A1's explicit "NOT a new protocol"; (b) a mode flag on the GPU runtime — conflates two physically distinct lanes in one conformer and one residency footprint. - -### D3. A pure `LaneArbiter` over a `LaneResidencyBudget` decides cross-lane concurrency; the GPU reply is never blocked by CPU work. -`LaneResidencyBudget` (pure, free-memory injected — no Metal) holds: chat weights (GPU, read once), KV per GPU stream, and the **ternary residency bytes** (a small constant — ~32× smaller weights). It answers "does the ternary model co-reside with the current GPU batch + KV under the 48 GB budget?" — and on this hardware it does, cheaply. `LaneArbiter` then admits CPU-lane work **concurrently** with GPU work, bounding CPU-lane concurrency on its OWN budget (a small fixed CPU-lane stream cap), and enforces two honest invariants: (1) a heavy GPU generation is NEVER made to wait on CPU-lane work; (2) CPU-lane bursts NEVER preempt or starve the foreground GPU reply (the CPU lane is bandwidth-frugal, so it doesn't need to). `now:` and free-memory are inputs (mirrors `DockHoverModel`/`ConcurrencyBudget`), so it is deterministically testable. **Rationale:** the two lanes are physically independent (GPU cores + accelerators vs CPU cores), and the ternary weights' tiny bandwidth footprint is exactly what makes true concurrency — not a second queue — correct; making the budget pure keeps the decision `swift test`-able without real Metal. **Alternatives rejected:** (a) let the CPU lane borrow GPU batch slots — defeats the purpose (it would re-serialize behind the GPU loop); (b) a global mutex across both lanes — re-serializes the very work we split out; (c) preempt the GPU reply for an urgent CPU burst — unnecessary (CPU work doesn't contend) and would stutter the visible answer. - -### D4. The lane-affinity hint is consumed ADDITIVELY by `ParkScheduler` (§3.5) and the batched runtime (§3.6); their pinned shapes are untouched. -Add a Core `LaneAffinity` value (`{ sessionID: AgentSessionID, lane: ComputeLane }`, derived from the session's `AgentWorkRole` via `DefaultLaneRouting`). The parked scheduler attaches an affinity to each session it returns from `runnableSessions(now:maxSlots:)` **without changing that method's signature** (the hint is read alongside the returned IDs through an additive accessor, not by mutating the pinned method). The dispatcher (the batched runtime + the lane arbiter) reads the affinity and dispatches `cpuTernary`-affined sessions to `TernaryCPURuntime` while the GPU `batchStep(...)` keeps serving `.gpu`-affined and foreground sessions. Net effect: a `parkedSubagent` session advances on the CPU lane **concurrently** with the foreground GPU generation. **This is specified as MODIFIED requirements on `on-device-ai-runtime`; the `ai-parked-sessions` and `ai-batched-runtime-and-context` change files are NOT edited.** **Rationale:** the addendum mandates additive consumption ("`runnableSessions(now:maxSlots:)` signature unchanged"; "`batchStep` … additive"); an additive accessor + a value hint keeps both seams as pinned while delivering the concurrency win. **Alternatives rejected:** (a) add a `lane:` parameter to `runnableSessions`/`batchStep` — breaks the pinned signatures and forces sibling-slice edits; (b) split the scheduler into two schedulers — duplicates the FIFO/needs-you/`nextRunAt` policy `ai-parked-sessions` owns. - -### D5. Honest constraint: the CPU lane serves short/frequent/structured bursts ONLY — never the long reply. -`DefaultLaneRouting` routes `foregroundGeneration` and `mediaDiffusion` to `.gpu` precisely because CPU per-token is slower. The router `structured()` turn, classify, memory retrieval, and parked-subagent advances are short and bounded, so even at a slower per-token rate they finish quickly and gain from running concurrently rather than queuing. The spec states this limit outright (a scenario asserts the long reply is GPU-only). **Rationale:** honesty about cost is a binding house rule; mis-routing the long reply to the CPU lane would be a regression, not a feature. **Alternatives rejected:** a "spill heavy work to CPU when GPU is busy" path — slower for the user and contradicts the bandwidth argument; the GPU's batched loop is already the heavy-work answer. - -### D6. Gated by `cpuLaneEnabled` under `fullPotentialEnabled`; OFF = today's one-lane behavior, no regression. -The CPU lane checks `cpuLaneEnabled` (read from §D1, owned by `ai-full-potential-toggle`) before installing `TernaryCPURuntime`. When off, `LaneRouting` still answers, but the dispatcher coerces every lane to `.gpu` (a `LaneRouting` decorator), so all work routes to the GPU batched runtime exactly as before — a one-lane / fleet-of-one build stays valid. **Rationale:** addendum decision 6 (default OFF, ships calm; every sub-capability discloses its cost — here a small extra resident ternary model + CPU heat under sustained bursts). **Alternatives rejected:** always-on CPU lane — violates the default-OFF master-gate decision and spends RAM/heat the user didn't ask for. - -### D7. Errors stay in `RuntimeError`; a `ComputeError` only if lane dispatch genuinely needs its own case. -A CPU-lane failure (ternary load/prepare/decode) maps at the `TernaryCPURuntime` boundary into `RuntimeError` (the existing taxonomy), surfaced via `AIError.message(for:)` as a clean headline — exactly like the GPU runtime. A `ComputeError` LocalizedError is added ONLY if a lane-dispatch case (e.g. "selected lane unavailable") cannot be carried by `RuntimeError`; prefer extending `RuntimeError`. A CPU-lane step that fails is an observable `.failed` for that turn (never a false "done"), never an `NSAlert`, never raw text in a headline. **Rationale:** one taxonomy + one translator is a binding invariant; a per-stream/per-turn failed state mirrors the batched slice's "one stream's failure does not abort the batch." **Alternatives rejected:** a standalone `ComputeError` for every case — proliferates error types against the house rule. - -## Target-split & verification (per component) - -| Component | Target | Verification | -|---|---|---| -| `ComputeLane`, `AgentWorkRole` (§A1) | MLX-free Core | `swift test` — enums construct, round-trip `Codable`; OWNED here verbatim | -| `LaneRouting` protocol + `DefaultLaneRouting` policy (D1) | MLX-free Core | `swift test` — exhaustive role→lane assertion per case (heavy→`.gpu`, structured→`.cpuTernary`) | -| `LaneAffinity` hint value (D4) | MLX-free Core | `swift test` — derived from role via `DefaultLaneRouting`; a `parkedSubagent` session yields `.cpuTernary` | -| `LaneResidencyBudget` co-residency math (D3) | MLX-free Core | `swift test` — ternary (~32× smaller) co-resides with chat weights + GPU KV under an injected free-memory budget; fits where a second chat model would not | -| `LaneArbiter` cross-lane concurrency (D3) | MLX-free Core | `swift test` (`now:`/free-mem injected) — GPU + CPU admit concurrently; a heavy GPU gen never waits on CPU work; CPU bursts never starve the foreground GPU reply; CPU-lane cap bounds CPU concurrency | -| `LaneRouting` OFF-coercion decorator (D6) | MLX-free Core | `swift test` — with `cpuLaneEnabled == false` every role coerces to `.gpu` (one-lane build stays valid) | -| `StubTernaryRuntime` (test-only `LLMRuntime` conformer) | MLX-free Core | `swift test` — scripts deterministic tokens/structured outcomes; de-mux to the right `AgentSessionID`; verifies the seam without real weights | -| Error mapping → `RuntimeError` / `AIError.message(for:)` (D7) | MLX-free Core | `swift test` — a simulated CPU-lane failure maps to a `RuntimeError` case → a clean `AIPresentedError` headline; never raw text | -| `TernaryCPURuntime` (real bitnet.cpp-class ternary `LLMRuntime` conformer on the CPU lane) | Native-linked (`GemmaRuntime`/sibling framework) | `xcodebuild` COMPILE-VERIFY ONLY; real per-token speed, true CPU/GPU concurrency, no cross-lane bleed, RAM/heat → **user's stable-signed build** (an agent never builds/signs the `.app`) | -| Lane-keyed `ModelProvisioner`/`runtimeFactory` wiring (D2) | Native-linked (`GemmaRuntime`) | `xcodebuild` compile; **user run-verify** the ternary conformer is injected for `lane: .cpuTernary` with no `ModelManager` API change | -| Additive lane-affinity dispatch into the batched loop (D4) | Native-linked (`GemmaRuntime`) | `xcodebuild` compile; **user run-verify** a parked subagent runs on CPU concurrently with a foreground GPU generation | -| `cpuLaneEnabled`/`fullPotentialEnabled` gate read (D6) | MLX-free Core (read) + native install gate | `swift test` for the gate logic; **user run-verify** OFF installs no CPU lane and routes all work to the GPU | - -> The metallib `*.bundle` → `Contents/Resources/` copy in `build-app.sh` must not regress; the agent compile-verifies the native conformer only — live two-lane concurrency, CPU per-token speed, heat, and RAM headroom are validated solely by the user's stable-signed build. diff --git a/openspec/changes/ai-compute-tiers/proposal.md b/openspec/changes/ai-compute-tiers/proposal.md deleted file mode 100644 index cc545da..0000000 --- a/openspec/changes/ai-compute-tiers/proposal.md +++ /dev/null @@ -1,34 +0,0 @@ -## Why - -The V2 batched runtime (`ai-batched-runtime-and-context`) folds K conversation streams into **one** GPU forward pass per decode step. That is the right answer for concurrency over a single weight read, but it makes one quiet assumption: **everything runs on the GPU lane.** On the target machine (M5 Pro, 16-core GPU, 48 GB unified) that means the foreground reply, the structured router turn, every cheap classification, every memory-TOC retrieval, and every parked subagent advance all queue behind the **same** GPU decode loop and the same ~153 GB/s memory bus. The heavy reply is what the user is watching; the light structured bursts are what the agent does constantly. Serializing the clerks behind the craftsman is the bottleneck this slice removes. - -The physics make a second lane affordable on M5 — and honest about its limits: - -1. **Prefill is compute-bound and the GPU neural accelerators win it.** M5's matmul accelerators give up to **~4× prompt-processing/prefill** over the prior generation. Long shared prefixes (system + skills TOC + memory TOC) are prefill-heavy, so the GPU is exactly where the heavy reply and media diffusion belong. -2. **Token-gen is bandwidth-bound, not compute-bound.** Each decode step reads the model weights from unified memory to produce one token per stream; the ceiling is the **~153 GB/s** bus, not the cores. Two heavy generations on one lane contend for that one bus. -3. **Ternary weights are ~32× smaller, so they barely touch the bus.** A small ternary/BitNet-class model's weights are roughly **32×** smaller than an FP16/Q4 chat model's. Reading them per token costs a tiny fraction of the bandwidth, so a CPU-lane ternary decode runs **concurrently** with a GPU generation **without** meaningfully stealing the bandwidth the GPU reply needs. Low contention on the shared bus is the whole reason a second lane is a win and not just a second queue. -4. **Be honest: CPU per-token is slower.** A CPU ternary decode produces tokens slower than the GPU produces a batched token. So the CPU lane is for **short, frequent, structured bursts** — the router's `structured()` turn, should-park / which-skill classification, memory retrieval, and parked-subagent advances — and **never** the long foreground reply. Putting the long reply on the CPU lane would be slower and is explicitly out of scope. - -The win is concrete: a **parked subagent advances on the CPU ternary lane CONCURRENTLY with a foreground GPU generation**, instead of waiting for a GPU batch slot. The router can decide the next tool while the GPU is still streaming the visible answer. None of this forks a seam: the CPU ternary model is **another `LLMRuntime` conformer** selected by lane (not a new protocol), the GPU lane stays the batched runtime untouched, and the lane-affinity hint is consumed **additively** by `ParkScheduler` and the batched runtime through their existing shapes. - -## What Changes - -- **A two-lane compute model (`ComputeLane`, addendum §A1).** Introduce `ComputeLane { .gpu, .cpuTernary }` (Core) and a pure `LaneRouting` policy mapping `AgentWorkRole` → `ComputeLane`: `foregroundGeneration` and `mediaDiffusion` → `.gpu`; `toolRoute`, `classify`, `memoryRetrieval`, and `parkedSubagent` → `.cpuTernary`. Pure and `swift test`-verified — time/role in, lane out, no Metal. -- **A second `LLMRuntime` conformer on the CPU lane (`TernaryCPURuntime`).** A SMALL ternary/BitNet-class model running on the CPU lane behind the EXISTING `LLMRuntime` seam (not a new protocol) — selected by lane via the model fleet (addendum §C1, `role: .ternaryChat`, `lane: .cpuTernary`). The real native-linked backend (bitnet.cpp-class) is **`xcodebuild` compile-verify only**; the Core seam plus a deterministic `StubTernaryRuntime` are **`swift test`-verified**. -- **Cross-lane concurrency + a residency budget.** A pure `LaneArbiter`/residency-budget model (Core) that lets the GPU lane and the CPU ternary lane run **concurrently** (the ternary weights co-reside cheaply under the 48 GB budget alongside the resident chat weights + KV) and bounds CPU-lane concurrency on its own (it does NOT borrow GPU batch slots). The arbiter is honest: a heavy GPU generation is never blocked by CPU-lane work, and CPU-lane bursts never starve the foreground GPU reply. -- **The lane-affinity hint, consumed ADDITIVELY.** A `LaneAffinity` hint (Core) that `ParkScheduler` (§3.5) attaches to a runnable parked session and the batched runtime (§3.6) reads, so a `parkedSubagent`-roled session is dispatched to the **CPU ternary lane** while the foreground GPU generation keeps its GPU slot — the two advance at the same time. The hint is additive: `runnableSessions(now:maxSlots:)` and `batchStep(...)` keep their pinned shapes. **This slice AMENDS the single-GPU assumption via MODIFIED requirements on `on-device-ai-runtime` — it does NOT edit the batched or parked slice files.** -- **Gated by the master toggle (addendum §D1).** The CPU lane is behind `cpuLaneEnabled`, itself under `fullPotentialEnabled` (owned by `ai-full-potential-toggle`; this slice reads the flag, does not own it). When off, the CPU lane is not installed and all work routes to the GPU lane exactly as today (a fleet-of-one / one-lane build stays valid). Its cost — a small extra resident model, CPU heat under sustained structured bursts — is disclosed where the toggle is offered. - -## Capabilities - -### Modified Capabilities - -- `on-device-ai-runtime`: AMEND the implicit single-GPU-lane assumption. ADD a two-lane compute model (`ComputeLane` + the pure `LaneRouting` role→lane policy); ADD the CPU-lane ternary `LLMRuntime` conformer (a second conformer selected by lane, NOT a new protocol) with an HONEST short-bursts-only constraint; ADD cross-lane concurrency + a residency budget that co-resides the ternary model without starving the GPU reply; ADD the additive lane-affinity hint consumed by the parked scheduler and the batched runtime so a parked subagent runs on CPU concurrently with a foreground GPU generation. The existing `generate`, `structured`, `chat()`, lifecycle, registry, reasoning, vision, and batched-runtime requirements are unchanged; the batched runtime stays the GPU lane. - -## Impact - -- **Code (MLX-free Core — `swift build` / `swift test`):** new `AI/Compute/` — `ComputeLane` + `AgentWorkRole` (addendum §A1, OWNED here); `LaneRouting` protocol + `DefaultLaneRouting` (the pure role→lane policy); `LaneAffinity` hint value type; `LaneArbiter` + `LaneResidencyBudget` (pure cross-lane concurrency + co-residency math, `now:`/free-memory injected, no Metal); a deterministic `StubTernaryRuntime` (test-only) conforming to the existing `LLMRuntime`. All pure / `swift test`-able. -- **Code (native-linked `GemmaRuntime`/sibling framework — `xcodebuild` compile-verify ONLY; real correctness needs the user's stable-signed build):** the concrete `TernaryCPURuntime` (`LLMRuntime` conformer wrapping a bitnet.cpp-class ternary model on the CPU lane), wired in via the existing `ModelProvisioner`/`runtimeFactory` seam keyed by `ComputeLane`. No `ModelManager` API change; the metallib `*.bundle` copy in `build-app.sh` must not regress. -- **Consumes (verbatim, never redefines):** `ComputeLane`/`AgentWorkRole`/`LaneRouting` (addendum §A1 — OWNED here, written exactly as pinned); `LLMRuntime`/`LLMChatRequest`/`Token`/`StructuredSchema`/`structured()` (the model seam, untouched); `ModelDescriptor`/`ModelRole`/`ModelRegistry` (addendum §C1, `ai-model-fleet` — read `role: .ternaryChat`, `lane: .cpuTernary`); `ParkScheduler.runnableSessions`/`didAdvance` (§3.5, `ai-parked-sessions` — consumed additively via the lane-affinity hint, NOT edited); `BatchedLLMRuntime.batchStep` (§3.6, `ai-batched-runtime-and-context` — the GPU lane, consumed additively, NOT edited); `AgentSessionID` (§3.1); `fullPotentialEnabled`/`cpuLaneEnabled` (addendum §D1, `ai-full-potential-toggle` — read, not owned); `RuntimeError` + `AIError.message(for:)` for all failure surfacing. -- **House rules:** Apple-Silicon M5 floor (M4 min), NO degraded/low-end path — the CPU lane exists BECAUSE ternary weights are bandwidth-frugal on this hardware, not as a fallback for weak hardware. One error taxonomy (`RuntimeError`; a `ComputeError` only if lane dispatch genuinely needs its own case — prefer extending `RuntimeError`), one translator, mapped at the layer boundary, surfaced bounded + non-blocking; a CPU-lane step that fails is an observable `.failed` for that turn with a clean headline, never a false "done," never an `NSAlert`, never raw error text in a headline. -- **Out of scope:** the GPU batched decode loop / KV-quant / context tuning (`ai-batched-runtime-and-context`); the parked store + notch rail + park lifecycle (`ai-parked-sessions`); the model registry/residency eviction policy itself (`ai-model-fleet` — this slice supplies the lane concept it consumes and the co-residency MATH for the ternary model, not the fleet's eviction policy); media diffusion (`ai-media-runtime` — it is named only as a GPU-lane role); the master toggle UI + cost-disclosure page (`ai-full-potential-toggle`); any cloud runtime. diff --git a/openspec/changes/ai-compute-tiers/specs/on-device-ai-runtime/spec.md b/openspec/changes/ai-compute-tiers/specs/on-device-ai-runtime/spec.md deleted file mode 100644 index 5e97cb6..0000000 --- a/openspec/changes/ai-compute-tiers/specs/on-device-ai-runtime/spec.md +++ /dev/null @@ -1,104 +0,0 @@ -## MODIFIED Requirements - -### Requirement: Targets capable hardware only -The runtime SHALL target current, high-end Apple Silicon and use the best model the configuration specifies; it SHALL NOT provide a degraded small-model path for low-end hardware in this version. If the hardware or model cannot satisfy the feature, the system SHALL report the feature as unavailable rather than silently running a worse experience. The runtime SHALL NOT assume a **single GPU compute lane**: it MAY additionally use a **CPU compute lane** running a small ternary model **concurrently** with the GPU lane to serve short, frequent, structured work — and this CPU lane SHALL exist as a **bandwidth optimization for capable hardware, NOT as a degraded fallback for weak hardware**. The small ternary model on the CPU lane SHALL NOT be construed as the "degraded small-model path" this requirement forbids; it is an additional concurrent lane for light work on capable hardware, never a substitute for the GPU model on incapable hardware. - -#### Scenario: Unsupported configuration reports unavailable -- **WHEN** the required model cannot be run on the current machine -- **THEN** the feature reports itself unavailable instead of degrading silently - -#### Scenario: The CPU ternary lane is an optimization, not a low-end fallback -- **WHEN** the CPU compute lane is in use on capable hardware -- **THEN** it runs a small ternary model for short structured work **alongside** the GPU model, and it is never offered as a substitute for the GPU model on weak hardware - -## ADDED Requirements - -### Requirement: Two compute lanes with a pure role-to-lane policy -The system SHALL model compute as **two physical lanes** — a **GPU lane** and a **CPU ternary lane** — and SHALL assign each unit of agent work to a lane via a **pure, total role-to-lane policy**. Heavy work SHALL route to the GPU lane: the **foreground generation** (the visible reply) and **media diffusion**. Short, frequent, structured work SHALL route to the CPU ternary lane: the **router structured turn**, **classification** decisions (such as should-park / needs-you / which-skill), **memory retrieval/index** work, and **parked-subagent** background advances. The policy SHALL be a deterministic function of the work role alone (the same role always maps to the same lane), so lane assignment is reproducible and cannot drift between call sites. The GPU lane SHALL remain the existing batched/continuous-batching runtime; this requirement adds the CPU lane beside it and does not change the GPU lane's batching behavior. - -#### Scenario: Heavy work routes to the GPU lane -- **WHEN** the work is the foreground reply or a media diffusion job -- **THEN** the role-to-lane policy assigns it to the GPU lane - -#### Scenario: Short structured work routes to the CPU ternary lane -- **WHEN** the work is a router structured turn, a classification, a memory retrieval, or a parked-subagent advance -- **THEN** the role-to-lane policy assigns it to the CPU ternary lane - -#### Scenario: The policy is deterministic and total -- **WHEN** the same work role is presented to the policy more than once -- **THEN** it always maps to the same lane, and every defined role has a lane - -### Requirement: The CPU ternary lane is a second runtime conformer, not a new protocol -The CPU ternary lane SHALL be served by **another conformer of the existing language-model runtime abstraction**, carrying a small ternary/BitNet-class model — NOT a new, separate runtime protocol. Feature code (the band, the executor, the router, the tasks) SHALL continue to depend only on the single runtime abstraction and SHALL select the CPU-lane runtime **by lane** (via the model registry's lane-tagged descriptor), never by referencing a concrete CPU runtime type. Adding the CPU lane SHALL therefore require no change to feature code beyond lane-aware selection. - -#### Scenario: The CPU lane is selected by lane, not by concrete type -- **WHEN** work is routed to the CPU ternary lane -- **THEN** the system resolves the lane's runtime through the single runtime abstraction tagged with that lane, and feature code never references a concrete CPU runtime type - -#### Scenario: Adding the CPU lane is additive to feature code -- **WHEN** the CPU ternary conformer is introduced -- **THEN** the band, executor, router, and tasks compile and run unchanged, depending only on the shared runtime abstraction - -### Requirement: The CPU ternary lane serves short bursts only, never the long reply -The CPU ternary lane SHALL serve only **short, frequent, structured** work. Because a CPU ternary decode is **slower per token** than a GPU batched decode, the **long foreground reply SHALL NEVER be routed to the CPU lane**; it SHALL always run on the GPU lane. The CPU lane's value SHALL come from running its short bursts **concurrently** with the GPU reply (not from being faster per token), justified by the measured hardware facts: GPU prefill is up to ~4× faster on the neural accelerators (heavy generation belongs on the GPU); token generation is bandwidth-bound on the ~153 GB/s unified-memory bus; and the ternary model's weights are roughly 32× smaller, so reading them per token consumes only a small fraction of that bandwidth — letting the CPU lane run without meaningfully contending with the GPU reply for the shared bus. - -#### Scenario: The long reply stays on the GPU lane -- **WHEN** the foreground reply is generated -- **THEN** it runs on the GPU lane and is never dispatched to the CPU ternary lane, even when the GPU lane is busy - -#### Scenario: Short structured bursts run concurrently on the CPU lane -- **WHEN** a router turn or a classification runs while a foreground reply streams on the GPU -- **THEN** the structured burst runs on the CPU ternary lane at the same time, rather than queuing behind the GPU decode loop - -### Requirement: Cross-lane concurrency with a residency budget that never starves the GPU reply -The system SHALL run the GPU lane and the CPU ternary lane **concurrently** under a **pure residency budget**. The budget SHALL account for the resident GPU chat weights (read once), the GPU key/value caches per stream, and the **small** ternary residency footprint (the ~32×-smaller ternary weights), and SHALL admit the ternary model to **co-reside** with the current GPU batch and key/value caches under the unified-memory budget. The arbiter SHALL bound CPU-lane concurrency on its **own** small cap and SHALL NOT borrow GPU batch slots. Two invariants SHALL hold: (1) a heavy GPU generation SHALL NEVER be made to wait on CPU-lane work; (2) CPU-lane bursts SHALL NEVER preempt or starve the foreground GPU reply. A CPU-lane unit that cannot be admitted under its own cap or the residency budget in a given step SHALL **wait** (remain runnable) rather than be treated as a failure. The residency budget and the arbiter SHALL be **pure** (free memory and time are inputs), so their decisions are deterministically testable without real GPU work. - -#### Scenario: The ternary model co-resides cheaply with the GPU batch -- **WHEN** the residency budget is computed with the GPU chat weights and key/value caches resident -- **THEN** the small ternary model is admitted to co-reside, fitting where a second full chat model would not - -#### Scenario: A heavy GPU generation is never blocked by CPU work -- **WHEN** CPU-lane bursts are active and a foreground GPU generation needs to advance -- **THEN** the GPU generation advances without waiting on the CPU-lane work - -#### Scenario: CPU bursts never starve the foreground reply -- **WHEN** many CPU-lane bursts are queued while the foreground GPU reply is streaming -- **THEN** the foreground reply continues unimpeded and the CPU bursts are bounded by their own cap - -#### Scenario: An unadmittable CPU burst waits, not fails -- **WHEN** a CPU-lane burst cannot be admitted under the CPU cap or residency budget in the current step -- **THEN** it waits for a later step and is not reported as a failure - -#### Scenario: The budget and arbiter are deterministic -- **WHEN** the residency budget and arbiter are evaluated with a given free-memory figure and timestamp -- **THEN** their decisions depend only on those inputs and the lane state, so they are reproducible in tests - -### Requirement: An additive lane-affinity hint dispatches a parked subagent to the CPU lane concurrently with a foreground GPU generation -The system SHALL carry a **lane-affinity hint** — the compute lane a runnable session prefers, derived from its work role via the role-to-lane policy — and SHALL consume it **additively**, without changing the pinned shapes of the parked-session scheduler's runnable-set request or the batched runtime's batch-step entry point. The dispatcher SHALL read each runnable session's lane affinity and route a **CPU-ternary-affined** session (such as a parked subagent) to the **CPU ternary lane** while the GPU batched runtime keeps serving the foreground and GPU-affined sessions. The net effect SHALL be that a **parked subagent advances on the CPU lane at the same time** as a foreground GPU generation, rather than waiting for a GPU batch slot. This hint SHALL be additive on the existing scheduler and batched-runtime seams; it SHALL NOT require those seams' methods to change signature. - -#### Scenario: A parked subagent runs on CPU while the foreground generates on GPU -- **WHEN** a parked subagent session is runnable and a foreground GPU generation is in flight -- **THEN** the subagent is dispatched to the CPU ternary lane and advances concurrently with the foreground GPU generation - -#### Scenario: The lane-affinity hint is additive to the existing seams -- **WHEN** the lane-affinity hint is introduced -- **THEN** the parked scheduler's runnable-set request and the batched runtime's batch-step entry point keep their existing shapes, and the hint is read alongside them rather than by changing their signatures - -#### Scenario: Affinity follows the work role -- **WHEN** a session's lane affinity is derived -- **THEN** it equals the lane the role-to-lane policy assigns to that session's work role (a parked-subagent advance maps to the CPU lane; a foreground generation maps to the GPU lane) - -### Requirement: The CPU lane is gated by the master toggle and off means one-lane behavior -The CPU ternary lane SHALL be gated by a sub-capability flag under the master full-potential toggle. When the flag is **off**, the system SHALL NOT install the CPU ternary runtime and SHALL route **all** work to the GPU lane, behaving exactly as a single-lane build — a one-lane (fleet-of-one) configuration SHALL remain valid. When the flag is on, the role-to-lane policy SHALL take effect. The cost of enabling the CPU lane — a small additional resident model and CPU heat under sustained structured bursts — SHALL be disclosed where the toggle is offered. - -#### Scenario: Off routes everything to the GPU lane -- **WHEN** the CPU-lane flag is off -- **THEN** no CPU ternary runtime is installed and every work role routes to the GPU lane, exactly as a single-lane build - -#### Scenario: On enables the two-lane policy -- **WHEN** the CPU-lane flag is on -- **THEN** the role-to-lane policy takes effect and short structured work routes to the CPU ternary lane - -#### Scenario: The CPU lane's cost is disclosed -- **WHEN** the CPU-lane toggle is offered to the user -- **THEN** its RAM and heat cost is stated alongside the capability, not hidden diff --git a/openspec/changes/ai-compute-tiers/tasks.md b/openspec/changes/ai-compute-tiers/tasks.md deleted file mode 100644 index 8483f31..0000000 --- a/openspec/changes/ai-compute-tiers/tasks.md +++ /dev/null @@ -1,46 +0,0 @@ -> Decomposed for a workflow fan-out: §1–§4 are the pure-Core substrate (do first, all `swift test`), §5 is the native-linked ternary conformer (`xcodebuild` compile-verify only — real correctness needs the user's stable-signed build), §6 wires lane dispatch additively into the existing scheduler/batched seams, §7 gates on the master toggle, §8 verifies. The CPU lane (§5) requires the user's stable-signed build for real validation — the agent never builds/signs the `.app`. - -## 1. `ComputeLane` + role→lane policy (pure Core) - -- [x] 1.1 Add `ComputeLane { .gpu, .cpuTernary }` and `AgentWorkRole { foregroundGeneration, mediaDiffusion, toolRoute, classify, memoryRetrieval, parkedSubagent }` to `AI/Compute/ComputeLane.swift` (Core, MLX-free) — verbatim from addendum §A1, OWNED here. *Verify: `swift build`; `swift test` — both enums construct and round-trip `Codable`.* -- [x] 1.2 Add the `LaneRouting` protocol (§A1) + `DefaultLaneRouting` (D1): `foregroundGeneration`/`mediaDiffusion` → `.gpu`; `toolRoute`/`classify`/`memoryRetrieval`/`parkedSubagent` → `.cpuTernary`. Pure total function. *Verify: `swift test` — one assertion per `AgentWorkRole` case asserts the expected lane (exhaustive).* -- [x] 1.3 Add `LaneAffinity` (`{ sessionID: AgentSessionID, lane: ComputeLane }`, D4), constructed from a session's `AgentWorkRole` via `DefaultLaneRouting`. Consumes `AgentSessionID` verbatim from `ai-conversation-runtime` (do NOT redefine). *Verify: `swift test` — a `parkedSubagent` session yields `.cpuTernary`; a `foregroundGeneration` session yields `.gpu`.* - -## 2. Cross-lane concurrency + residency budget (pure Core) - -- [x] 2.1 Add `LaneResidencyBudget` (D3, Core, free-memory injected — no Metal): holds chat weight bytes (GPU, read once), KV bytes per GPU stream, and `ternaryResidencyBytes` (small, ~32× smaller weights); answers `ternaryCoResides(freeBytes:gpuStreams:contextTokens:) -> Bool`. *Verify: `swift test` — the ternary model co-resides with chat weights + GPU KV under a budget where a SECOND chat model would not fit; toggling `ternaryResidencyBytes` up/down flips the boundary as expected.* -- [x] 2.2 Add `LaneArbiter` (D3, Core, `now:` + free-memory injected): admits CPU-lane work CONCURRENTLY with GPU work, bounds CPU-lane concurrency on its own small cap, and enforces (1) a heavy GPU generation never waits on CPU work, (2) CPU bursts never starve/preempt the foreground GPU reply. *Verify: `swift test` — a GPU gen + a CPU burst both admitted in the same tick; the GPU gen is never deferred behind CPU work; the CPU cap rejects an over-cap CPU burst (it waits, not fails); deterministic for a fixed `now:`/free-mem.* -- [x] 2.3 Add a `StubTernaryRuntime` (Core, test-only) conforming to the EXISTING `LLMRuntime` (`generate`/`structured`/`capabilities`/`chat()`), scripting deterministic tokens + structured outcomes. *Verify: `swift test` — drives a `structured()` route turn + a short `generate`; tokens de-mux to the right `AgentSessionID`; no real weights.* - -## 3. OFF-coercion gate decorator (pure Core) - -- [x] 3.1 Add a `LaneRouting` decorator that, when `cpuLaneEnabled == false`, coerces every role's lane to `.gpu` (D6) so a one-lane / fleet-of-one build stays valid. Reads `cpuLaneEnabled`/`fullPotentialEnabled` (addendum §D1, owned by `ai-full-potential-toggle` — read, do NOT define). *Verify: `swift test` — with the flag off, every `AgentWorkRole` coerces to `.gpu`; with it on, `DefaultLaneRouting` mapping holds.* - -## 4. Error mapping (pure Core) - -- [x] 4.1 Map CPU-lane failures (load/prepare/decode) into the existing `RuntimeError` taxonomy at the conformer boundary; add a `ComputeError` LocalizedError ONLY if a lane-dispatch case genuinely cannot be carried by `RuntimeError` (D7 — prefer extending `RuntimeError`). Route through `AIError.message(for:)`. *Verify: `swift test` — a simulated CPU-lane failure produces a clean `AIPresentedError` headline (no raw text); a cancelled CPU-lane turn is NOT a failure.* -- [x] 4.2 Assert a failed CPU-lane step is an observable `.failed` for that turn with a clean headline, never a false "done." *Verify: `swift test` — a stub failure leaves the turn in `.failed` carrying the translated headline, never `.done`.* - -## 5. `TernaryCPURuntime` conformer (native-linked — `xcodebuild` compile-verify ONLY) - -> Native-linked. The agent NEVER builds/signs/installs the `.app` (ad-hoc signing breaks TCC grants). Real validation (live CPU/GPU concurrency, CPU per-token speed, no cross-lane bleed, RAM/heat) requires the **user's stable-signed build** — see §8.3. The metallib `*.bundle` → `Contents/Resources/` copy in `build-app.sh` must not regress. - -- [x] 5.1 Add `TernaryCPURuntime` (GemmaRuntime/sibling framework) conforming to the EXISTING `LLMRuntime` — a SMALL ternary/BitNet-class model on the CPU lane (NOT a new protocol, D2). Map vendor/OS errors (bitnet.cpp-class, `Process`, file IO) into `RuntimeError` at the boundary. *Verify: `xcodebuild` compile.* -- [x] 5.2 Implement the short-bursts-only decode path (D5): `structured()` route turns, classify, memory-retrieval, and parked-subagent `generate`/`chat` — honest that per-token is slower than the GPU, so the long foreground reply is NEVER routed here. *Verify: `xcodebuild` compile; **user run-verify** structured bursts complete on CPU while the GPU streams the reply.* -- [x] 5.3 Confirm `capabilities` advertises `text` (and not the heavy GPU-only modalities); a vision/media role is never routed to the CPU lane. *Verify: `xcodebuild` compile; `swift test` against the stub — `lane(for: .mediaDiffusion) == .gpu` keeps media off the CPU lane.* - -## 6. Lane-keyed wiring + additive lane-affinity dispatch (native-linked — `xcodebuild` compile-verify ONLY) - -- [x] 6.1 Wire `TernaryCPURuntime` into the existing `ModelProvisioner`/`runtimeFactory` keyed by `ComputeLane` + the fleet descriptor (`role: .ternaryChat`, `lane: .cpuTernary`, addendum §C1 — consume, do NOT redefine) — NO `ModelManager` API change. *Verify: `xcodebuild` compile; **user run-verify** the ternary conformer is injected for `.cpuTernary`.* -- [x] 6.2 Read `ParkScheduler.runnableSessions(now:maxSlots:)`'s returned IDs and attach `LaneAffinity` via an ADDITIVE accessor (D4) — the pinned `runnableSessions` signature is UNCHANGED and the `ai-parked-sessions` files are NOT edited. *Verify: `xcodebuild` compile; **user run-verify** a `parkedSubagent` session carries `.cpuTernary` affinity.* -- [x] 6.3 In the dispatcher, route `.cpuTernary`-affined sessions to `TernaryCPURuntime` while the GPU `batchStep(...)` (§3.6) keeps serving `.gpu`-affined + foreground sessions — ADDITIVE, the `ai-batched-runtime-and-context` files are NOT edited. *Verify: `xcodebuild` compile; **user run-verify** a parked subagent advances on CPU CONCURRENTLY with a foreground GPU generation.* - -## 7. Master-toggle gate (Core read + native install gate) - -- [x] 7.1 Gate CPU-lane installation on `cpuLaneEnabled` under `fullPotentialEnabled` (D6, §D1 — read, owned by `ai-full-potential-toggle`): off → no `TernaryCPURuntime` installed, the OFF-coercion decorator (§3.1) routes all work to the GPU. *Verify: `swift test` for the decorator; **user run-verify** OFF installs no CPU lane and behaves exactly as today.* **(wire-compute-fleet: `makeModelManager(optedIn:fullPotentialEnabled:cpuLaneEnabled:)` now CALLS `LaneDispatch.installCPULane(...)` behind the live `fullPotentialEnabled && cpuLaneEnabled` gate — threaded from `AppSettings` via `AIRuntimeInjection.modelManagerFactory` + `main.swift`. A `.cpuTernary`-lane fleet descriptor resolves to the installed `TernaryCPURuntime` through the UNCHANGED provisioner seam keyed by `descriptor.lane`; OFF → nil → GPU lane alone, today's behavior.)** - -## 8. Spec sync + verification - -- [x] 8.1 Update `openspec/specs/on-device-ai-runtime/spec.md` with the MODIFIED two-lane requirements after implementation lands (AMEND the single-GPU assumption; do NOT touch the batched/parked slice specs). *Verify: `openspec validate ai-compute-tiers --strict`.* -- [x] 8.2 `swift build` + `swift test` green for all of §1–§4, §7 (the pure-Core majority). *Verify: CI / local `swift test`.* -- [ ] 8.3 **User's stable-signed build only:** verify live CPU/GPU concurrency (a parked subagent advances on CPU while the GPU streams a reply), CPU per-token speed is acceptable for short bursts, no cross-lane bleed, ternary co-residency fits the 48 GB budget, and CPU heat under sustained structured bursts is disclosed. *Verify: `INSTALL=1 ./scripts/build-app.sh`, then exercise a parked subagent during a foreground reply.* diff --git a/openspec/changes/ai-conversation-runtime/.openspec.yaml b/openspec/changes/ai-conversation-runtime/.openspec.yaml deleted file mode 100644 index 38f7628..0000000 --- a/openspec/changes/ai-conversation-runtime/.openspec.yaml +++ /dev/null @@ -1,2 +0,0 @@ -schema: spec-driven -created: 2026-06-22 diff --git a/openspec/changes/ai-conversation-runtime/design.md b/openspec/changes/ai-conversation-runtime/design.md deleted file mode 100644 index 7b83828..0000000 --- a/openspec/changes/ai-conversation-runtime/design.md +++ /dev/null @@ -1,223 +0,0 @@ -## Context - -This is **Wave 1** of the V2 agent decomposition (`docs/ai-agent-v2-blueprint.md`) — the *type home*. It evolves the runtime + executor from a stateless single-prompt fire into a multi-turn **session**, and owns the canonical conversation types every other slice consumes. It must land before `ai-tool-routing`, `ai-conversational-canvas`, `ai-parked-sessions`, `ai-agent-memory`, and `ai-background-autonomy`, which all import its types verbatim. - -Ground truth in the existing code (read before judging this design): - -- **`AI/LLMRuntime.swift`** — `LLMRuntime` protocol (`generate`/`structured`/`capabilities`), `LLMRequest(prompt:image:parameters:reasoning:)`, `GenerationParameters`, `Token`/`TokenChannel(.response/.thinking)`, `StructuredSchema`/`StructuredOutcome`, the `RuntimeError` taxonomy + `LocalizedError`. **The `.thinking`/`.response` channel split already exists** — V2 reuses it; this slice adds no third channel. -- **`AI/AICommandExecutor.swift`** — `@MainActor ObservableObject` with an `enum State` (`.idle`/`.loadingModel`/`.noInput`/`.streaming(partial:)`/`.ready(result:)`/`.reviewingAction(TaskReview)`/`.declined(reason:)`/`.failed(message:)`/`.unavailable`/`.committed`), a `@Published thinking: String`, `canvasAtTop`, a per-fire `generationTask`, and `run(_:)` which streams tokens splitting `.thinking`→`thinking` and `.response`→`accumulated`→`.streaming`/`.ready`. -- **`AI/StubLLMRuntime.swift`** — the deterministic scriptable runtime: `scriptedTokens`/`scriptedThinking`, `interTokenDelayNanos`, `StructuredScript`, cancellation observation. Today it scripts ONE generation; this slice makes it script a **sequence** of generations (one per turn). -- **`AI/PromptTemplate.swift`** — `FireContext` + `PromptTemplate.resolve(...)` (`{input}`/`{date}`/`{app}`/`{url}`/`{lang}`). Reused to build turn-1 seed text. -- **`AI/AIError.swift`** — the single `AIError.message(for:) -> AIPresentedError` translator; every `.failed` headline routes through it. -- **`openspec/specs/on-device-ai-runtime/spec.md`** — the existing capability spec this slice delta-modifies. - -The blueprint pins the exact shapes (§3.1, §3.2). This design implements them and the compaction logic; it does NOT redefine `ToolRoute`/`ToolStepResult` (owned by `ai-tool-routing`) — `AgentMessage` references them as the blueprint sketches, so the field types are declared here in their final form (this slice owns `AgentMessage`) but the `ToolRoute`/`ToolStepResult`/`ToolDescriptor` *definitions* are imported from the tool-routing slice. See D9 for the ordering resolution. - -## Goals / Non-Goals - -**Goals:** -- Own the canonical `AgentRole`/`AgentMessage`/`AgentSessionID`/`AgentConversation`/`AgentTurn` (blueprint §3.1), MLX-free Core, `Codable`, `Equatable`, `Sendable`. -- Add `LLMChatRequest` + a default-implemented `LLMRuntime.chat(_:)` (blueprint §3.2) — additive; existing conformers keep compiling via a messages→prompt flatten. -- Evolve `AICommandExecutor` into a session: hold history, append user + assistant turns, exclude `.thinking` from re-fed history, add multi-turn `State` cases without breaking the one-shot cases. -- Stream `.thinking`/`.response` per turn; store thinking on the message for display, response in history. -- Compaction: a pure windowing/estimate decision + a summarization model call producing `compactedSummary`, triggered by a budget read through an injected `ContextBudgetProviding`. -- Per-turn cancellation + per-turn `.failed` via `RuntimeError`/`AIError`. -- A multi-turn-scriptable `StubLLMRuntime` so `swift test` drives full deterministic conversations. - -**Non-Goals:** -- The tool-route loop, `ToolRoute`/`ToolDescriptor`/`ToolStepResult`/`ToolRegistry` — `ai-tool-routing` owns these. This slice only references them as `AgentMessage` fields. -- The conversational canvas UX, the gesture compass, float-up, the overscroll-park trigger — `ai-conversational-canvas`. -- Durable on-disk conversation storage and the parked-session lifecycle — `ai-parked-sessions` owns the store; this slice owns the `Codable` type only. -- The `agentContextTokens` user slider + persisted key, `ModelDescriptor.maxContextTokens` plumbing, KV-quant, and the batched `chat` override — `ai-batched-runtime-and-context`. This slice reads the budget through an injected provider (C3). -- The real Gemma chat-template (`enable_thinking`, Gemma turn markers) — GemmaRuntime slice; this slice ships only the Core flatten-default + the Stub. -- A new `Error` — turn failures fit `RuntimeError`; do not add a taxonomy. - -## Decisions - -### D1. The canonical conversation types (blueprint §3.1, verbatim shapes) - -In `AI/Agent/AgentConversation.swift` (Core, MLX-free): - -``` -public enum AgentRole: String, Codable, Sendable { case user, assistant, system, tool } - -public struct AgentMessage: Codable, Equatable, Identifiable, Sendable { - public let id: UUID - public var role: AgentRole - public var text: String // committed user/response text — NEVER thinking - public var thinking: String? // reasoning, retained for DISPLAY only; never re-fed as ground truth - public var image: Data? // optional per-turn image (PNG); mirrors LLMRequest.image - public var toolCalls: [ToolRoute]? // assistant turn that routed to tools (type from ai-tool-routing) - public var toolResult: ToolStepResult? // role == .tool: the executed step's outcome (ai-tool-routing) - public var createdAt: Date -} - -public struct AgentSessionID: Hashable, Codable, Sendable { public let raw: UUID } - -public struct AgentConversation: Codable, Equatable, Identifiable, Sendable { - public let id: AgentSessionID - public var title: String - public var messages: [AgentMessage] - public var createdAt: Date - public var updatedAt: Date - public var compactedSummary: String? // compaction output: prefix summary replacing dropped turns - public var skillID: String? // the active skill driving this session, if any -} - -public struct AgentTurn: Sendable { - public var messages: [AgentMessage] // the windowed/compacted message list for THIS generation - public var image: Data? // convenience: latest turn's image - public var reasoning: Bool - public var parameters: GenerationParameters -} -``` - -- `text` is the contract for **what is re-fed**; `thinking` is the contract for **what is shown but never re-fed** — this is how "`.thinking` excluded from history" is enforced *by the type*, not by a runtime convention. Assembling an `LLMChatRequest` reads `message.text` only. -- `AgentSessionID` is the identity threaded through every slice, **stable across park/restore** (the parked slice persists the conversation under this id; the batched runtime keys streams by it). -- `AgentTurn` is the post-compaction unit handed to the runtime; it is the in-memory product of "windowed/compacted `messages` + the latest image + per-turn reasoning + parameters." It is **not** `Codable` (it's transient) — only `AgentConversation` persists. - -### D2. `LLMChatRequest` + `chat()` — additive, default-flattened (blueprint §3.2; integration fix on existing seam) - -In `AI/LLMRuntime.swift`, alongside the unchanged `LLMRequest`: - -``` -public struct LLMChatRequest: Sendable { - public var messages: [AgentMessage] - public var image: Data? - public var parameters: GenerationParameters - public var reasoning: Bool - public var tools: [ToolDescriptor]? // route-mode advertisement (ai-tool-routing); nil = plain chat -} - -extension LLMRuntime { - public func chat(_ request: LLMChatRequest) -> AsyncThrowingStream { - // DEFAULT: flatten messages → one prompt via ChatTemplate, then call generate(_:). - let prompt = ChatTemplate.flatten(request.messages) - return generate(LLMRequest(prompt: prompt, image: request.image, - parameters: request.parameters, reasoning: request.reasoning)) - } -} -``` - -- **Additive**: `chat` is a protocol extension with a default body, so `StubLLMRuntime`, `DevAIRuntime`, and the existing Gemma conformer compile unchanged and immediately work multi-turn (via flatten). The batched MLX conformer overrides `chat` for true KV-reuse later — this slice does not write that override. -- `tools` is declared so the type is final (consumers don't have to widen it later), but this slice ignores it — only `ai-tool-routing` populates/reads it. `ToolDescriptor` is imported from that slice; see D9. -- The default `chat` honors `reasoning` and `image` by passing them straight through to `generate(_:)`, so the channel split and vision path are inherited for free. - -### D3. `ChatTemplate.flatten` — the Core, model-agnostic assembler - -In `AI/Agent/ChatTemplate.swift` (Core). The default `chat` impl needs *a* prompt from a message list; the real Gemma chat-template (with `` markers + `enable_thinking`) is a GemmaRuntime concern. So: - -- `ChatTemplate.flatten(_ messages: [AgentMessage]) -> String` produces a plain, deterministic, role-labeled transcript (e.g. `System: …\n\nUser: …\n\nAssistant: …\n\nUser: …\n\nAssistant:`), reading **only `message.text`** (never `thinking`). A `.tool` message renders its `toolResult?.summary` as a `Tool:` line. The trailing `Assistant:` cue invites the next turn. -- This is enough for the Stub and the flatten-default to drive multi-turn deterministically and for tests to assert assembly. The **real** Gemma markers + `enable_thinking` flag are **flagged for the GemmaRuntime slice** (a `// FLAGGED: GemmaRuntime` comment + a spec scenario), where the batched conformer's `chat` override builds Gemma's native chat template instead of calling `flatten`. -- `flatten` is pure/`nonisolated`/static, unit-tested for role ordering, thinking-exclusion, tool-line rendering, and the trailing cue. - -### D4. `AICommandExecutor` becomes a SESSION — additive `State` cases - -Today the executor is one-shot. The evolution keeps every existing case and path working (the preset-command fire is unchanged) and **adds** session state: - -- A new stored `private(set) var conversation: AgentConversation?` — the in-memory thread for the active session. `nil` until the first turn opens it. (Durable persistence is `ai-parked-sessions`; here it lives for the session's life in memory.) -- New additive `State` cases (the canvas slice will render these; they are observable here): - - `.conversing(partial: String)` — an assistant turn is streaming *within an open thread* (distinct from the one-shot `.streaming`, which the preset path keeps using so its commit semantics are untouched). `partial` is the response-channel accumulation for the in-flight turn. - - `.awaitingTurn` — the thread is open and idle, waiting for the next user turn (Enter = send, from the canvas). - - The existing `.failed(message:)` carries per-turn failures. - - `.committed`/`.ready` stay one-shot only. NOTE: `.awaitingApproval`/`.parked` are **owned by `ai-conversational-canvas`/`ai-tool-routing`** (blueprint §5.1) — this slice does not add them; it adds only the runtime-level conversing/awaiting states, leaving room for those slices to add theirs additively. -- `State` equality is extended for the new cases (mirroring the existing hand-written `==`). -- **Seed → turn 1.** A new `startConversation(seedText:image:parameters:reasoning:)` (or the existing `fire(_:)` evolves to call it for an agent-mode command) creates the `AgentConversation` with `messages = [AgentMessage(role:.user, text: seedText, image: image, createdAt: now)]`. The seed text comes from the SAME acquisition + `PromptTemplate.resolve` the one-shot path uses today — the seed is just turn 1. -- **A turn loop** `runTurn()` (private, `async`): - 1. Append the pending user `AgentMessage` (if not already the last message). - 2. Assemble the `LLMChatRequest` via `assembleRequest()` (D5) — applying compaction first (D6). - 3. `state = .conversing(partial: "")`; reset the live `thinking` for this turn (so a new turn never shows the prior turn's reasoning). - 4. Stream `runtime.chat(request)`: `.thinking` tokens → live `thinking` (display); `.response` tokens → `accumulated` → `.conversing(partial:)`. - 5. On completion: append `AgentMessage(role:.assistant, text: accumulated, thinking: thinking.isEmpty ? nil : thinking, createdAt: now)`. **`text` = response only; `thinking` is stored for display but never re-fed.** `state = .awaitingTurn`. - 6. On `RuntimeError.cancelled` / `CancellationError`: return (a discard is not a failure; the partial assistant turn is NOT appended). On any other error: `state = .failed(message: AIError.message(for:).headline)`. - -### D5. `assembleRequest()` — conversation → `LLMChatRequest` (channel-honest) - -- Builds the message list fed to the runtime from `conversation.messages`, **prefixing the `compactedSummary`** as a synthetic `AgentMessage(role:.system, text: compactedSummary)` when present (so the model still "remembers" the dropped turns). -- **Re-feeds `message.text` only** — `thinking` is structurally excluded (it lives in a separate field that assembly never reads). This is the core "thinking never bloats context" guarantee. -- `image` = the latest user turn's image (convenience field on `AgentTurn`/`LLMChatRequest`); reasoning = the resolved per-command/global reasoning (reusing today's `command.resolvedReasoning(globalDefault:)` plumbing); parameters = `GenerationParameters` as today. - -### D6. Compaction — pure windowing decision + a summarization model call (this slice OWNS it) - -The hard part. Split into a pure decision and an impure pass: - -- **`ContextBudget.swift` (Core):** - - `protocol ContextBudgetProviding: Sendable { var maxContextTokens: Int { get } }` — the **injected** budget seam (integration fix **C3**). The executor depends on this protocol, NEVER on the concrete `agentContextTokens` slider. The app wires a real provider (backed by `ModelDescriptor.maxContextTokens` ∩ the user's `agentContextTokens`, both owned by `ai-batched-runtime-and-context`); tests pass a fixed-budget stub. This is why this slice can land before the batched-runtime slice. - - `TokenEstimator` — a pure, deterministic estimate of an `[AgentMessage]`'s token cost (a character/word-ratio heuristic; estimates `text` only, never `thinking`, since thinking is not re-fed). Honest about being an estimate, not a real tokenizer — the budget keeps a safety margin (e.g. compact at 80% of `maxContextTokens`). -- **`ConversationCompactor.swift` (Core):** - - `func needsCompaction(_ conversation: AgentConversation, budget: ContextBudgetProviding) -> Bool` — pure: estimate the assembled messages (incl. any existing `compactedSummary` prefix); true when the estimate crosses the compaction threshold (margin-adjusted budget). - - `func plan(_ conversation: AgentConversation, budget: ContextBudgetProviding) -> CompactionPlan` — pure: decides which prefix of `messages` to collapse (always keep the most recent `keepRecentTurns` turns verbatim; collapse everything older, including any prior summary, into the new summary's *input*). Returns the to-summarize slice + the to-keep tail. Deterministic and unit-testable with a fixed-budget stub. - - `func summarize(_ plan: CompactionPlan, runtime: LLMRuntime) async throws -> String` — the impure pass: a single `runtime.generate(LLMRequest(prompt: summarizationPrompt))` call (reasoning OFF) that condenses the to-summarize slice into a compact factual summary string. Maps any error through `RuntimeError`/`AIError`. - - Applying the plan: `conversation.compactedSummary = summary`; `conversation.messages = plan.keptTail`. The summary then enters the next `assembleRequest()` as the `system` prefix (D5). -- **Trigger timing:** compaction runs inside `runTurn` *before* assembly, only when `needsCompaction` is true. The compaction summarization call is itself cancellable (it's part of the turn's task). A failed summarization is a turn `.failed` (never silently drops history → never a false continuation). -- **Coordinate the budget with the batched-runtime/context slice (C3):** the provider protocol is the contract; the concrete `maxContextTokens` field on `ModelDescriptor` and the `agentContextTokens` user slider are owned there. This slice ships the protocol + a test stub + a default provider (a constant fallback) so it builds standalone. - -### D7. Channels across turns (reuse the existing split, repeat per turn) - -- Per turn, the streaming loop is the same channel split as today's `run(_:)`: `.thinking` → live `thinking` (the canvas's collapsible section, updated live), `.response` → `accumulated`. The ONLY change is that this now runs per turn and the results are persisted onto an `AgentMessage` (D4.5): `text = response`, `thinking = the accumulated reasoning (display only)`. -- `thinking` (the `@Published` live string) is reset at the START of each turn so a new turn never shows the prior turn's reasoning live; the prior turn's reasoning remains available on its `AgentMessage.thinking` for the transcript. -- No third channel is added (blueprint: do not add one without cross-slice sign-off). - -### D8. `StubLLMRuntime` multi-turn scripting (so `swift test` drives full conversations) - -- Add a `scriptedTurns: [TurnScript]` queue where `TurnScript { tokens: [String], thinking: [String] }`. Each `chat`/`generate` call dequeues the next `TurnScript` (FIFO); when the queue is exhausted it falls back to today's `scriptedTokens` behavior (so existing single-generation tests are byte-identical). A dedicated `scriptedSummary: String?` (or a reserved turn) lets a test script the compaction summarization call's output. -- The default `chat` impl flattens then calls `generate`, so the Stub's `generate` is the single place that consumes a `TurnScript` — no separate `chat` override needed in the Stub (it inherits the default). The Stub keeps honoring per-call cancellation (per-turn discard) and can be scripted to throw a `RuntimeError` on a chosen turn (per-turn `.failed`). -- This lets a test assert: turn-1 response, turn-2 sees turn-1 in the assembled prompt (via a flatten assertion), `.thinking` never appears in the re-fed prompt, a compaction summary replaces old turns, a mid-turn cancel leaves no assistant message, a scripted turn error → `.failed`. - -### D9. Type-ownership ordering: `AgentMessage` references tool types owned by a *later* slice - -`AgentMessage.toolCalls: [ToolRoute]?` and `.toolResult: ToolStepResult?` reference types `ai-tool-routing` owns (Wave 2, *after* this slice). Resolution (no DAG back-edge, no forked type): - -- This slice OWNS `AgentMessage` and declares those two fields in their final shape. `ai-tool-routing` lands `ToolRoute`/`ToolStepResult`/`ToolDescriptor`/`WritePolicyTier` as Core types. Both are in the same `ThreeFingerSwitcherCore` module, so `AgentMessage` can reference them without an import cycle once both exist. -- **Sequencing reality:** to let this slice's *code* compile standalone before tool-routing lands, the implementing PR for THIS slice introduces minimal placeholder `ToolRoute`/`ToolStepResult`/`ToolStepStatus`/`ToolDescriptor`/`WritePolicyTier` value types EXACTLY as blueprint §3.3/§3.7 sketch them, in a Core file, and `ai-tool-routing` then *takes ownership* (moves the loop/registry logic onto them) without changing the shapes. The blueprint pins the shapes precisely so this hand-off is mechanical. The tasks list this explicitly (task 7) and the spec delta notes the cross-slice ownership. -- This is the only place this slice touches another slice's types; everything else (`LLMChatRequest.tools`) is just declared-and-ignored here. - -### D10. Errors — reuse the taxonomy; no new `Error` - -- Every turn failure (generation error, summarization error, decode) maps through the existing `RuntimeError` taxonomy and surfaces via `AIError.message(for:).headline` into `.failed(message:)`. Cancellation is benign (a discard) — not a failure. A summarization that fails is a turn `.failed`, never a silent history drop. No new error type is justified (blueprint: add `Error` only where `RuntimeError`/`TaskError` cannot carry the case — they can here). - -### D11. `@MainActor` + concurrency - -- `AICommandExecutor` stays `@MainActor ObservableObject` (its convention). The turn loop is a retained `Task` (like today's `generationTask`) so a discard cancels it. Per-turn streaming mutates `@Published state`/`thinking` on the main actor as today. -- The Core value types (`AgentMessage` et al.) and the pure compaction logic are `Sendable`/`nonisolated`, so they cross the actor boundary cleanly and are testable off the main actor. - -## File-level touch list (target + verification) - -| File | Target | Change | Verified by | -|---|---|---|---| -| `AI/Agent/AgentConversation.swift` (new) | Core | `AgentRole`/`AgentMessage`/`AgentSessionID`/`AgentConversation`/`AgentTurn` (D1) | `swift test` (Codable round-trip, thinking-exclusion invariant, equality) | -| `AI/Agent/ChatTemplate.swift` (new) | Core | `ChatTemplate.flatten` (D3); real Gemma markers FLAGGED for GemmaRuntime | `swift test` (role order, thinking excluded, tool line, trailing cue) | -| `AI/Agent/ContextBudget.swift` (new) | Core | `ContextBudgetProviding`, `TokenEstimator`, default constant provider (D6, C3) | `swift test` (estimate monotonicity, margin, injected stub) | -| `AI/Agent/ConversationCompactor.swift` (new) | Core | `needsCompaction`/`plan`/`summarize` + apply (D6) | `swift test` (plan keeps recent N, collapses prefix incl. prior summary; summarize via Stub; failure → throws) | -| `AI/Agent/ToolPlaceholders.swift` (new, temporary) | Core | placeholder `ToolRoute`/`ToolStepResult`/`ToolStepStatus`/`ToolDescriptor`/`WritePolicyTier` (D9) — handed to `ai-tool-routing` | `swift build` (compiles); `ai-tool-routing` takes ownership later | -| `AI/LLMRuntime.swift` | Core | add `LLMChatRequest` + `LLMRuntime.chat(_:)` default impl (D2) | `swift test` (default flatten path), `xcodebuild` (Gemma conformer still compiles) | -| `AI/AICommandExecutor.swift` | Core | session history, `.conversing`/`.awaitingTurn` States + equality, `startConversation`/`runTurn`/`continueConversation`/`assembleRequest` (D4/D5/D7) | `swift test` (full scripted conversation, per-turn cancel, per-turn fail, compaction trigger) | -| `AI/StubLLMRuntime.swift` | Core | `scriptedTurns` queue + scripted summary + per-turn throw (D8) | `swift test` | -| `Tests/ThreeFingerSwitcherTests/AgentConversationTests.swift` (new) | Test | type + assembly + thinking-exclusion | `swift test` | -| `Tests/ThreeFingerSwitcherTests/ConversationCompactionTests.swift` (new) | Test | windowing/plan/summarize | `swift test` | -| `Tests/ThreeFingerSwitcherTests/ConversationSessionTests.swift` (new) | Test | executor multi-turn session machine | `swift test` | -| `AI/GemmaRuntime/*` (the real `chat` override + Gemma chat-template) | GemmaRuntime | **FLAGGED, not in this slice** — owned by `ai-batched-runtime-and-context` | `xcodebuild` compile-only when it lands | - -**Verification split:** essentially all of this slice is MLX-free Core verified by `swift build` + `swift test`. The only `xcodebuild` concern is confirming the additive `chat` default + `LLMChatRequest` don't break the GemmaRuntime conformer's compile (the agent runs `xcodebuild` compile-verify only; never builds/signs/installs the `.app`). The real Gemma chat-template override is explicitly deferred. To compile-check this slice in isolation from sibling uncommitted files, use a throwaway `git worktree` + `swift build`. - -## Edge cases - -- **First turn with no input.** A seed with empty/whitespace text AND no image → the existing `.noInput` path (no conversation opened, no model call) — preserved from today. -- **Mid-turn discard.** Cancelling during streaming: the partial assistant turn is NOT appended (no half-message in history); the conversation stays at the prior `.awaitingTurn`/closed state; not a failure. -- **Compaction mid-turn fails.** The summarization model call throws → turn `.failed` with a clean headline; history is NOT dropped (the plan is applied only on a successful summary), so the next retry sees the full thread. -- **Thinking leak.** A bug that re-fed `thinking` would bloat the window and violate the contract — assembly reads `text` only, and the type stores thinking in a separate field; a unit test asserts the assembled prompt never contains a turn's `thinking` string. -- **Budget smaller than the kept tail.** If even `keepRecentTurns` exceeds the budget, compaction collapses to the minimum viable (most-recent turn + summary); the estimator's margin means we compact early rather than overflow. (M5/48GB: the model max is large; this is a guard, not a hot path.) -- **Empty conversation assembly.** Assembling with zero messages is a no-op guarded before any model call. -- **Reasoning off.** No `.thinking` tokens arrive; `AgentMessage.thinking` stays `nil`; identical to today's response-only stream. -- **Image only on turn 1.** A vision seed carries the image on turn 1's user message; later turns have no image (the convenience `AgentTurn.image` is the latest turn's image, nil after turn 1) — matches today's single-image fire. -- **Stub turn queue exhausted.** Falls back to legacy `scriptedTokens` so existing tests are unaffected. - -## Rejected alternatives - -- **Replace `LLMRequest`/`generate` with messages everywhere.** Rejected — breaks every existing conformer and the one-shot preset path, violates "additive, never breaking" (blueprint §0). The default-flattened `chat` extension gives multi-turn for free while leaving `generate` untouched. -- **A new `ConversationError` taxonomy.** Rejected — `RuntimeError` already carries cancellation/decode/unavailable/couldNotProduceValid; turn failures fit it. The blueprint forbids a new `Error` where the existing taxonomy suffices. -- **Store `thinking` in `text` and strip at assembly.** Rejected — fragile (a strip bug leaks reasoning into context). Storing thinking in a separate field makes exclusion structural, not procedural. -- **Read the `agentContextTokens` slider directly for the budget.** Rejected — creates a hard dependency on `ai-batched-runtime-and-context` landing first (DAG back-edge). The injected `ContextBudgetProviding` (C3) breaks the cycle; the concrete slider is wired by the app later. -- **A separate `ConversationStore` here.** Rejected — `ai-parked-sessions` owns durable storage; duplicating it would fork the store. This slice owns only the in-memory `AgentConversation` + its `Codable` conformance. -- **Summarize via `structured()`.** Rejected for v1 — a free-text summary fed back as a `system` prefix is simpler and robust; structured summarization adds schema brittleness for no gain. (`structured()` stays the router's job in tool-routing.) -- **A third token channel for tool/summary output.** Rejected — blueprint pins `.thinking`/`.response` and bars a third channel without sign-off; the summarization call uses plain `.response`. diff --git a/openspec/changes/ai-conversation-runtime/proposal.md b/openspec/changes/ai-conversation-runtime/proposal.md deleted file mode 100644 index f9f02b8..0000000 --- a/openspec/changes/ai-conversation-runtime/proposal.md +++ /dev/null @@ -1,35 +0,0 @@ -## Why - -The on-device AI feature today is **one-shot**: `AICommandExecutor.fire(_:)` acquires input, resolves a single prompt template, streams one `LLMRequest(prompt:)` into `state`, and commits. There is no notion of a *thread* — every fire starts from zero, the model never sees what was said two turns ago, and the executor's `State` machine ends in `.committed`/`.ready`/`.failed` with no concept of "continue the conversation." - -V2 turns this into a **conversational agent**: the user (and, later, background autonomy) carry a multi-turn dialogue with the model, tools execute mid-thread, and a session can be parked, restored, and advanced in the background. None of that is possible until there is a **canonical conversation type home** and a **runtime that speaks turns, not prompts**. This slice is that foundation — Wave 1 in the V2 dependency DAG. Seven other slices (`ai-tool-routing`, `ai-conversational-canvas`, `ai-parked-sessions`, `ai-agent-memory`, `ai-background-autonomy`, `ai-batched-runtime-and-context`, `ai-claude-handoff`) consume the types this slice owns; if their shapes diverge, the whole agent fractures. - -The existing seams already lean this way: `LLMRequest` carries an optional per-turn `image`; `Token`/`TokenChannel` already split `.thinking` from `.response` (the channel work is done — V2 reuses it verbatim); `StructuredOutcome` already lets the model `.declined`; `AIError.message(for:)` already gives every surface one clean headline. This slice is an **additive** evolution of those seams — never a fork. - -The two hard problems this slice owns and must get right once, for everyone: - -1. **The canonical multi-turn types** — `AgentMessage` / `AgentRole` / `AgentConversation` / `AgentSessionID` / `AgentTurn` (blueprint §3.1). Define them exactly as the blueprint pins them so no consumer redefines a conflicting shape. -2. **Compaction** — a long thread will exceed the model's context window. When the assembled turn estimate approaches a budget, older turns must collapse into a compact summary (a model call) so the conversation can continue without context overflow, and so `.thinking` never bloats the window (it is excluded from history by construction). - -## What Changes - -- **`LLMRequest` grows a messages path, additively.** The existing `LLMRequest(prompt:image:parameters:reasoning:)` and `structured(...)` stay byte-identical. A new `LLMChatRequest{messages:[AgentMessage], image, parameters, reasoning, tools}` and a default-implemented `LLMRuntime.chat(_:) -> AsyncThrowingStream` are added; the default impl **flattens messages → a single prompt** via a new `ChatTemplate` assembler and calls `generate(_:)`, so `StubLLMRuntime`/`DevAIRuntime` and the real Gemma conformer keep compiling. The batched MLX conformer (later slice) overrides `chat` for true KV-reuse. `structured()` keeps working unchanged and is the router's mechanism (tool-routing slice). -- **This slice OWNS the canonical conversation types** — `AgentRole`, `AgentMessage`, `AgentSessionID`, `AgentConversation`, `AgentTurn` (blueprint §3.1), in MLX-free Core. A seed (image and/or text) becomes turn 1: the first `AgentMessage(role:.user)` carries the acquired input text and/or the captured/clipboard image, exactly mirroring how `fire(_:)` builds today's single `LLMRequest`. -- **`AICommandExecutor` evolves from one-shot to a SESSION.** It gains an in-memory `AgentConversation` (history). A turn appends the user `AgentMessage`, streams the assistant turn, and appends an assistant `AgentMessage` carrying **only the response channel** — `.thinking` is retained on the message for display but is **excluded from re-fed history** so it never bloats the window. New additive `State` cases represent an ongoing thread; the existing one-shot cases are preserved so today's preset-command path is unchanged. -- **Channels across turns.** Per turn, `.thinking` streams into the live observable `thinking` (the canvas's collapsible section) and `.response` accumulates into the committed turn — exactly as `run(_:)` does today, now repeated per turn and persisted onto `AgentMessage.thinking` (display) vs `AgentMessage.text` (re-fed). -- **Compaction.** A pure, injected token-budget estimator + a windowing decision in Core decide *when* older turns must collapse; a summarization pass (an `LLMRuntime` call) produces `AgentConversation.compactedSummary`, which replaces the dropped raw turns as a synthetic `system`/prefix turn in the next assembled `LLMChatRequest`. The budget is read through an **injected provider** (`ContextBudgetProviding`), never the concrete `agentContextTokens` slider — so this slice never depends on `ai-batched-runtime-and-context` landing first (integration fix C3). -- **`StubLLMRuntime` becomes scriptable for multi-turn.** A per-turn script queue lets `swift test` drive a full deterministic conversation (turn 1 → tokens, turn 2 → tokens, a scripted compaction-summary turn), with per-turn cancellation and per-turn `.failed` outcomes, so the whole session machine is covered without a model. -- **Per-turn cancellation + per-turn `.failed`.** Each turn is individually cancellable (a discard mid-turn is not a failure); a turn-level error transitions the session to a `.failed` state carrying a clean `AIError` headline — never silence, never a false "Done." - -## Capabilities - -### Modified Capabilities - -- `on-device-ai-runtime`: the runtime seam gains a **messages/conversation** entry point (`LLMChatRequest` + `LLMRuntime.chat()`) alongside the existing prompt-string `generate(_:)`/`structured(...)`; the canonical multi-turn conversation types (`AgentMessage`/`AgentConversation`/`AgentSessionID`/`AgentTurn`); chat-template assembly; per-turn channel storage; and **compaction** (windowing + a summarization model call) with the budget read through an injected provider. The existing single-prompt contract, the thinking-channel split, structured output, vision input, lifecycle, and capability routing are all preserved unchanged. - -## Impact - -- **Code (MLX-free Core, `swift test`):** new `AI/Agent/AgentConversation.swift` (the canonical types), `AI/Agent/ChatTemplate.swift` (the messages→prompt assembler used by the default `chat` impl), `AI/Agent/ContextBudget.swift` (the injected budget provider seam + a pure token estimator), `AI/Agent/ConversationCompactor.swift` (the pure windowing decision + the summarization-pass driver), and the `LLMChatRequest` value type + `chat` default-impl added to `AI/LLMRuntime.swift`. `AICommandExecutor` gains the session history, the new `State` cases, and a `continueConversation(_:)` / per-turn loop. `StubLLMRuntime` gains a multi-turn script queue. -- **Code (GemmaRuntime, `xcodebuild` compile-verify only):** the real Gemma chat-template wiring (the `enable_thinking` flag + Gemma's turn markers) is **flagged for the `ai-batched-runtime-and-context` / GemmaRuntime slice** — this slice ships only the Core flatten-default and the Stub; the real conformer's `chat` override lands there. No app build/sign/install by the agent. -- **Reuse, not rebuild:** `LLMRuntime`, `Token`/`TokenChannel`, `StructuredOutcome`, `RuntimeError`, `AIError.message(for:)`, `AICommandExecutor` + its `State`, `GenerationParameters`, `PromptTemplate` (for turn-1 seed text), `FireContext`. No new error taxonomy is needed — turn failures map through the existing `RuntimeError`/`AIError`. -- **Out of scope (owned by other slices):** the tool-route loop and `ToolRoute`/`ToolStepResult` (`ai-tool-routing`); the canvas multi-turn UX, the gesture compass, the float-up/park trigger (`ai-conversational-canvas`); durable on-disk conversation storage (`ai-parked-sessions` owns the store; this slice owns the `Codable` type); the `agentContextTokens` user slider + persisted key and the batched KV-reuse `chat` override (`ai-batched-runtime-and-context`); the real Gemma chat-template (GemmaRuntime). diff --git a/openspec/changes/ai-conversation-runtime/specs/on-device-ai-runtime/spec.md b/openspec/changes/ai-conversation-runtime/specs/on-device-ai-runtime/spec.md deleted file mode 100644 index 6c1a80d..0000000 --- a/openspec/changes/ai-conversation-runtime/specs/on-device-ai-runtime/spec.md +++ /dev/null @@ -1,127 +0,0 @@ -## MODIFIED Requirements - -### Requirement: Swappable model runtime abstraction -The system SHALL access all language-model functionality through a single `LLMRuntime` abstraction that exposes the runtime's capabilities (at least `text`, `vision`; later `audio`), a streaming **single-prompt** text-generation call, a streaming **multi-turn conversation** call that accepts a role-tagged message list, and a structured-output call that returns a typed, schema-validated value. Feature code (the band, the executor, the tasks) SHALL depend only on this abstraction and SHALL NOT reference any concrete model or framework directly, so that an additional model (another Gemma 4 size, a future Gemma, Apple Foundation Models, or a cloud model) can be added later as one new conformer without changing feature code. The multi-turn conversation call SHALL be **additive**: it SHALL be default-implemented in terms of the single-prompt call (flattening the message list into one prompt) so that every existing conformer keeps working unchanged, while a conformer MAY override it to reuse the model's key/value cache across turns. - -#### Scenario: Feature code is model-agnostic -- **WHEN** the band executor needs a result -- **THEN** it calls the `LLMRuntime` abstraction and never a concrete model type - -#### Scenario: Adding a model is additive -- **WHEN** a new model conformer is introduced -- **THEN** it can be selected without modifying the band, executor, or task code - -#### Scenario: The conversation call is additive over the single-prompt call -- **WHEN** an existing conformer that implements only the single-prompt generation call receives a multi-turn conversation request -- **THEN** the default conversation behavior assembles the message list into one prompt and serves it through the single-prompt call, so the conformer needs no change to participate in conversations - -## ADDED Requirements - -### Requirement: Canonical multi-turn conversation types -The system SHALL define one canonical set of conversation value types — a role-tagged **message** (user/assistant/system/tool), a stable **session identity**, a **conversation** (an ordered message list with a title and timestamps), and a **turn** (the per-generation unit) — in the MLX-free core so that every part of the AI feature (the runtime, the executor, tool routing, parked sessions, memory, and the conversational canvas) shares the same shapes rather than inventing conflicting ones. A message SHALL carry its committed text separately from the model's reasoning text, so that reasoning is structurally distinguishable from re-fed content. The session identity SHALL be stable across stashing and restoring a conversation. These types SHALL be encodable for durable storage (owned by the parked-sessions capability) without this capability owning the store. - -#### Scenario: One conversation type is shared across the feature -- **WHEN** any AI subsystem (runtime, executor, routing, parked sessions, memory, canvas) needs a message or a conversation -- **THEN** it uses the single canonical message/conversation/session-identity types rather than a locally-defined shape - -#### Scenario: Committed text and reasoning are separable on a message -- **WHEN** a message is constructed for an assistant turn that reasoned before answering -- **THEN** its committed-text field holds only the answer and its reasoning field holds the thinking, so the two never conflate - -#### Scenario: Session identity survives a stash and restore -- **WHEN** a conversation is stashed and later restored -- **THEN** it keeps the same session identity so every subsystem still refers to the same session - -### Requirement: Multi-turn session with channel-honest history -The executor SHALL evolve from a one-shot single-prompt fire into a multi-turn **session**: it SHALL hold the conversation history, append the user turn and the assistant turn, and continue the thread across turns. When appending the assistant turn, the executor SHALL store **only the response-channel** text as the turn's committed/re-fed content and SHALL store the thinking-channel text on the turn for **display only** — the thinking SHALL NEVER be re-fed into the model's context on a later turn. Assembling the context for a turn SHALL read each message's committed text only, so a turn's reasoning can never bloat the context window. The evolution SHALL be additive: the existing one-shot preset-command path and its states SHALL be preserved unchanged, and new session states (an in-flight conversing turn and an idle awaiting-next-turn state) SHALL be added alongside them. - -#### Scenario: A turn is appended to the running thread -- **WHEN** an assistant turn finishes generating within an open conversation -- **THEN** the assistant message is appended to the history and the session becomes ready for the next turn - -#### Scenario: Thinking is excluded from re-fed history -- **WHEN** a later turn's context is assembled from the conversation history -- **THEN** only each prior turn's committed response text is included and no prior turn's thinking text appears in the assembled context - -#### Scenario: A later turn sees earlier turns -- **WHEN** the user sends a second turn in an open conversation -- **THEN** the assembled context for that turn contains the prior user and assistant turns' committed text - -#### Scenario: The one-shot path is preserved -- **WHEN** a preset single-shot command is fired as before -- **THEN** it streams and commits exactly as it did prior to the session evolution, with its existing states unchanged - -### Requirement: Seed becomes the first turn -A conversation SHALL be opened from a **seed** — acquired input text and/or a per-turn image — that becomes the conversation's first user turn, assembled from the same input acquisition and prompt-template resolution the one-shot path uses. When the seed has neither non-empty text nor an image, the system SHALL surface the existing "no input" state and SHALL NOT open a conversation or invoke the model. - -#### Scenario: A text seed opens turn one -- **WHEN** a conversation is started from acquired input text -- **THEN** the conversation's first message is a user turn carrying that resolved text - -#### Scenario: An image seed opens turn one -- **WHEN** a conversation is started from a captured or clipboard image -- **THEN** the conversation's first user turn carries that image as its per-turn image - -#### Scenario: An empty seed opens no conversation -- **WHEN** a seed has empty/whitespace text and no image -- **THEN** the system surfaces "no input" and does not open a conversation or call the model - -### Requirement: Per-turn channels, cancellation, and failure -Each turn SHALL stream the model's reasoning and answer as the existing two channels — thinking and response — with the thinking streamed live to the reasoning display and reset at the start of every turn so a new turn never shows the prior turn's reasoning, while only the response accumulates into the committed turn. Each turn SHALL be individually cancellable: discarding a turn mid-flight SHALL stop generation promptly, SHALL append no partial assistant message to the history, and SHALL NOT be treated as a failure. A turn-level error (generation, decode, or a failed compaction summarization) SHALL transition the session to an observable failed state carrying a clean, user-facing headline from the single error translator — never silence and never a false "done." - -#### Scenario: Thinking streams live per turn and never commits -- **WHEN** a turn with reasoning enabled streams -- **THEN** its thinking streams to the reasoning display while only its response accumulates into the committed turn, and the next turn starts with the reasoning display cleared - -#### Scenario: A discarded turn leaves no partial message -- **WHEN** the user discards a turn before it completes -- **THEN** generation stops, no partial assistant message is added to the history, and the session is not marked failed - -#### Scenario: A turn error is observable, not silent -- **WHEN** a turn fails to generate or its compaction summarization fails -- **THEN** the session enters a failed state carrying a clean headline from the single error translator, and the conversation history is not silently dropped - -### Requirement: Context compaction by token budget -When the assembled context for a turn approaches a configured token budget, the system SHALL **compact** the conversation: it SHALL keep the most recent turns verbatim and summarize the older turns (and any existing summary) into a single compact summary via a model call, then SHALL replace those older raw turns with that summary, which SHALL be carried into subsequent turns' assembled context as a synthetic prefix turn. The summarization input SHALL exclude reasoning text (only committed text is summarized). The token budget SHALL be read through an **injected provider** rather than a concrete user setting, so that this capability does not depend on the batched-runtime/context capability landing first; the user-adjustable context size and the model's maximum context are owned by the batched-runtime/context capability and supplied through that provider. A compaction whose summarization call fails SHALL be a turn failure and SHALL NOT drop the raw history (so a retry still sees the full thread). - -#### Scenario: Old turns collapse into a summary near the budget -- **WHEN** the assembled context for the next turn would approach the token budget -- **THEN** the system summarizes the older turns into a compact summary and drops those raw turns, keeping the most recent turns verbatim - -#### Scenario: The summary is carried forward -- **WHEN** a conversation has been compacted -- **THEN** subsequent turns' assembled context includes the compact summary as a prefix in place of the dropped raw turns - -#### Scenario: The budget is injected, not hard-wired to a setting -- **WHEN** the compaction logic needs the token budget -- **THEN** it reads the budget through an injected provider, so it builds and runs without depending on the concrete user-facing context-size setting - -#### Scenario: A failed summarization does not lose history -- **WHEN** the compaction summarization model call fails -- **THEN** the turn enters the failed state and the raw conversation history is retained for a retry, rather than being dropped - -### Requirement: Deterministic multi-turn test runtime -The deterministic test runtime SHALL be scriptable for **multi-turn** conversations so that the full session machine can be exercised under automated tests without a model: it SHALL accept an ordered sequence of per-turn scripts (each with its own response and thinking chunks), serve them one per turn, allow a chosen turn to be cancelled mid-stream or to fail with a typed error, and allow the compaction summarization call's output to be scripted. When its multi-turn script is exhausted, it SHALL fall back to its existing single-generation behavior so existing single-shot tests remain unaffected. - -#### Scenario: A full conversation is scripted deterministically -- **WHEN** a test scripts a sequence of per-turn responses (and optionally per-turn thinking) -- **THEN** the runtime serves each turn's script in order so a multi-turn session can be asserted end-to-end - -#### Scenario: A scripted turn cancellation or failure is observable -- **WHEN** a test scripts a chosen turn to be cancelled mid-stream or to throw a typed error -- **THEN** the session reflects a benign discard (no partial message, not failed) for cancellation, or a failed state with a clean headline for the error - -#### Scenario: Single-shot tests are unaffected -- **WHEN** no multi-turn script is provided -- **THEN** the runtime behaves exactly as its existing single-generation scripting did - -### Requirement: Real chat-template assembly is deferred to the model conformer -The core SHALL provide a model-agnostic, deterministic message-list-to-prompt assembler that the default conversation call uses (a role-labeled transcript reading committed text only), and the **real** model-specific chat-template assembly — for the on-device Gemma conformer, its native turn markers and its `enable_thinking` reasoning flag — SHALL be provided by the model conformer's overriding conversation call, not by the core assembler. The core assembler SHALL never read reasoning text, so the default conversation path cannot leak thinking into the prompt. - -#### Scenario: The default assembler is model-agnostic and thinking-safe -- **WHEN** the default conversation call assembles a prompt from a message list -- **THEN** it produces a deterministic role-labeled transcript reading only committed text, with no message's thinking included - -#### Scenario: The Gemma chat-template lives in the conformer -- **WHEN** the on-device Gemma conformer serves a conversation with reasoning enabled -- **THEN** it builds the model's native chat template (its turn markers and the reasoning flag) in its overriding conversation call rather than relying on the core assembler diff --git a/openspec/changes/ai-conversation-runtime/tasks.md b/openspec/changes/ai-conversation-runtime/tasks.md deleted file mode 100644 index 6ece2dd..0000000 --- a/openspec/changes/ai-conversation-runtime/tasks.md +++ /dev/null @@ -1,53 +0,0 @@ -> Wave 1 of the V2 agent. §1–§3 are the canonical types + runtime seam (the type home other slices import); §4 is compaction; §5 evolves the executor into a session; §6 makes the Stub multi-turn; §7 is the tool-type hand-off seam; §8 verifies. All MLX-free Core unless noted — `swift build`/`swift test` is the verification, with `xcodebuild` compile-verify only to confirm the additive seam doesn't break the GemmaRuntime conformer. The agent NEVER builds/signs/installs the `.app`. - -## 1. Canonical conversation types (Core) - -- [x] 1.1 Add `AI/Agent/AgentConversation.swift` with `AgentRole`, `AgentMessage`, `AgentSessionID`, `AgentConversation`, `AgentTurn` EXACTLY as blueprint §3.1 / design D1 (Core, MLX-free, `Codable`/`Equatable`/`Sendable`; `AgentMessage`/`AgentConversation` also `Identifiable`). `text` = re-fed content; `thinking` = display-only; thinking is structurally separate so it is never re-fed. Verify: `swift test` Codable round-trip + equality. -- [x] 1.2 Document the invariant on the type (doc comment): assembly reads `text` only; `thinking` is never re-fed as ground truth. Verify: review + the assembly test (4.x/5.x). -- [x] 1.3 Unit-test the types: Codable round-trip preserves all fields incl. `compactedSummary`/`skillID`/`image`; `AgentSessionID` Hashable stable; equality. Verify: `swift test`. - -## 2. Runtime messages seam (Core) - -- [x] 2.1 Add `LLMChatRequest{messages, image, parameters, reasoning, tools}` to `AI/LLMRuntime.swift` (blueprint §3.2 / design D2), `Sendable`. `tools` declared in final shape, ignored by this slice. Verify: `swift build`. -- [x] 2.2 Add the default-implemented `extension LLMRuntime { func chat(_:) -> AsyncThrowingStream }` that flattens messages via `ChatTemplate.flatten` and calls `generate(_:)`, passing `image`/`reasoning`/`parameters` through (so channel split + vision are inherited). Verify: `swift test` (default path streams), `xcodebuild` (Gemma conformer + app still compile against the additive seam). -- [x] 2.3 Confirm `StubLLMRuntime`, `DevAIRuntime`, and the existing Gemma conformer compile UNCHANGED against the additive `chat` (no override required for them). Verify: `swift build` + `xcodebuild` compile-only. - -## 3. Chat-template assembler (Core) - -- [x] 3.1 Add `AI/Agent/ChatTemplate.swift` with `static func flatten(_ messages: [AgentMessage]) -> String` (design D3): deterministic role-labeled transcript reading `message.text` ONLY; `.tool` messages render `toolResult?.summary`; trailing `Assistant:` cue. Pure/`nonisolated`/static. Verify: `swift test`. -- [x] 3.2 Mark the real Gemma chat-template (`` markers + `enable_thinking` flag) as FLAGGED for the GemmaRuntime / `ai-batched-runtime-and-context` slice with a `// FLAGGED: GemmaRuntime` note; this slice ships only the Core flatten-default. Verify: review; spec scenario references the flag. -- [x] 3.3 Unit-test `flatten`: role ordering preserved; a turn's `thinking` NEVER appears in the output; tool-line rendering; trailing cue present; empty list → empty/cue-only. Verify: `swift test`. - -## 4. Compaction (Core — this slice OWNS it) - -- [x] 4.1 Add `AI/Agent/ContextBudget.swift`: `protocol ContextBudgetProviding { var maxContextTokens: Int }` (the INJECTED budget seam, integration fix C3 — never the concrete `agentContextTokens` slider), a pure `TokenEstimator` (estimates `text` only, with a safety margin), and a default constant-budget provider so the slice builds standalone. Verify: `swift test` (estimate grows with content; margin applied). -- [x] 4.2 Add `AI/Agent/ConversationCompactor.swift`: pure `needsCompaction(_:budget:) -> Bool` (assembled estimate incl. existing summary crosses the margin-adjusted budget) and pure `plan(_:budget:) -> CompactionPlan` (keep most-recent `keepRecentTurns` verbatim; collapse the older prefix incl. any prior summary into the to-summarize input). Verify: `swift test` (deterministic with a fixed-budget stub). -- [x] 4.3 Add `summarize(_ plan:runtime:) async throws -> String` — a single `runtime.generate` call (reasoning OFF) condensing the to-summarize slice; map errors through `RuntimeError`/`AIError`. Apply: set `compactedSummary`, replace `messages` with the kept tail. Verify: `swift test` (Stub-scripted summary; a thrown error propagates and does NOT drop history). -- [x] 4.4 Unit-test compaction end-to-end with a fixed-budget stub: a long thread triggers `needsCompaction`; `plan` keeps recent N; `summarize` produces the prefix; the next assembly carries the summary as a `system` prefix and the dropped raw turns are gone; thinking never enters the summary input. Verify: `swift test`. - -## 5. Executor → session (Core) - -- [x] 5.1 Add `private(set) var conversation: AgentConversation?` to `AICommandExecutor` (in-memory thread; durable storage is `ai-parked-sessions`). Verify: `swift build`. -- [x] 5.2 Add additive `State` cases `.conversing(partial: String)` and `.awaitingTurn` + extend the hand-written `==` for them; leave ALL existing one-shot cases (`.streaming`/`.ready`/`.committed`/`.reviewingAction`/`.declined`/`.failed`/`.unavailable`/`.noInput`) and `isCommittable` unchanged. (`.awaitingApproval`/`.parked` are owned by tool-routing/canvas — NOT added here.) Verify: `swift test` (existing one-shot tests stay green) + `swift build`. -- [x] 5.3 Add `startConversation(seedText:image:parameters:reasoning:)` — open `AgentConversation` with turn-1 user `AgentMessage` from the SAME acquisition + `PromptTemplate.resolve` seed the one-shot path uses; empty-and-imageless seed → `.noInput` (preserved). Verify: `swift test`. -- [x] 5.4 Add the private `runTurn()` loop (design D4): append the user turn, compact-if-needed (§4), assemble (§5.5), stream `chat` splitting `.thinking`→live `thinking` / `.response`→`.conversing(partial:)`, then append the assistant `AgentMessage(text: response, thinking: reasoning-or-nil)` and go `.awaitingTurn`. Per-turn cancellation: a mid-turn discard appends NO assistant message and is not a failure. A non-cancel error → `.failed(AIError.message(for:).headline)`. Verify: `swift test`. -- [x] 5.5 Add `assembleRequest()` (design D5): build `LLMChatRequest` from `conversation.messages` reading `text` ONLY, prefixing `compactedSummary` as a synthetic `system` message; latest image; resolved reasoning (reuse `command.resolvedReasoning(globalDefault:)`); `GenerationParameters` as today. Verify: `swift test` (turn-2 assembly contains turn-1 text; never any `thinking`). -- [x] 5.6 Add `continueConversation(_ userText:)` (the next-turn entry the canvas slice calls on Enter=send) → append user turn → `runTurn()`. Reset live `thinking` at each turn start so a new turn never shows the prior turn's reasoning. Verify: `swift test` (two-turn scripted conversation; live thinking resets between turns). -- [x] 5.7 Per-turn `.failed`: a scripted turn error transitions to `.failed` with a clean headline (never silence, never a false continuation); cancellation stays benign. Verify: `swift test`. - -## 6. Multi-turn Stub (Core) - -- [x] 6.1 Add `scriptedTurns: [TurnScript]` (FIFO, each `{tokens, thinking}`) to `StubLLMRuntime`; each `generate` dequeues one; queue exhausted → fall back to legacy `scriptedTokens` (existing single-gen tests byte-identical). Add a `scriptedSummary`/reserved-turn hook for the compaction call, and a per-turn throw hook for per-turn `.failed`. The default `chat` (flatten→generate) consumes the queue through `generate` — no Stub `chat` override. Verify: `swift test`. -- [x] 6.2 Keep per-call cancellation observation working per turn (a mid-turn discard stops emitting). Verify: `swift test` (cancel mid-turn-2; no assistant message appended). - -## 7. Tool-type hand-off seam (Core, temporary) - -- [x] 7.1 Introduce minimal placeholder value types `ToolRoute`/`ToolStepResult`/`ToolStepStatus`/`ToolDescriptor`/`WritePolicyTier` in `AI/Agent/ToolPlaceholders.swift` EXACTLY as blueprint §3.3/§3.7 sketch them, so `AgentMessage.toolCalls`/`.toolResult` and `LLMChatRequest.tools` compile standalone (design D9). Mark the file `// HAND-OFF: ai-tool-routing takes ownership; do not change the shapes here`. Verify: `swift build`. -- [x] 7.2 Document (in the file + the spec delta) that `ai-tool-routing` (Wave 2) takes ownership of these types and the route loop, with the shapes unchanged. Verify: review; cross-slice note in the spec. - -## 8. Verify - -- [x] 8.1 `swift build` + `swift test` green: the canonical types, `flatten`, compaction (windowing/plan/summarize), the executor session machine (multi-turn, per-turn cancel, per-turn fail, compaction trigger, thinking-exclusion), and the multi-turn Stub are all covered. Existing one-shot executor + runtime tests stay green. Verify: `swift test`. -- [x] 8.2 `xcodebuild` compile-verify the full `ThreeFingerSwitcher` product (Core + GemmaRuntime/MLX) so the additive `chat`/`LLMChatRequest` seam is confirmed not to break the existing Gemma conformer (compile ONLY — the agent never signs/installs). Verify: `xcodebuild` compile. -- [x] 8.3 `openspec validate --strict` passes; the `on-device-ai-runtime` delta (ADDED/MODIFIED requirements + scenarios) matches the implementation. Verify: `openspec validate --strict`. -- [ ] 8.4 **User run-verify** (stable-signed build, optional until a consumer slice surfaces the UX): a multi-turn conversation continues across turns with the model seeing prior turns; thinking shows live but is never re-fed; a long thread compacts without overflow; a mid-turn discard leaves no half-message. (This slice has no UI of its own — the canvas slice surfaces it; this is a smoke check via a debug harness if available.) diff --git a/openspec/changes/ai-full-potential-toggle/.openspec.yaml b/openspec/changes/ai-full-potential-toggle/.openspec.yaml deleted file mode 100644 index a4ac4d7..0000000 --- a/openspec/changes/ai-full-potential-toggle/.openspec.yaml +++ /dev/null @@ -1,2 +0,0 @@ -schema: spec-driven -created: 2026-06-23 diff --git a/openspec/changes/ai-full-potential-toggle/design.md b/openspec/changes/ai-full-potential-toggle/design.md deleted file mode 100644 index 4c08757..0000000 --- a/openspec/changes/ai-full-potential-toggle/design.md +++ /dev/null @@ -1,133 +0,0 @@ -## Context - -This is the **last** slice of the V2.5 compute/media/fleet wave (addendum §4 implementation order: it gates everything new + the two existing heavy slices). It OWNS the **master gate** (`fullPotentialEnabled`) + the five per-capability sub-flags (addendum §D1), the **pure gating logic** every heavy slice checks before activating, and the **disclosure UX** — one Hub page where each sub-toggle states its RAM / heat / latency / $-cost in the same breath it offers the capability. - -Read these before the design — the ground truth this slice plugs into, not forks: - -- **`docs/ai-agent-v2-addendum-compute-media-fleet.md` §D1** — pins the gate's exact key set: the master `fullPotentialEnabled` (default false) and the five sub-flags `cpuLaneEnabled` / `batchedRuntimeEnabled` / `mediaGenEnabled` / `backgroundAutonomyEnabled` / `fleetCloudEscalationEnabled`. "Each heavy slice CHECKS its flag before activating." This slice writes the real types behind those names. §5.6 (Default OFF) and §5.5 (a heavy gen evicts chat) are the honesty mandates this disclosure UX implements. -- **`docs/ai-agent-v2-blueprint.md`** — base conventions: one error taxonomy + one `AIError.message(for:)` translator, bounded + non-blocking surfacing, never `NSAlert.runModal`/raw-error-in-headline, reuse-don't-reinvent (`AppSettings`, the Hub page pattern), no degraded/low-end paths (M5/M4 only). -- **`openspec/specs/tunable-settings/spec.md`** — the existing opt-in pattern this gate mirrors: the **AI commands opt-in** (`enableAICommands`, default OFF, gates the band + model), the **clipboard/device-link** opt-ins (immediate, no relocation/permission/re-login), and the **reset-to-defaults** preserve-set (gesture relocations, clipboard/AI opt-ins, excluded apps, selected model are preserved, not zeroed). The Full Potential flags join that preserve-set. -- **`openspec/specs/configuration-hub/spec.md`** — the AI feature page already exists (the model-management section). This slice adds a **Full Potential** section to it, in the shared Liquid Glass language, mirroring how the launcher/Space-row opt-ins disable their dependent tunables. -- **Sibling slices (authored concurrently)** — `ai-compute-tiers` (`ComputeLane`/`LaneRouting`, §A1), `ai-media-runtime` (`MediaRuntime`, §B1) + `ai-local-image-generation`/`ai-video-animation-generation`, `ai-model-fleet` (`ModelRegistry`/`ModelDescriptor`, §C1), and the existing heavy `ai-batched-runtime-and-context` + `ai-background-autonomy`. This slice **references** their types as the things the sub-flags gate; it depends on the addendum's pinned `fullPotentialEnabled` (§D1), NOT on sibling change files existing yet. - -This slice owns no UI gesture and no recognizer state. It is a pure Core flag + gate that the heavy slices read and one Hub page renders. - -## Goals / Non-Goals - -**Goals:** -- One **master gate** `fullPotentialEnabled` (default OFF → V2.5 ships calm) + five per-capability sub-flags, each persisted, default OFF, preserved by reset like the other AI opt-ins. -- A pure, total **`FullPotentialGate`** (`swift test`-verified) that answers "is capability X unlocked": `master ∧ subFlag ∧ aiCommandsEnabled`. Turning the master OFF closes every gate at once (the calm panic-off). Never builds/signs/touches MLX. -- A **disclosure UX**: each sub-toggle states its RAM / heat / latency / $-cost inline, persistently (not behind a tooltip). The media + cloud rows state the hard truths plainly (chat eviction; real $ + data off-device, budget-capped). -- **Progressive enablement** on one Hub page: master first, then five sub-toggles disabled-and-visibly-relocked until the master is on; flipping the master OFF relocks all five while RETAINING their values. -- The **weld** specified as **ADDED** shared-contract requirements on `tunable-settings`/`configuration-hub` (the capabilities this slice owns) — **without editing the heavy slices' files**; each owning slice authors its own check-site requirement. - -**Non-Goals:** -- The sub-capabilities' behavior, runtimes, residency math, costs-as-incurred, or any UI beyond the gate + its disclosure (each owned by its slice). This slice *discloses* costs; the owning slices *incur* them. -- The AI-commands master opt-in + model download + Calendar prompt (already `tunable-settings`; the gate consumes `enableAICommands`, it does not redefine it). -- Defining `ComputeLane`/`MediaRuntime`/`ModelRegistry`/`ModelDescriptor`/`WritePolicyTier`/`BatchedLLMRuntime` (owned by siblings) — referenced verbatim, never redefined. -- Changing the reset-to-defaults *mechanism* — the new keys join the existing AI opt-in preserve-set; reset's behavior is unchanged. -- Any Intel/low-end fallback, degraded path, new permission, gesture relocation, or re-login. Apple-Silicon M5 (M4 min) only. - -## Decisions - -### 1. The flag set — one master, five sub-flags, exactly the §D1 names - -The persisted shape (in `AppSettings`), each a `Bool` default `false`: - -| Key | Gates | Owning slice | Honest cost the row discloses | -|---|---|---|---| -| `fullPotentialEnabled` | **the master** | this slice | "Lights up the agent fleet. Each capability below states its own cost." | -| `cpuLaneEnabled` | the CPU ternary lane | `ai-compute-tiers` | **Heat / battery** — a second (CPU) lane runs concurrently; short structured bursts only, CPU per-token is slower. | -| `batchedRuntimeEnabled` | K-stream GPU batched runtime + growable context | `ai-batched-runtime-and-context` | **RAM + latency** — multiplexes K sessions over one weight read; larger context = more resident KV; latency rises under load. | -| `mediaGenEnabled` | image/video generation tools | `ai-media-runtime` + backends | **RAM (eviction) + latency + disk** — a heavy gen **evicts chat** ("the assistant goes quiet while it paints"); minutes per clip; tens of GB of weights. | -| `backgroundAutonomyEnabled` | parked auto-vs-escalate + whitelist + audit | `ai-background-autonomy` | **Unattended action** — the agent may act while you are away (whitelisted/contained writes only; dangerous ones still escalate; all audited). | -| `fleetCloudEscalationEnabled` | cloud members (Claude / GLM-5.2) | `ai-model-fleet` | **\$ + network + data off-device** — sends prompts to a paid cloud model; **budget-capped + audited**; off until armed. | - -**Rationale:** the addendum (§D1, §1 persisted keys) pins these exact names; using them verbatim lets each sibling reference `settings.cpuLaneEnabled` etc. without negotiation. Five flags (not one per micro-feature) match the five heavy capabilities; the two image/video backends ride the single `mediaGenEnabled` because they share the `MediaRuntime` seam and the media-gen Hub experience is one concept to the user. - -**Alternatives rejected:** (a) One flat list of independent opt-ins with no master — rejected: loses the single deliberate "release full potential" act and the one-switch panic-off; the calm default would be six separate "off"s the user must individually trust. (b) A single master with no sub-flags — rejected: the user cannot, say, allow the CPU lane (cheap) while keeping cloud spend (expensive) off; progressive, cost-aware enablement needs per-capability granularity. (c) Sub-flags persisted under a nested dictionary — rejected: flat camelCase `Bool`s match every existing `AppSettings` opt-in and the addendum's pinned key names. - -### 2. The gating rule — `master ∧ subFlag ∧ aiCommandsEnabled`, total and pure - -```swift -// Core, MLX-free. The single source of truth for "is capability X actually unlocked." -public enum FullPotentialCapability: String, CaseIterable, Codable, Sendable { - case cpuLane // ai-compute-tiers → cpuLaneEnabled - case batchedRuntime // ai-batched-runtime-and-context → batchedRuntimeEnabled - case mediaGen // ai-media-runtime + backends → mediaGenEnabled - case backgroundAutonomy // ai-background-autonomy → backgroundAutonomyEnabled - case fleetCloud // ai-model-fleet cloud members → fleetCloudEscalationEnabled -} - -public struct FullPotentialFlags: Equatable, Sendable { - public var aiCommandsEnabled: Bool // the existing AI feature opt-in (enableAICommands) - public var fullPotentialEnabled: Bool // the master - public var cpuLane: Bool - public var batchedRuntime: Bool - public var mediaGen: Bool - public var backgroundAutonomy: Bool - public var fleetCloud: Bool -} - -public struct FullPotentialGate: Sendable { - public let flags: FullPotentialFlags - public func isUnlocked(_ capability: FullPotentialCapability) -> Bool { - // master closed → every capability closed (the calm panic-off) - guard flags.aiCommandsEnabled, flags.fullPotentialEnabled else { return false } - switch capability { - case .cpuLane: return flags.cpuLane - case .batchedRuntime: return flags.batchedRuntime - case .mediaGen: return flags.mediaGen - case .backgroundAutonomy: return flags.backgroundAutonomy - case .fleetCloud: return flags.fleetCloud - } - } -} -``` - -**Rationale:** the fleet is a strict **subset** of the AI feature — if the AI-commands opt-in is off there is no model resident at all, so every fleet capability is meaningless; folding `aiCommandsEnabled` into the gate means a heavy slice does ONE check (`gate.isUnlocked(.mediaGen)`) instead of three scattered guards, and the master closing closes everything in one place. The gate is **total** (no throw, no async, no IO) so it is trivially `swift test`-able and can be called on any thread, in any sink, before activating. `FullPotentialCapability` is `CaseIterable` so the Hub can render the five rows by iterating, and a test can assert every case is gated. - -**Alternatives rejected:** (a) Five free functions instead of an enum+gate — rejected: an enum gives the Hub one render loop and the tests one exhaustiveness check; a stringly-typed flag name invites typos. (b) Resolving the gate inside each sub-flag's setter (so `cpuLaneEnabled` auto-falses when the master is off) — rejected: that *destroys* the user's sub-flag choices on panic-off; the spec requires values be **retained** (inert) so re-arming the master restores the prior selection (mirrors the launcher tunables going inert, not zeroed). The gate computes unlock at read time; it never mutates the stored flags. (c) Omitting `aiCommandsEnabled` and trusting callers to also check it — rejected: scatters the invariant and risks a fleet capability activating with no model. - -### 3. Default OFF, persistence, and reset — join the AI opt-in preserve-set - -All six keys default **false**. They persist in `AppSettings` exactly like `enableAICommands` / `keepClipboardHistory` / `enableDeviceLink`. Settings written before this wave load with all six OFF (no key present → false), like every prior opt-in's legacy-load. - -On **reset-to-defaults**, the six keys are **preserved** (not reset to off), joining the existing preserve-set (gesture relocations, clipboard/AI opt-ins, excluded apps, selected model) — for the same reason: re-acquiring full potential is a deliberate, possibly costly act (a media download, a cloud-budget decision), and a reset should not silently re-arm or silently disarm a fleet the user deliberately configured. **Rationale:** consistency with the documented reset semantics (`tunable-settings` Requirement "Feature pages preserve all tunables and persistence" / the configuration-hub Danger-zone reset). **Alternative rejected:** resetting the flags to off on reset-to-defaults — rejected: a reset is a *tunable* reset, and these are opt-ins (the spec already excludes opt-ins from reset); zeroing them would diverge from the AI-commands opt-in's own preserved behavior. - -### 4. Disclosure UX — cost in the same breath, never behind a tooltip - -Each sub-toggle row carries a **persistent, always-visible cost line** (RAM / heat / latency / $, per Decision 1's table), rendered as the row's caption beneath its title — not a hover tooltip, not a disclosure the user might never open. The two highest-cost rows state the hard truths in plain words: **mediaGen** — "the assistant goes quiet while it paints" (a heavy gen evicts chat, addendum §5.5); **fleetCloud** — "spends real money and sends data off-device; budget-capped + audited" (addendum §5.6, the Claude-handoff honesty pattern). - -**Rationale:** the project's honest-surface ethos (CLAUDE.md: never let a fan scream / a bill arrive / the assistant fall silent as a surprise) applied to *capability cost*. A cost the user must hunt for is a hidden cost. **Alternatives rejected:** (a) Cost behind an info "i" popover — rejected: a popover is opt-in attention; the mandate is "in the same breath." (b) A single shared disclaimer for the whole section — rejected: each capability's cost differs in kind (heat vs RAM vs $); a per-row line is the truthful granularity. (c) Showing live RAM/heat telemetry — rejected: out of scope (no measurement subsystem here) and the disclosure is about *what the capability costs in principle*, which is stable copy, not a live gauge. - -### 5. Progressive enablement + panic-off — master gates the rows visually; values retained - -On the Hub Full Potential section: the **master toggle renders first**; the five sub-toggles render **disabled (visibly relocked)** whenever the master is off. Flipping the master OFF relocks all five **in the UI** while **retaining** their persisted values (so re-arming the master restores the prior selection). This mirrors the existing pattern where launcher/Space-row tunables are inert (no behavioral effect) while their opt-in is off, but their stored values survive. - -**Rationale:** matches both the gate logic (Decision 2 reads, never mutates) and the established Hub idiom (disabled-but-reachable controls, `configuration-hub` "Disabled feature page still reachable"). **Alternative rejected:** hiding the sub-rows entirely while the master is off — rejected: the user can't preview what releasing full potential would offer, and the configuration-hub spec prefers *disabled-and-shown* over *hidden* for gated controls. - -### 6. The weld — one boolean consult per heavy slice, specified not edited - -Each heavy slice adds exactly **one** `gate.isUnlocked(capability)` consult before activating its runtime/lane/sink/cloud member; when the gate is closed the slice behaves as it does today with its feature off (no lane, no batched runtime, no media tool registered, no background auto-run, no cloud member resident/dispatchable). This change specifies the welds as: -- **ADDED** requirements on `tunable-settings` (the new Full Potential keys, the gate rule, and the shared "each heavy capability consults the gate" contract + per-flag persistence/default/reset) — these are genuinely new (no existing `tunable-settings` requirement names Full Potential or the gate, so this is a TRUE delta, not a rewrite), -- **ADDED** requirement on `configuration-hub` (the new Hub Full Potential section — also genuinely new; it neither renames nor rewrites the existing AI-page model-management requirements), -- and it does **NOT** edit `ai-compute-tiers` / `ai-batched-runtime-and-context` / `ai-media-runtime` / `ai-background-autonomy` / `ai-model-fleet` source or spec files. Those slices consume `FullPotentialGate` (a Core type), check it, and author the per-slice check-site requirement in their OWN deltas; the shared gate type + cross-slice contract live here. - -**Rationale:** the addendum (§D1) says each heavy slice CHECKS its flag; centralizing the *gate type* + the *contract* here, while leaving the *check site* to each slice, keeps this slice from reaching into sibling files (which would conflict with concurrent authorship). **Alternative rejected:** this slice editing each heavy slice to insert the check — rejected: violates concurrent-authorship isolation and the binding rule ("without editing those slices' files"). - -### 7. No new error type — the gate is total - -The gate does not throw; it returns a `Bool`. A capability being locked is **not** an error — it is the calm default. So no `Error` is born here. The only failure surface is a persistence read/write of `AppSettings`, which is already the existing settings store's concern and already routes through the app's settled error path; this slice adds no new boundary. **Rationale:** the blueprint allows a new error enum *only if* `RuntimeError`/`TaskError` cannot carry it — here there is no failure to carry. **Alternative rejected:** a `FullPotentialError.locked` — rejected: locked is expected steady state, not an error; surfacing it as one would violate "a failure is observable `.failed`, never a false anything" by inventing a false failure. - -## Target-split & verification - -| Component | Target | Verified by | -|---|---|---| -| `AI/FullPotential/FullPotentialGate.swift` — `FullPotentialCapability`, `FullPotentialFlags`, `FullPotentialGate.isUnlocked(_:)` (pure resolver) | **MLX-free Core** | `swift build` + `swift test` — exhaustive truth table per capability (master off ⇒ all locked; ai-commands off ⇒ all locked; each sub-flag gates only its capability; `CaseIterable` exhaustiveness). | -| `AppSettings` persisted keys — `fullPotentialEnabled` + the five sub-flags (default false; legacy-load false; reset preserve-set) | **MLX-free Core** | `swift build` + `swift test` — defaults false; persist round-trip; legacy decode with keys absent ⇒ false; reset-to-defaults preserves all six (no new default leak). | -| `FullPotentialFlags`-from-`AppSettings` adapter (maps the stored keys + `enableAICommands` into the gate's input) | **MLX-free Core** | `swift test` — a known settings fixture maps to the expected `FullPotentialFlags`; `enableAICommands` flows into `aiCommandsEnabled`. | -| Hub **Full Potential** section (`Hub/` view code) — master toggle, five cost-disclosing sub-toggle rows, disabled-until-master, panic-off relock, Liquid Glass | **App target (native-linked)** | `xcodebuild` **compile-verify only** for the agent; **the user run-verifies** on a stable-signed build: rows disabled until master on, panic-off relocks while retaining values, cost lines visible inline. (An agent never builds/signs the `.app` — ad-hoc signing breaks TCC.) | -| The weld (each heavy slice's one `gate.isUnlocked` consult) | **the OWNING heavy slice's target** (Core or native, per slice) | The shared contract is specified here as an **ADDED** `tunable-settings` requirement; the per-slice **check site** is **implemented + verified in each owning slice**, not in this change's files. | - -No component of this slice links MLX, downloads weights, spawns a process, or touches the build/sign path. The gate, the flags, and the adapter are pure Core (the majority of the slice). Only the Hub section needs the real app, and only for visual/interaction run-verification by the user. diff --git a/openspec/changes/ai-full-potential-toggle/proposal.md b/openspec/changes/ai-full-potential-toggle/proposal.md deleted file mode 100644 index 7a9b7d8..0000000 --- a/openspec/changes/ai-full-potential-toggle/proposal.md +++ /dev/null @@ -1,36 +0,0 @@ -## Why - -V2.5 evolves the on-device agent from a single-GPU, single-model, text-only companion into a **two-lane (GPU+CPU), multi-model, media-generating** fleet (see `docs/ai-agent-v2-addendum-compute-media-fleet.md`). Each new capability is real work the hardware can serve — but each also has a **real cost the user feels**: a CPU ternary lane (more heat, more battery), a batched runtime (more resident RAM, latency under load), local image/video generation (the companion **goes quiet while it paints** — a heavy gen *evicts chat* under the 48 GB budget — plus minutes-per-clip and tens of gigabytes of weights), background autonomy (the agent acts while you are away), and cloud escalation (Claude / GLM-5.2 — **real dollars**, network, data leaving the device). - -The project's ethos is the **honest surface**: never let a fan scream, a bill arrive, or the assistant fall silent as a *surprise*. So the entire fleet ships **calm by default**. There is one deliberate, user-owned act — **Release Full Potential** — that lights it up, and every sub-capability discloses its RAM / heat / latency / $-cost **in the same breath** it offers itself. Nothing escalates silently; cloud spend stays off until explicitly armed. - -This is also the single **gating contract** the heavy slices depend on. Each heavy slice (the existing `ai-batched-runtime-and-context` and `ai-background-autonomy`, and the new `ai-compute-tiers`, `ai-media-runtime` + its backends, and `ai-model-fleet`'s cloud members) must **check a flag before activating**. This slice OWNS that flag set, the **pure gating logic** that resolves "is capability X actually unlocked," and the **one Hub page** that presents it — so the gate-and-consult contract is specified here as **new (ADDED) requirements** on `tunable-settings`/`configuration-hub` (the capabilities this slice owns the keys + Hub page on), **without editing the heavy slices' own files**. The heavy slices' deltas (their per-slice files) carry the actual check-site requirement; this slice contributes the shared gate type + the cross-slice contract those checks honor. - -## What Changes - -- **The master gate `fullPotentialEnabled`** (`AppSettings`, default **OFF**) and its five per-capability sub-flags, each gated *under* the master (addendum §D1): `cpuLaneEnabled` (`ai-compute-tiers`), `batchedRuntimeEnabled` (`ai-batched-runtime-and-context`), `mediaGenEnabled` (`ai-media-runtime` + backends), `backgroundAutonomyEnabled` (`ai-background-autonomy`), `fleetCloudEscalationEnabled` (`ai-model-fleet` cloud members). All default OFF; all persisted; all preserved by reset-to-defaults like the other AI opt-ins. -- **A pure `FullPotentialGate` (MLX-free Core, `swift test`-verified).** The single source of truth for "is capability X unlocked": a sub-capability is unlocked **only when the master is on AND its own sub-flag is on AND the AI-commands opt-in is on** (the fleet is a strict subset of the AI feature). Turning the master OFF closes **every** gate at once (the calm "panic-off" — the fleet goes quiet without per-flag fiddling). Each heavy slice consults this gate via one boolean check before activating; the gate never builds, signs, or touches MLX. -- **The disclosure UX — cost stated in the same breath as the capability.** Each sub-toggle row states, inline and persistently, its **RAM** (resident weight footprint / eviction consequence), **heat/battery**, **latency** (per-token / per-clip), and **$-cost** (cloud only). The media + cloud rows state the hard truths plainly: a heavy gen **evicts chat** ("the assistant goes quiet while it paints"); cloud escalation **spends real money and sends data off-device**, and is **budget-capped + audited** (the Claude-handoff pattern). No cost is hidden behind a tooltip the user might never open. -- **Progressive enablement on one Hub page.** A new **Full Potential** section on the Hub AI feature page: the master toggle first, then the five sub-toggles, each **disabled (and visibly so) until the master is on**. Flipping the master OFF visibly relocks all five (their persisted values are retained, just inert) — mirroring how the launcher/Space-row opt-ins gate their tunables. -- **The gate welds, specified without editing sibling files.** This change's spec deltas state, as **ADDED requirements** on `tunable-settings`/`configuration-hub` (the capabilities this slice owns), the shared contract that each heavy slice's activation is conditioned on `FullPotentialGate.isUnlocked(...)`. The deltas do NOT touch the heavy slices' own capabilities (`on-device-ai-runtime`, `ai-generative-media`, …) — the per-slice check-site requirement is authored in each sibling's files. The sibling slices consume the gate; they are authored concurrently and depend on the addendum's pinned `fullPotentialEnabled` (§D1), not on this change's files existing yet. - -## Capabilities - -### New Capabilities - -_None._ This slice OWNS the master gate but contributes it as deltas to existing capabilities (per addendum §1, the toggle slice writes against `configuration-hub` and `tunable-settings`, not a new spec dir). - -### Modified Capabilities - -- `tunable-settings`: new persisted **Full Potential** keys — the master `fullPotentialEnabled` (default OFF) and the five sub-flags (`cpuLaneEnabled`, `batchedRuntimeEnabled`, `mediaGenEnabled`, `backgroundAutonomyEnabled`, `fleetCloudEscalationEnabled`, all default OFF), with the pure **gating rule** (`master ∧ subFlag ∧ aiCommandsEnabled`), their default-OFF / persistence / reset semantics (preserved like the other AI opt-ins, not zeroed), legacy-load behavior, and the contract that **each heavy slice consults the gate before activating**. -- `configuration-hub`: the AI feature page gains a **Full Potential** section — the master toggle plus the five cost-disclosing sub-toggles (each disabled until the master is on), the panic-off relock, and the honest cost lines, all in the shared Liquid Glass presentation. - -## Impact - -- **Code (MLX-free Core unless noted):** new `AI/FullPotential/FullPotentialGate.swift` (the pure `master ∧ subFlag ∧ aiCommandsEnabled` resolver + a `FullPotentialCapability` enum keyed to the five sub-flags) — `swift test`-verified with no weights. New `AppSettings` persisted keys (the master + five sub-flags). The Hub **Full Potential** section (`Hub/`) is App-target view code (`xcodebuild` compile-verify; the user run-verifies the disable/relock + disclosure on a stable-signed build). -- **Consumes verbatim (never redefines):** `fullPotentialEnabled` + the five sub-flag key names (addendum §D1); the existing AI-commands opt-in (`enableAICommands`) from `tunable-settings`; `AIError.message(for:)`/`AIPresentedError` only if a (rare) gate-surfacing error is ever needed (the gate itself is total and does not throw). It references — but does not define — `ComputeLane`/`LaneRouting` (§A1), `MediaRuntime` (§B1), `ModelRegistry`/`ModelDescriptor` (§C1) as the things the sub-flags gate. -- **The weld is checks, not rewrites.** Each heavy slice adds **one boolean consult** of `FullPotentialGate` before activating its runtime/lane/sink/cloud member. This change specifies the shared "every heavy capability consults the gate" contract as an **ADDED requirement** on `tunable-settings` (a capability this slice owns); it does **not** edit `ai-compute-tiers`, `ai-batched-runtime-and-context`, `ai-media-runtime`, `ai-background-autonomy`, or `ai-model-fleet` source/spec files — each owning slice adds its own check-site requirement. -- **MLX-free Core:** the gate, the capability enum, and the resolution rule are pure and `swift test`-able; the Hub views need the real app (compile-verify via `xcodebuild`, run-verify by the user — an agent never builds/signs the `.app`; ad-hoc signing breaks TCC). -- **No signing, no permission, no TCC interaction, no gesture relocation, no re-login.** Like the clipboard/device-link opt-ins, every Full Potential flag takes effect immediately when toggled; none relocates a native gesture or requests a new permission (the underlying AI feature already owns the model download + Calendar prompt). -- **Honest costs surfaced (not incurred by this slice):** CPU lane = heat/battery; batched runtime = resident RAM + latency under load; media = chat eviction + minutes/clip + GB of weights; background autonomy = the agent acts unattended (audited); cloud escalation = $, network, data off-device (budget-capped). This slice *discloses* them; the owning slices *incur* them. -- **Out of scope:** the sub-capabilities' own behavior, runtimes, residency math, or UI beyond the gate/disclosure (each owned by its slice); the AI-commands master opt-in + model download (already `tunable-settings`); any change to the existing reset-to-defaults *mechanism* (the new keys join the AI opt-in preserve-set, they do not alter how reset works). diff --git a/openspec/changes/ai-full-potential-toggle/specs/configuration-hub/spec.md b/openspec/changes/ai-full-potential-toggle/specs/configuration-hub/spec.md deleted file mode 100644 index ba1b449..0000000 --- a/openspec/changes/ai-full-potential-toggle/specs/configuration-hub/spec.md +++ /dev/null @@ -1,41 +0,0 @@ -## ADDED Requirements - -### Requirement: The AI page hosts the Full Potential gate with cost disclosure - -The Hub's **AI** feature page SHALL provide a **Full Potential** section that is the single surface for releasing and configuring the heavy AI capabilities. The section SHALL present, first, the **Release Full Potential** master toggle (`fullPotentialEnabled`), and beneath it the five per-capability sub-toggles — the **CPU lane** (`cpuLaneEnabled`), the **batched runtime** (`batchedRuntimeEnabled`), **media generation** (`mediaGenEnabled`), **background autonomy** (`backgroundAutonomyEnabled`), and **cloud escalation** (`fleetCloudEscalationEnabled`). All toggles SHALL persist their values with the same defaults (all OFF) and the same reset-to-defaults preservation as the other AI opt-ins, and SHALL use the shared **Liquid Glass** presentation consistent with the rest of the Hub. - -Each sub-toggle SHALL state its **cost** — RAM, heat/battery, latency, and dollar cost as applicable — **inline and always visible** (as the row's caption, not behind a tooltip or a disclosure the user might never open), in the same breath it offers the capability. In particular the **media generation** row SHALL state plainly that a heavy generation makes the assistant go quiet while it paints (it evicts chat under the memory budget), and the **cloud escalation** row SHALL state plainly that it spends real money and sends data off-device and is budget-capped and audited. No Full Potential cost SHALL be hidden from the row that offers it. - -#### Scenario: The Full Potential section is reachable on the AI page - -- **WHEN** the user opens the Hub and selects the AI feature page -- **THEN** a Full Potential section shows the master toggle followed by the five sub-toggles (CPU lane, batched runtime, media generation, background autonomy, cloud escalation) - -#### Scenario: Every sub-toggle discloses its cost inline - -- **WHEN** the Full Potential section is shown -- **THEN** each sub-toggle row displays its RAM / heat / latency / dollar cost as an always-visible caption, with the media row stating it evicts chat (the assistant goes quiet while it paints) and the cloud row stating it spends real money and sends data off-device (budget-capped + audited) - -#### Scenario: Toggling persists and is preserved by reset - -- **WHEN** the user releases full potential and enables a sub-capability, then relaunches -- **THEN** the selections are restored, and a reset-to-defaults preserves them like the other AI opt-ins - -### Requirement: The Full Potential sub-toggles are progressively enabled by the master - -The Hub's Full Potential section SHALL gate the five sub-toggles behind the master: while the master `fullPotentialEnabled` is **off**, the five sub-toggles SHALL be shown but **disabled (visibly relocked)**, mirroring how a disabled feature's controls remain reachable but inert. Turning the master **off** SHALL relock all five sub-toggles in the UI while **retaining** their persisted values, so re-arming the master restores the prior selection rather than clearing it. - -#### Scenario: Sub-toggles disabled until the master is on - -- **WHEN** the Full Potential section is shown with the master off -- **THEN** the five sub-toggles appear disabled (visibly relocked) and cannot be changed until the master is turned on - -#### Scenario: Panic-off relocks without clearing - -- **WHEN** the user has the master and some sub-toggles on and turns the master off, then turns it back on -- **THEN** the sub-toggles relock while the master is off, and on re-arming the master the previously-enabled sub-toggles are restored to on (their values were retained, not cleared) - -#### Scenario: A Full Potential persistence failure is non-blocking - -- **WHEN** persisting or loading a Full Potential setting fails -- **THEN** the page surfaces a bounded, non-blocking message with a clean headline (details behind an opt-in disclosure) and the rest of the page stays usable, with no app-modal alert and no raw error text in the headline diff --git a/openspec/changes/ai-full-potential-toggle/specs/tunable-settings/spec.md b/openspec/changes/ai-full-potential-toggle/specs/tunable-settings/spec.md deleted file mode 100644 index f2d8f51..0000000 --- a/openspec/changes/ai-full-potential-toggle/specs/tunable-settings/spec.md +++ /dev/null @@ -1,75 +0,0 @@ -## ADDED Requirements - -### Requirement: Release Full Potential master gate and sub-capability flags - -The settings SHALL expose a **Release Full Potential** master opt-in, `fullPotentialEnabled`, that defaults to **OFF**, and five per-capability sub-flags it unlocks — `cpuLaneEnabled` (the CPU ternary lane), `batchedRuntimeEnabled` (the K-stream batched GPU runtime + growable context), `mediaGenEnabled` (image/video generation), `backgroundAutonomyEnabled` (parked auto-vs-escalate, whitelist, audit), and `fleetCloudEscalationEnabled` (cloud fleet members such as Claude / GLM-5.2) — each also defaulting to **OFF**. Like the clipboard-history and device-link opt-ins, none of these flags SHALL relocate any native gesture, require a re-login, or request a new permission; each SHALL take effect immediately when toggled. They SHALL persist across launches, and settings written before this feature SHALL load with all six OFF (no key present reads as false), leaving existing settings unchanged. - -The flags SHALL be **preserved by reset-to-defaults** exactly like the other AI opt-ins (the AI-commands opt-in, the selected model): a reset SHALL NOT turn them on, nor turn them off — re-acquiring full potential is a deliberate, possibly costly act and a tunable reset SHALL NOT silently re-arm or disarm a configured fleet. - -#### Scenario: Master and sub-flags default off - -- **WHEN** the app loads with no prior Full Potential settings -- **THEN** `fullPotentialEnabled` and all five sub-flags (`cpuLaneEnabled`, `batchedRuntimeEnabled`, `mediaGenEnabled`, `backgroundAutonomyEnabled`, `fleetCloudEscalationEnabled`) are OFF - -#### Scenario: Toggling needs no re-login, permission, or gesture change - -- **WHEN** the user turns the master or any sub-flag on -- **THEN** it takes effect immediately, with no re-login, no native-gesture relocation, and no new permission prompt - -#### Scenario: Flags persist across launches - -- **WHEN** the user enables the master and a sub-flag and relaunches -- **THEN** both remain enabled and are reapplied - -#### Scenario: Legacy settings load with the flags off - -- **WHEN** settings written before this feature are loaded -- **THEN** all six keys read as false (no key present) and the existing settings are not reset - -#### Scenario: Reset to defaults preserves the flags - -- **WHEN** the user has enabled the master and some sub-flags and then resets to defaults -- **THEN** the six Full Potential flags retain their values (like the other AI opt-ins), while ordinary tunables return to their defaults - -### Requirement: Full Potential gating rule resolves capability unlock - -The system SHALL provide a pure, total gating rule that resolves whether a given Full Potential capability is **unlocked**: a capability SHALL be unlocked **only when** the AI-commands opt-in is on **AND** the master `fullPotentialEnabled` is on **AND** that capability's own sub-flag is on. When the master is OFF, **every** capability SHALL be locked at once (a single panic-off), and when the AI-commands opt-in is OFF every capability SHALL be locked (the fleet is a strict subset of the AI feature, which owns the resident model). The rule SHALL be pure (no side effects, no IO, no model linkage): it SHALL NOT mutate the stored flags — turning the master off SHALL relock the capabilities by computation while **retaining** each sub-flag's stored value, so re-arming the master restores the prior selection. - -#### Scenario: Master off locks everything - -- **WHEN** the AI-commands opt-in is on, every sub-flag is on, but `fullPotentialEnabled` is OFF -- **THEN** every Full Potential capability resolves as locked - -#### Scenario: AI-commands off locks everything - -- **WHEN** the AI-commands opt-in is OFF -- **THEN** every Full Potential capability resolves as locked regardless of the master or sub-flags - -#### Scenario: A sub-flag gates exactly its own capability - -- **WHEN** the AI-commands opt-in and the master are both on and only `mediaGenEnabled` is on among the sub-flags -- **THEN** the media-generation capability resolves as unlocked and no other capability does - -#### Scenario: Panic-off retains sub-flag values - -- **WHEN** the user has several sub-flags on, turns the master OFF, then turns the master back ON -- **THEN** the previously-on sub-flags are still on (their stored values were retained, not zeroed) and their capabilities resolve as unlocked again - -### Requirement: Each heavy capability consults the gate before activating - -Every gated heavy capability SHALL consult the Full Potential gating rule for its own capability and SHALL activate **only** when that capability resolves as unlocked; when it resolves as locked the capability SHALL behave exactly as it does with its feature off (no activation, no resident weights for it, no registered tool, no background action, no cloud member dispatched). Specifically: the CPU ternary lane SHALL NOT run unless `cpuLane` is unlocked; the batched runtime + growable context SHALL NOT activate unless `batchedRuntime` is unlocked; the image/video generation tools SHALL NOT be available unless `mediaGen` is unlocked; parked background auto-execution SHALL NOT run unless `backgroundAutonomy` is unlocked; and a cloud fleet member SHALL NOT be dispatchable unless `fleetCloud` is unlocked. This consult SHALL be the single gate each heavy slice checks; it SHALL NOT require any heavy slice to re-implement the master/AI-commands conjunction. - -#### Scenario: A locked capability does not activate - -- **WHEN** the master is off (so a capability is locked) and the agent would otherwise use that capability -- **THEN** the capability does not activate and the agent behaves as if that capability's feature were off - -#### Scenario: Cloud escalation stays off until armed - -- **WHEN** `fleetCloudEscalationEnabled` is off (or the master is off) -- **THEN** no cloud fleet member is dispatchable, no cloud spend can occur, and no data leaves the device for a fleet capability - -#### Scenario: Unlocking one capability does not unlock the others - -- **WHEN** the user unlocks only the CPU lane (`cpuLaneEnabled` on, master + AI-commands on, other sub-flags off) -- **THEN** the CPU ternary lane may run while the batched runtime, media generation, background autonomy, and cloud escalation all remain locked and inactive diff --git a/openspec/changes/ai-full-potential-toggle/tasks.md b/openspec/changes/ai-full-potential-toggle/tasks.md deleted file mode 100644 index a7336ad..0000000 --- a/openspec/changes/ai-full-potential-toggle/tasks.md +++ /dev/null @@ -1,38 +0,0 @@ -> Decomposed for a workflow fan-out: §1 is the pure gate substrate (do first), §2 is persistence, §3 the settings→gate adapter, §4 the Hub disclosure UX (App target — compile-verify + user run-verify), §5 the cross-slice weld contract (specified, not edited into siblings), §6 verifies. Every item is MLX-free Core verified by `swift test` unless noted; this slice OWNS no UI gesture and **never builds/signs the `.app`**. The key names `fullPotentialEnabled` / `cpuLaneEnabled` / `batchedRuntimeEnabled` / `mediaGenEnabled` / `backgroundAutonomyEnabled` / `fleetCloudEscalationEnabled` are pinned by addendum §D1 and used verbatim; `enableAICommands` is CONSUMED from `tunable-settings`. - -## 1. The pure gate (Core type substrate — do first) - -- [x] 1.1 Add `AI/FullPotential/FullPotentialGate.swift`: `FullPotentialCapability` enum (`cpuLane`/`batchedRuntime`/`mediaGen`/`backgroundAutonomy`/`fleetCloud`, `String`/`CaseIterable`/`Codable`/`Sendable`) mapped 1:1 to the five sub-flags. DO NOT redefine `ComputeLane`/`MediaRuntime`/`ModelRegistry`/`WritePolicyTier` (those are sibling-owned; this enum only NAMES the capabilities the flags gate). *Verify: `swift test` — `allCases.count == 5`; each case's `rawValue` is stable.* -- [x] 1.2 Add `FullPotentialFlags` (the gate's pure input: `aiCommandsEnabled` + `fullPotentialEnabled` + the five sub-flags, `Equatable`/`Sendable`). *Verify: `swift test` — value-type equality.* -- [x] 1.3 Add `FullPotentialGate.isUnlocked(_:) -> Bool`: `master ∧ subFlag ∧ aiCommandsEnabled`; master OFF ⇒ every capability locked; `ai-commands` OFF ⇒ every capability locked; each sub-flag gates ONLY its own capability. Total, pure, no throw/async/IO. *Verify: `swift test` — full truth table: (a) `aiCommandsEnabled=false` ⇒ all five locked regardless; (b) `fullPotentialEnabled=false` ⇒ all five locked; (c) with both masters on, flipping one sub-flag unlocks exactly one capability and no other; (d) iterate `allCases` to assert exhaustive gating.* - -## 2. Persistence (Core — default OFF, legacy-load, reset preserve-set) - -- [x] 2.1 Add the six persisted `Bool` keys to `App/AppSettings.swift`: `fullPotentialEnabled` + `cpuLaneEnabled` + `batchedRuntimeEnabled` + `mediaGenEnabled` + `backgroundAutonomyEnabled` + `fleetCloudEscalationEnabled`, each **default false**. *Verify: `swift test` — fresh store reads all six false.* -- [x] 2.2 Ensure legacy-load: settings written before this wave decode with all six absent ⇒ false (mirroring `enableDeviceLink`/`enableAICommands` legacy behavior; no existing settings reset). *Verify: `swift test` — decode a fixture lacking the keys ⇒ all six false; pre-existing keys unchanged.* -- [x] 2.3 Add the six keys to the AI-opt-in **reset preserve-set** (reset-to-defaults retains them, like `enableAICommands`/`keepClipboardHistory`/selected model). *Verify: `swift test` — set the six true, reset-to-defaults, read back true (NOT zeroed); a normal tunable still resets.* - -## 3. Settings → gate adapter (Core) - -- [x] 3.1 Add a pure mapping from `AppSettings` (the six keys + the existing `enableAICommands`) to `FullPotentialFlags`, and a convenience `AppSettings.fullPotentialGate -> FullPotentialGate`. CONSUME `enableAICommands` verbatim; do not redefine the AI-commands opt-in. *Verify: `swift test` — a known settings fixture maps to the expected `FullPotentialFlags`; `enableAICommands` flows into `aiCommandsEnabled`; `gate.isUnlocked(.mediaGen)` matches the fixture.* - -## 4. The Hub Full Potential section (App target — compile-verify + user run-verify) - -- [x] 4.1 Add a **Full Potential** section to the Hub **AI** feature page (`Hub/`): the master toggle (`fullPotentialEnabled`) rendered FIRST, then the five sub-toggle rows (iterate `FullPotentialCapability.allCases`), shared Liquid Glass presentation. *Verify: `xcodebuild` compile; **user run-verifies** the section appears on the AI page in a stable-signed build.* -- [x] 4.2 Render each sub-toggle row's **persistent cost line** inline as its caption (RAM / heat / latency / $ per design Decision 1's table) — NOT a tooltip/disclosure. The `mediaGen` row states "the assistant goes quiet while it paints" (chat eviction); the `fleetCloud` row states "spends real money + sends data off-device; budget-capped + audited". *Verify: `xcodebuild` compile; **user run-verifies** every row shows its cost line inline, always visible.* -- [x] 4.3 Progressive enablement: the five sub-toggles render **disabled (visibly relocked)** while the master is off; flipping the master OFF relocks all five in the UI while RETAINING their persisted values (no zeroing). *Verify: `xcodebuild` compile; **user run-verifies** sub-toggles disabled until master on; panic-off relocks while a sub-flag's prior value survives a re-arm.* -- [x] 4.4 Any settings-store persist/load failure surfaces bounded + non-blocking (the existing AppSettings error path; clean headline, opt-in details) — **never** `NSAlert.runModal`, never raw error text in a headline. (The gate itself is total and surfaces no error.) *Verify: `xcodebuild` compile; review — no app-modal alert, no raw error interpolation.* - -## 5. The cross-slice weld contract (specified here; consumed by siblings — NOT edited into their files) - -- [x] 5.1 In this change's spec deltas, state the shared contract: each heavy slice (`ai-compute-tiers`, `ai-batched-runtime-and-context`, `ai-media-runtime` + backends, `ai-background-autonomy`, `ai-model-fleet` cloud members) consults `FullPotentialGate.isUnlocked(capability)` before activating, and behaves as feature-off when locked. DO NOT edit any sibling slice's source or spec files. *Verify: review — the deltas are **ADDED** requirements on `tunable-settings`/`configuration-hub` only (genuinely new, not rewrites of existing requirements); `git status` shows no sibling change-dir edits.* -- [x] 5.2 Confirm the consumed names match the addendum §D1 pins exactly (`fullPotentialEnabled` + the five sub-flags) and `enableAICommands` matches `tunable-settings`. *Verify: review — verbatim key names; no conflicting redefinition of a sibling-owned type.* - -## 6. Spec sync + validation - -- [x] 6.1 Write the `tunable-settings` delta (ADDED: the Full Potential keys, gate rule, default-OFF/persistence/legacy/reset, the heavy-slice-consults-the-gate contract). *Verify: `openspec validate ai-full-potential-toggle --strict`.* -- [x] 6.2 Write the `configuration-hub` delta (ADDED: the Hub AI-page Full Potential section — master + five cost-disclosing sub-toggles, disabled-until-master, panic-off relock, Liquid Glass). *Verify: `openspec validate ai-full-potential-toggle --strict`; review the deltas are true ADDED requirements (not rewrites of existing AI-page requirements).* - -## 7. User run-verify (stable-signed build — agent never builds/signs the .app) - -- [ ] 7.1 **User run-verify:** in a stable-signed build, the Full Potential section appears on the Hub AI page; the five sub-toggles render visibly disabled until the master is on; flipping the master OFF relocks all five while RETAINING their persisted values (no zeroing); each sub-toggle's RAM/heat/latency/`$` cost line is always visible (not behind a tooltip); a settings persist/load failure surfaces bounded + non-blocking with no `NSAlert.runModal`. — Verify: USER stable-signed build (`xcodebuild` compile already GREEN). diff --git a/openspec/changes/ai-local-image-generation/.openspec.yaml b/openspec/changes/ai-local-image-generation/.openspec.yaml deleted file mode 100644 index a4ac4d7..0000000 --- a/openspec/changes/ai-local-image-generation/.openspec.yaml +++ /dev/null @@ -1,2 +0,0 @@ -schema: spec-driven -created: 2026-06-23 diff --git a/openspec/changes/ai-local-image-generation/design.md b/openspec/changes/ai-local-image-generation/design.md deleted file mode 100644 index 6b8efd1..0000000 --- a/openspec/changes/ai-local-image-generation/design.md +++ /dev/null @@ -1,89 +0,0 @@ -## Context - -The generative-media seam already exists by the time this slice lands. `ai-media-runtime` owns the `MediaRuntime` protocol, the `MediaKind`/`MediaParameters`/`MediaRequest`/`MediaProgress`/`MediaAsset` value types (addendum §B1, verbatim), the `generate_image`/`generate_video` `ToolDescriptor`s, the `MediaGenSink` route-loop executor, the Files-band gallery + canvas player output, the swipe-DOWN extract compass, and the `MediaError` taxonomy + its `AIError.message(for:)` mapping. This slice does **not** touch any of that. It plugs ONE concrete backend in behind the seam. - -The fleet already exists too. `ai-model-fleet` (addendum §C1) evolved `ModelManager` into a `ModelRegistry` of `ModelDescriptor`s with residency/eviction under the 48 GB unified-memory budget; `ComputeLane` (§A1) and the `mediaDiffusion → .gpu` role→lane policy come from `ai-compute-tiers`. This slice **consumes** those: it supplies the image-role `ModelDescriptor`s, asks `ensureResident`, and classifies the resulting co-reside-vs-evict reality for the UI. The eviction *decision* is the fleet's; the *honest surfacing* of it is shared between the fleet spec and this backend's "busy painting" state. - -The hardware fact that justifies *local* image (vs the cloud-default video, §B3): **diffusion is compute-bound on M5**, the opposite of token decode. Token decode re-reads the whole weight set per token (bandwidth-bound at ~153 GB/s, which is why the GPU lane batches). A denoise step is dense matmul over a small latent against a once-resident UNet/transformer — arithmetic-bound, so it rides the M5 neural accelerators (~3.8× vs M4 for this workload class). An image is small enough to fit and fast enough to run on-device; round-tripping it to a datacenter would be the worse trade. Video stays cloud-default because LTXV-class local is 35 GB+ and minutes/clip (§B3) — a different cost regime. - -The seed contract is already the seam's: `MediaRequest.seed: Data?` is an optional PNG first frame — the screen-region or clipboard-image capture (the existing `.screenRegion`/`.clipboardImage` capture surfaces). This slice reads it for **img2img / inpaint**; absent, it is text-to-image. - -## Goals / Non-Goals - -**Goals:** -- A concrete `MediaRuntime` conformer for **local image** via mflux/FLUX-class **MLX-native in-process** diffusion (`MFluxImageRuntime`), conforming to the seam *as written* — no seam changes. -- Quant `ModelDescriptor` variants with **honest `residencyBytes`**: Q4 (~7 GB, co-resident) default, FP16 (~24 GB, evicts chat) opt-in. -- A **pure, testable** residency classifier (co-reside vs evict-chat) consumed by the cost disclosure and the "busy painting" state — the eviction *decision* stays the fleet's. -- **Seed-driven img2img / inpaint** from `MediaRequest.seed`, statically requiring a seed-capable descriptor. -- **Honest cost disclosure** (RAM / heat / latency / eviction) where the image model is picked and before a gen fires. -- The pure majority (descriptors, classifier, param/seed validation, a stub backend) is `swift test`-verified; the MLX diffusion conformer is `xcodebuild` compile-verified by an agent and run-verified by the user. - -**Non-Goals:** -- The seam, the tools, the sink, the Files/canvas output, the extract compass, the `MediaError` taxonomy (all `ai-media-runtime`). Referenced, never re-specified. -- The residency *decision* / eviction mechanics / the registry itself / the 48 GB budget math (all `ai-model-fleet`). Consumed. -- The `ComputeLane` enum + the role→lane policy (`ai-compute-tiers`). Consumed (`mediaDiffusion → .gpu`). -- The master / sub-capability toggles (`ai-full-potential-toggle`). Consumed (`mediaGenEnabled` under `fullPotentialEnabled`). -- **All video** (`ai-video-animation-generation`). A remote/cloud image provider (local-only here). -- A degraded/low-end path (M5 floor, M4 min; addendum decision 1). - -## Decisions - -### D1. The backend is a `MediaRuntime` conformer, not a new seam. -`MFluxImageRuntime` conforms to `MediaRuntime` (§B1) exactly: `capabilities: Set = [.image]`; `generate(_ request: MediaRequest) -> AsyncThrowingStream`. The denoise loop emits `.step(index:total:preview:)` once per diffusion step (the optional `preview: Data?` is a low-res latent decode for the canvas's live progress, gated so it never dominates step time), terminating in `.finished(MediaAsset)` after writing the PNG to disk (which `MediaGenSink` then turns into a Files-band entry). -- **Rationale:** the blueprint's reuse-don't-reinvent rule and §B1's explicit "owns the seam, NOT the backends." A second image seam would fork the sink and the output path. -- **Alternatives rejected:** a bespoke `ImageRuntime` protocol (forks the seam; the whole point of `MediaRuntime` is backend-swappability, exactly as `LLMRuntime` lets Gemma be swapped). Returning bytes synchronously instead of a progress stream (loses the canvas live-preview + cancellation the seam already carries). - -### D2. mflux/FLUX-class, MLX-native, **in-process** — not a subprocess/ComfyUI. -The image backend links MLX and runs the diffusion graph **in the app process**, like `GemmaMLXRuntime`. This is the §B2 mandate and the reason image is in scope now: it shares the one resident-weights / unified-memory model with chat, so residency math is *one* budget, not two processes fighting for RAM. -- **Rationale:** in-process MLX means the fleet's single 48 GB residency budget governs everything; a Q4 image model genuinely co-resides with chat in the same address space. It also reuses the metallib `*.bundle` path `build-app.sh` already ships. -- **Alternatives rejected:** a ComfyUI/MPS subprocess (that is explicitly the *video* frontier model, §B3 — 35 GB+, minutes/clip, out-of-process; wrong cost regime for image and would double-count RAM outside the fleet budget). Calling a cloud image API (local is feasible now; cloud image would be a silent-spend surprise the addendum's decision 6 forbids by default). - -### D3. Quant variants are `ModelDescriptor`s with honest `residencyBytes`; the fleet registers + selects. -This slice supplies two image descriptors (§C1 shape, verbatim): Q4 `residencyBytes ≈ 7 GB` (default, co-resident) and FP16 `residencyBytes ≈ 24 GB` (opt-in, evicts chat), each `role: .image`, `lane: .gpu`, `provider: .onDevice`, `capabilities: ["image", "img2img", "inpaint"]` as supported. `imageModelID` (persisted, addendum §1 key) selects which. The fleet's `ensureResident(_ id:)` does the actual residency/eviction. -- **Rationale:** `residencyBytes` is the eviction-budget input the fleet already consumes; supplying it honestly per quant is how "Q4 co-resides, FP16 evicts chat" becomes *math*, not a guess. `ModelDescriptor` is the §C1 contract — consumed as written. -- **Alternatives rejected:** a single fixed model (denies the user the co-reside-vs-quality trade the hardware affords). Hiding the quant choice (violates the disclosure ethos — the RAM/quality trade is the whole honest story). Inventing a backend-local descriptor type (conflicts with §C1; consumers must use `ModelDescriptor` verbatim). - -### D4. `ImageResidencyClass` — a pure classifier this slice OWNS; the eviction DECISION stays the fleet's. -A small pure value/function in Core: given a chosen image `ModelDescriptor` (its `residencyBytes`) and the fleet's current `resident()` set + the 48 GB ceiling, classify the outcome as **`.coResident`** (fits alongside the resident chat + ternary + KV) or **`.evictsChat`** (forces the chat model out). This is `swift test`-verified with fixed inputs (no Metal, no real weights). It is the input to (a) the **pre-fire cost disclosure** and (b) the runtime's honest **"busy painting"** state when FP16 is in flight. -- **Rationale:** the fleet owns *whether/how* to evict (`ensureResident`); this slice owns *telling the truth about it* at the image surface. Classification is pure math over the §C1 descriptors → unit-testable without GPU, satisfying the "majority is Core" target-split. Mirrors the batched slice's pure `ConcurrencyBudget` (residency math is testable; real probing is injected at the boundary). -- **Alternatives rejected:** letting the runtime silently decide eviction (double-owns the fleet's job and hides it). Surfacing eviction only *after* it happens (the disclosure ethos requires stating cost *in the same breath* the capability is offered — classification must run pre-fire). Re-deriving the 48 GB budget here (the fleet owns it; this consumes `resident()` + a ceiling the fleet exposes, it does not re-specify the budget). - -### D5. "Busy painting" is an honest observable state, never a pretended co-residency. -When the classifier says `.evictsChat` and an FP16 generation is in flight, the agent honestly surfaces that chat is unavailable ("busy painting") rather than appearing to answer while the chat weights are gone. This rides the existing bounded, non-blocking surface (a state, not an `NSAlert`); a generation that fails to land becomes `.failed` with a clean `MediaError` headline (never a false "Done"). Cancellation is not a failure. -- **Rationale:** addendum decision 5 ("surface 'busy painting' honestly") + blueprint invariant ("a side effect that did not land is `.failed`, never a false Done"; bounded + non-blocking). -- **Alternatives rejected:** queueing chat turns silently behind the paint (looks hung; no honest signal). Reloading chat per chat-turn mid-paint (thrashes weights, blows the budget). An `NSAlert.runModal` "busy" dialog (banned — freezes the window). - -### D6. Seed → img2img / inpaint; a seed/inpaint request statically requires a seed-capable descriptor. -When `MediaRequest.seed` is non-nil, the runtime runs image-to-image (and inpaint, where the seed PNG's alpha channel masks the region to repaint) from that PNG first frame. NB: the pinned seam (§B1) carries ONLY `MediaRequest.seed: Data?` — there is **no** mask field on `MediaRequest`/`MediaParameters`. This slice does **not** invent one; inpaint is driven by the seed PNG (alpha as mask). Any first-class mask channel, if it is ever needed, is a SEAM extension owned by `ai-media-runtime` — this backend consumes whatever the seam provides and must not assume a `MediaParameters`/`MediaRequest` field the pinned sketch lacks. The capability is advertised in the descriptor (`"img2img"`/`"inpaint"` tags) and the runtime's `capabilities`; a seed-bearing request against a non-seed-capable selection maps to `MediaError` **at this backend's boundary** — never a silent text-only fallback (mirrors the vision-required static rule for the chat runtime). -- **Rationale:** the seed contract is already the seam's first-frame field; honoring it for img2img/inpaint is the §B2 mandate. Static capability-gating matches the existing vision-required pattern (no silent degrade). -- **Alternatives rejected:** ignoring the seed and always doing text-to-image (drops a declared seam capability). A runtime-time fallback to text-only on mismatch (silent degrade — the house bans it; map to `MediaError` and surface it). - -### D7. Vendor/OS errors map to `MediaError` at THIS backend's boundary; Core stays MLX-free. -mflux/MLX failures (model load, OOM at residency, Metal/GPU faults, denoise failure), `Process`/file-IO failures (PNG write), and capability mismatches are converted into `MediaError` (owned by `ai-media-runtime`) **inside `MFluxImageRuntime`** (the layer boundary), exactly where vendor types cross into app code. Core never sees mflux/MLX types. Everything surfaces through the single `AIError.message(for:) → AIPresentedError` translator: bounded, non-blocking, clean headline, raw vendor text only in logs / opt-in copyable details. -- **Rationale:** blueprint "map at the layer boundary" + "one taxonomy, one translator." Core's `swift test` target must stay MLX-free, so the mapping lives in the native-linked conformer. -- **Alternatives rejected:** a new image-specific error enum (the addendum says reuse `MediaError`; a backend does not own a taxonomy). Raw `"\(error)"` in a headline (banned). - -### D8. The image stub makes the whole pure path test-verified without GPU. -A `StubMediaRuntime`-class image stub (Core, no weights) emits deterministic `.step(index:total:preview:)` progress then a `.finished(MediaAsset)` pointing at a tiny written/placeholder PNG, so the route → progress → asset → Files-entry → disclosure path is `swift test`-verified end to end. Vision/seed mismatch, param validation, and `ImageResidencyClass` are all exercised against the stub. -- **Rationale:** the §B1/§2 target-split mandates a stub so the seam-consuming logic is tested without real weights; mirrors `StubLLMRuntime`/`StubMediaRuntime`. -- **Alternatives rejected:** testing only against the real MLX runtime (can't — `swift test` is MLX-free; an agent never builds the `.app`). - -## Target-split & verification (per component) - -| Component | Target | Verified by | -|---|---|---| -| Image `ModelDescriptor` variant table (Q4/FP16, honest `residencyBytes`, role/lane/provider/capabilities) | **Core (MLX-free)** | `swift test` — Q4 ≈ 7 GB & FP16 ≈ 24 GB; `role == .image`, `lane == .gpu`, `provider == .onDevice`; capability tags present; `imageModelID` selects the right descriptor | -| `ImageResidencyClass` pure classifier (co-reside vs evict-chat) | **Core (MLX-free)** | `swift test` — Q4 + resident{chat,ternary,KV} → `.coResident`; FP16 → `.evictsChat`; boundary at the ceiling; classifier consumes injected `resident()` + ceiling (no real probe) | -| Seed/param/capability validation (img2img/inpaint requires seed-capable descriptor; param bounds) | **Core (MLX-free)** | `swift test` — seed-bearing request vs non-seed descriptor → mismatch error; valid img2img passes; out-of-range steps/size rejected | -| Image stub (`StubMediaRuntime`-class, deterministic progress) | **Core (MLX-free)** | `swift test` — emits ordered `.step` then `.finished(MediaAsset)` with a valid written PNG URL; cancellation ends the stream without a `.finished` | -| `MFluxImageRuntime` — mflux/FLUX-class MLX diffusion conformer (real denoise loop, seed img2img/inpaint, PNG write) | **Native-linked (`GemmaRuntime`/sibling framework)** | `xcodebuild` **compile-verify only** for an agent; **real image output, step preview, latency, and the metallib `*.bundle` path need the USER's stable-signed build** | -| Vendor→`MediaError` boundary mapping (mflux/MLX/`Process`/file-IO → `MediaError`) | **Native-linked** | `xcodebuild` compile-verify; **user** run-verify that a forced load/OOM/write failure surfaces a clean bounded card (not `NSAlert`, not raw text) | -| Real residency/eviction with real weights (Q4 co-reside; FP16 evict-chat → "busy painting") | **User stable-signed build** | **user** run-verify only — co-residency + the eviction reality + the "busy painting" state can only be observed with real weights resident under the real 48 GB budget; an agent never builds/signs the `.app` | -| Honest cost disclosure surface (RAM/heat/latency where `imageModelID` is chosen + pre-fire) | **Core values + app UI** | `xcodebuild` compile-verify the UI; `swift test` the underlying classifier/values; **user** run-verify the displayed RAM/eviction note tracks the chosen quant | - -## Risks / honest costs - -- **The MLX diffusion conformer is the only place real correctness can't be agent-verified.** No cross-step latent corruption, correct sampler/scheduler, seed reproducibility (`seedNumber`), and the metallib bundle path are all `xcodebuild`-compile-only for an agent and need the user's stable-signed build. Mitigation: keep every *schedulable/classifiable* thing (descriptors, residency class, validation, stub) pure in Core and tested; the spec scenarios make the user run-verify unambiguous. -- **FP16 evicts chat — this is a real cost, surfaced, not a bug.** ~24 GB FP16 forces the chat weights out; the companion genuinely cannot talk while it paints. Disclosed before firing and surfaced as "busy painting." Q4 (~7 GB) is the default precisely so the common case keeps chat alive. -- **Sustained GPU diffusion is heat + latency.** Even at the M5 neural-accelerator sweet spot, default-step image gen is seconds-to-tens-of-seconds of sustained GPU burn. Surfaced in the same breath the capability is offered (disclosure ethos); the gen parks via `ParkScheduler` (the seam's, `ai-media-runtime`) so it never blocks the foreground. -- **M5 floor, no degraded path.** No Intel/low-end fallback; gated behind `mediaGenEnabled` under `fullPotentialEnabled`. An unsupported host simply does not offer the capability rather than offering a degraded one. diff --git a/openspec/changes/ai-local-image-generation/proposal.md b/openspec/changes/ai-local-image-generation/proposal.md deleted file mode 100644 index 17c426c..0000000 --- a/openspec/changes/ai-local-image-generation/proposal.md +++ /dev/null @@ -1,34 +0,0 @@ -## Why - -The generative-media seam (`MediaRuntime`, addendum §B1) is owned by `ai-media-runtime` — the tools, the sink, the route-loop wiring, the Files-band gallery, the canvas player, and the `MediaError` taxonomy all live there. But a seam with no backend paints nothing. This slice ships the **concrete local image backend** behind that seam: an mflux/FLUX-class, MLX-native `MediaRuntime` conformer that actually runs diffusion in-process on the user's machine. - -Local image generation is **feasible NOW on M5** (addendum §B2, cross-cutting decision 1). M5 diffusion is **compute-bound**, not bandwidth-bound like token decode — every denoise step is dense matmul over a small latent, so it lands squarely in the neural-accelerator sweet spot (measured ~3.8× vs M4 for diffusion-class workloads). That is the inversion that makes a *local* image model the honest default here while *video* (§B3) still escalates to the cloud: an image fits and runs fast enough on-device that round-tripping it to a datacenter would be the worse choice. - -The hard, honest part is **residency**. A Q4 image model (~7 GB) can **co-reside** with the resident Gemma chat model + ternary CPU model + KV under the 48 GB budget — the companion keeps talking while it paints. An FP16 image model (~24 GB) **evicts chat** — the companion goes quiet, and the UI must say "busy painting" rather than pretend co-residency (addendum §C1, cross-cutting decision 5). Residency is **decided by the fleet** (`ai-model-fleet`, §C1) and **consumed here**: this slice supplies the image `ModelDescriptor`s (with honest `residencyBytes` per quant) and a pure residency-classification helper, asks the registry to `ensureResident`, and surfaces the resulting co-reside-vs-evict reality through the existing bounded, non-blocking error/state surface. - -## What Changes - -- **A concrete local image `MediaRuntime`** — `MFluxImageRuntime` (working name), an mflux/FLUX-class **MLX-native, in-process** diffusion backend. It conforms to the `MediaRuntime` protocol *as written* in addendum §B1 (it does not redefine the seam); `capabilities = [.image]`; `generate(_:)` returns the `AsyncThrowingStream` of `.step(index:total:preview:)` denoise progress ending in `.finished(MediaAsset)`. Native-linked → **`xcodebuild` compile-verify only** for an agent; real correctness needs the user's stable-signed build. -- **Quant variants as `ModelDescriptor`s** — this slice supplies the image-role descriptors the fleet registers: a default **Q4 (~7 GB, co-resident)** and an opt-in **FP16 (~24 GB, evicts chat)**, each with honest `residencyBytes`, `role: .image`, `lane: .gpu`, `provider: .onDevice`, `capabilities: ["image"]` (+ `"img2img"`/`"inpaint"`). Selected by the persisted `imageModelID`. -- **A pure residency-classification helper** (`ImageResidencyClass`, Core, `swift test`-verified): given an image descriptor's `residencyBytes` + the fleet's current resident set, classify **co-resident** vs **evicts-chat**, so the UI can disclose the cost *before* the user fires and the runtime can honestly report "busy painting" while FP16 paints. The eviction *decision* stays the fleet's (`ensureResident`); this slice only classifies + consumes. -- **img2img / inpaint from the seed** — the `MediaRequest.seed: Data?` (the screen-region or clipboard-image capture, already the seam's first-frame contract) drives image-to-image and inpaint when present; a vision/seed-bearing request **statically requires** a seed-capable image descriptor (the runtime declares the capability; mismatch maps to `MediaError` at the boundary, never a silent text-only fallback). -- **Honest cost disclosure** — RAM (Q4 ~7 GB co-resident vs FP16 ~24 GB evict-chat), heat/compute (M5 neural-accelerator sweet spot, still a sustained GPU burn), and latency (seconds-to-tens-of-seconds per image at default steps) are surfaced where the user picks `imageModelID` and before a generation fires, in the same breath the capability is offered (addendum §D1 disclosure ethos). The whole backend is gated by `mediaGenEnabled` under the `fullPotentialEnabled` master (consumed, not redefined). -- **No new seam, no new gesture, no new error taxonomy.** The seam (`MediaRuntime`/`MediaRequest`/`MediaProgress`/`MediaAsset`), the tools (`generate_image`), the sink (`MediaGenSink`), the Files-band output, the canvas player, the swipe-DOWN extract compass, and the `MediaError` taxonomy are all `ai-media-runtime`'s — referenced, never re-specified. Vendor errors (mflux/MLX, `Process`, file IO) map **into `MediaError` at this backend's boundary**. - -## Capabilities - -### New Capabilities - -- *(none)* — this slice ADDS backend-only requirements to the **`ai-generative-media`** capability owned by `ai-media-runtime`. It does not introduce a new capability spec. - -### Modified Capabilities - -- `ai-generative-media`: ADDED backend requirements — a concrete local image `MediaRuntime` conformer, its quant `ModelDescriptor` variants, the pure residency-classification (co-reside vs evict-chat), seed-driven img2img/inpaint, and honest RAM/heat/latency disclosure + the "busy painting" eviction surface. These are deltas *against the seam requirements `ai-media-runtime` defines* — they do not redefine the seam. - -## Impact - -- **Code (native-linked, `GemmaRuntime`/sibling framework):** `MFluxImageRuntime` (the mflux/FLUX-class MLX diffusion conformer) + the vendor→`MediaError` boundary mapping. Verified `xcodebuild` compile-only by an agent; real image output, the metallib `*.bundle` path, and the residency reality need the user's stable-signed build. -- **Code (MLX-free Core, `swift test`-verified):** the image `ModelDescriptor` variant table (Q4/FP16 + honest `residencyBytes`), the `ImageResidencyClass` pure classifier, the seed-capability/parameter-validation logic, and a **`StubMediaRuntime`-class image stub** (deterministic fake `.step`/`.finished` progress, no weights) so the full route → progress → asset → disclosure path is test-verified without GPU. -- **Reuse, not rebuild:** the `MediaRuntime` seam + `MediaGenSink` + Files-band gallery + canvas player + `MediaError` translator (`ai-media-runtime`); `ModelRegistry`/`ModelDescriptor`/`ensureResident` + the 48 GB residency budget (`ai-model-fleet`); `ComputeLane.gpu` + the `mediaDiffusion` role→lane policy (`ai-compute-tiers`); `ModelProvisioner`/`runtimeFactory` residency lifecycle (`ModelManager`); the screen-region / clipboard-image capture as the seed source (existing); the `AIError.message(for:)` translator + bounded non-blocking surface; `fullPotentialEnabled`/`mediaGenEnabled` gating (`ai-full-potential-toggle`). -- **Honest cost (M5 floor, no degraded path):** Q4 co-resides (~7 GB) and chat stays alive; FP16 (~24 GB) **evicts chat** — disclosed up front, surfaced as an honest "busy painting" state, never a pretended co-residency. Sustained GPU diffusion is a real heat/latency cost, surfaced before firing. No Intel/low-end fallback; an image gen is gated behind `mediaGenEnabled` under the master toggle. -- **Out of scope:** the seam/tools/sink/output/taxonomy (`ai-media-runtime`); the residency *decision*/eviction mechanics + the registry itself (`ai-model-fleet`); the lane policy (`ai-compute-tiers`); the master/sub toggles (`ai-full-potential-toggle`); **all video** (`ai-video-animation-generation`); a remote/cloud image provider (local-only here). diff --git a/openspec/changes/ai-local-image-generation/specs/ai-generative-media/spec.md b/openspec/changes/ai-local-image-generation/specs/ai-generative-media/spec.md deleted file mode 100644 index 68b24cc..0000000 --- a/openspec/changes/ai-local-image-generation/specs/ai-generative-media/spec.md +++ /dev/null @@ -1,106 +0,0 @@ -## ADDED Requirements - -> These are BACKEND-only deltas to the `ai-generative-media` capability. The seam itself — the `MediaRuntime` protocol, the `MediaKind`/`MediaParameters`/`MediaRequest`/`MediaProgress`/`MediaAsset` value types, the `generate_image` tool, the `MediaGenSink` executor, the Files-band gallery + canvas player output, the swipe-DOWN extract compass, and the `MediaError` taxonomy + its `AIError.message(for:)` mapping — is defined by `ai-media-runtime` and is REFERENCED here, never redefined. Residency/eviction and the `ModelRegistry`/`ModelDescriptor`/`ensureResident` mechanics are defined by `ai-model-fleet` and CONSUMED here. - -### Requirement: Local image MediaRuntime backend (mflux/FLUX-class, MLX-native, in-process) - -The system SHALL provide a concrete `MediaRuntime` conformer for **local image generation** via an mflux/FLUX-class, MLX-native diffusion model running **in-process**. It SHALL conform to the `MediaRuntime` seam exactly as defined by `ai-media-runtime` (it SHALL NOT redefine the seam): `capabilities` SHALL contain `.image`, and `generate(_:)` SHALL return an `AsyncThrowingStream` that emits ordered `.step(index:total:preview:)` diffusion-progress values terminating in `.finished(MediaAsset)`. The model SHALL run on Apple-Silicon M5 (M4 minimum) with NO degraded/low-end path. The backend SHALL be gated by `mediaGenEnabled` under the `fullPotentialEnabled` master toggle. - -#### Scenario: text-to-image generation streams progress then finishes - -- **WHEN** a `MediaRequest` with `kind == .image` and no `seed` is generated against the local image backend -- **THEN** the returned stream emits `.step(index:total:preview:)` values in ascending `index` order up to `total`, and terminates with a single `.finished(MediaAsset)` whose `kind == .image` and whose `width`/`height` match the requested `MediaParameters.size` -- **AND** the `MediaAsset.url` points at a written PNG that `ai-media-runtime`'s sink turns into a Files-band entry - -#### Scenario: capabilities advertise image only - -- **WHEN** the backend's `capabilities` are read -- **THEN** the set contains `.image` and does NOT contain `.video` (video is `ai-video-animation-generation`'s backend) - -#### Scenario: gated behind the master toggle - -- **WHEN** `mediaGenEnabled` is false (or `fullPotentialEnabled` is false) -- **THEN** the local image capability is NOT offered and no image descriptor is selectable -- **WHEN** both flags are true -- **THEN** the capability is offered and a `generate_image` request routes the diffusion work to the `.gpu` compute lane (the `mediaDiffusion` role→lane policy owned by `ai-compute-tiers`) - -### Requirement: Quant model descriptors with honest residency bytes - -The system SHALL supply image-role `ModelDescriptor`s (the `ai-model-fleet` §C1 type, used verbatim) for the local image backend: a default **Q4** variant with `residencyBytes` of approximately 7 GB, and an opt-in **FP16** variant with `residencyBytes` of approximately 24 GB. Each SHALL have `role: .image`, `lane: .gpu`, `provider: .onDevice`, and `capabilities` tags reflecting what it supports (`"image"`, plus `"img2img"`/`"inpaint"` where supported). The persisted `imageModelID` SHALL select which variant is used; the default SHALL be the Q4 variant. The fleet's `ModelRegistry` SHALL register these descriptors and own the actual residency/eviction via `ensureResident(_:)`. - -#### Scenario: Q4 default and FP16 opt-in descriptors - -- **WHEN** the image descriptors are enumerated -- **THEN** a Q4 descriptor with `residencyBytes ≈ 7 GB`, `role == .image`, `lane == .gpu`, `provider == .onDevice` is present -- **AND** an FP16 descriptor with `residencyBytes ≈ 24 GB` and the same role/lane/provider is present -- **AND** with no `imageModelID` set the Q4 descriptor is the selected one - -#### Scenario: imageModelID selects the variant - -- **WHEN** `imageModelID` is set to the FP16 descriptor's id -- **THEN** the FP16 descriptor is the selected image backend -- **WHEN** `imageModelID` is set to an id that matches no image descriptor -- **THEN** selection is rejected (it is NOT silently coerced to a default) - -### Requirement: Residency classification — co-resident vs evicts-chat - -The system SHALL provide a pure, testable residency classification (`ImageResidencyClass`) that, given the chosen image descriptor's `residencyBytes`, the fleet's current resident model set, and the unified-memory ceiling (consumed from `ai-model-fleet`, not re-derived), classifies the outcome of making that image model resident as either **co-resident** (it fits alongside the resident chat + ternary models + KV) or **evicts-chat** (it forces the chat model out). The classification SHALL be free of real memory probing (the resident set and ceiling are injected), so it is unit-testable with fixed inputs. The classification — NOT a fresh eviction decision — SHALL be the single input to both the pre-fire cost disclosure and the "busy painting" runtime state. The actual eviction DECISION remains `ai-model-fleet`'s `ensureResident(_:)`. - -#### Scenario: Q4 co-resides with chat - -- **WHEN** the Q4 image descriptor (~7 GB) is classified against a resident set of {chat, ternary} + KV under the 48 GB ceiling -- **THEN** the classification is `.coResident` - -#### Scenario: FP16 evicts chat - -- **WHEN** the FP16 image descriptor (~24 GB) is classified against that same resident set under the 48 GB ceiling -- **THEN** the classification is `.evictsChat` - -#### Scenario: classification is pure (no real probe) - -- **WHEN** the classifier runs in a Core unit test with an injected resident set and ceiling -- **THEN** it produces a deterministic result with no Metal / real-weight / live-memory dependency - -### Requirement: Seed-driven img2img and inpaint - -When a `MediaRequest` carries a non-nil `seed` (the optional PNG first frame the seam defines — a screen-region or clipboard-image capture), the local image backend SHALL run image-to-image generation from that seed (and inpaint, where the seed PNG's alpha channel masks the region to repaint). The backend SHALL use ONLY the seam fields defined by `ai-media-runtime` (`MediaRequest.seed: Data?`) — it SHALL NOT introduce a mask field on `MediaRequest`/`MediaParameters`; a first-class mask channel, if ever required, is a seam extension owned by `ai-media-runtime`, consumed here. A seed-bearing or inpaint request SHALL statically require a seed-capable image descriptor; a mismatch SHALL be converted to a `MediaError` at the backend boundary and surfaced as a bounded, non-blocking failure — it SHALL NOT silently fall back to text-to-image. The `MediaParameters.seedNumber` SHALL drive reproducible RNG. - -#### Scenario: img2img from a seed - -- **WHEN** a `MediaRequest` with `kind == .image` and a non-nil `seed` is generated against a seed-capable descriptor -- **THEN** the backend generates from that PNG first frame (image-to-image) and finishes with a `MediaAsset` - -#### Scenario: seed against a non-seed-capable descriptor is an error, not a degrade - -- **WHEN** a seed-bearing request targets an image descriptor whose capabilities do not include `"img2img"` -- **THEN** the request is converted to a `MediaError` at the backend boundary and surfaced as a bounded, non-blocking failure -- **AND** the backend does NOT silently drop the seed and run text-to-image - -#### Scenario: seedNumber is reproducible - -- **WHEN** two generations use the same prompt, parameters, and `MediaParameters.seedNumber` -- **THEN** they are reproducible (the seam's reproducibility contract holds; verified in the user's stable-signed build) - -### Requirement: Honest cost disclosure and busy-painting state - -The local image backend SHALL disclose its cost — RAM (Q4 ~7 GB co-resident vs FP16 ~24 GB evicts-chat, derived from `ImageResidencyClass`), heat/compute (sustained M5 GPU diffusion), and latency (seconds to tens of seconds at default steps) — at the point the image model is chosen and before a generation fires, in the same breath the capability is offered. When `ImageResidencyClass` is `.evictsChat` and an FP16 generation is in flight, the system SHALL honestly surface that chat is unavailable ("busy painting") as a bounded, non-blocking observable state — never via `NSAlert.runModal`, never with raw error text in a headline. A generation that fails SHALL become a `.failed` state carrying a clean `MediaError` headline (mapped through `AIError.message(for:)`), never a false "Done"; cancellation SHALL NOT be treated as a failure. - -#### Scenario: cost is disclosed before firing - -- **WHEN** the user selects an image model and is about to fire a generation -- **THEN** the RAM cost (with the co-resident-vs-evicts-chat consequence), a heat/compute note, and a latency note are presented before the generation starts - -#### Scenario: FP16 in flight surfaces busy-painting honestly - -- **WHEN** an FP16 image generation is in flight and its classification is `.evictsChat` -- **THEN** the agent surfaces that chat is unavailable ("busy painting") as a bounded, non-blocking state -- **AND** it does NOT appear to answer chat as though the chat model were still resident -- **AND** the surface is NOT an `NSAlert.runModal` - -#### Scenario: a failed generation is observable, never a false done - -- **WHEN** a generation fails (model load, OOM, denoise, or PNG write) -- **THEN** the vendor/OS error is mapped to a `MediaError` at the backend boundary and the state becomes `.failed` with a clean headline routed through `AIError.message(for:)` -- **AND** raw vendor text appears only in logs or opt-in copyable details, never in the headline -- **WHEN** the user cancels a generation -- **THEN** the result is a cancellation, NOT a `.failed` state diff --git a/openspec/changes/ai-local-image-generation/tasks.md b/openspec/changes/ai-local-image-generation/tasks.md deleted file mode 100644 index 5d31772..0000000 --- a/openspec/changes/ai-local-image-generation/tasks.md +++ /dev/null @@ -1,39 +0,0 @@ -> Backend-only slice for the `ai-media-runtime` seam. §1–§3 are the pure Core substrate (do first; `swift test`-verified). §4 is the native-linked MLX conformer (`xcodebuild` compile-verify only for an agent). §5 is disclosure/state wiring. §6 verifies. This slice CONSUMES `MediaRuntime`/`MediaRequest`/`MediaProgress`/`MediaAsset`/`MediaError` (`ai-media-runtime`), `ModelDescriptor`/`ModelRegistry`/`ensureResident` (`ai-model-fleet`), `ComputeLane.gpu` + `mediaDiffusion` policy (`ai-compute-tiers`), and `mediaGenEnabled`/`fullPotentialEnabled` (`ai-full-potential-toggle`) — it does not define them. - -## 1. Image model descriptors (pure Core) - -- [x] 1.1 Define the image `ModelDescriptor` variant table using the §C1 type **verbatim**: a Q4 default (`residencyBytes ≈ 7 GB`) and an FP16 opt-in (`residencyBytes ≈ 24 GB`), each `role: .image`, `lane: .gpu`, `provider: .onDevice`, `capabilities: ["image", "img2img", "inpaint"]` as supported, stable `id`s. *(Verify: `swift test` asserts the two descriptors' bytes/role/lane/provider/capabilities.)* -- [x] 1.2 Wire `imageModelID` (persisted, addendum §1 key) selection → the chosen descriptor; default = the Q4 id. *(Verify: `swift test` — default resolves to Q4; setting the FP16 id resolves to FP16; an unknown id is rejected, not silently defaulted.)* -- [x] 1.3 Confirm these descriptors are handed to the fleet's `ModelRegistry` for registration (consume `ai-model-fleet`; do not build a registry). *(Verify: `swift test` against a stub/in-memory registry that the image descriptors appear in `descriptors()`.)* - -## 2. Residency classification (pure Core — this slice OWNS the classifier, fleet OWNS the decision) - -- [x] 2.1 Define `ImageResidencyClass` (`.coResident` / `.evictsChat`) as a pure function over a chosen image descriptor's `residencyBytes` + the fleet's injected `resident()` set + the 48 GB ceiling (consumed from the fleet, not re-derived). *(Verify: `swift test` — Q4 alongside resident{chat,ternary,KV} → `.coResident`; FP16 → `.evictsChat`; exact-ceiling boundary is deterministic.)* -- [x] 2.2 Keep real free-RAM probing OUT of the pure classifier — inject the resident set + ceiling so the math is unit-testable with fixed inputs (mirror `ConcurrencyBudget`'s injected-probe pattern). *(Verify: `swift test` runs with no Metal/real weights.)* -- [x] 2.3 The classifier is the single input to both the pre-fire disclosure (§5.1) and the "busy painting" state (§5.2). *(Verify: `swift test` — the same classifier value drives both a disclosure value and the runtime's pre-flight state decision.)* - -## 3. Request validation + image stub (pure Core) - -- [x] 3.1 Validate `MediaRequest` for the image backend: `kind == .image`; param bounds (steps/size/guidance) within the descriptor's supported ranges; a non-nil `seed` (img2img) or an inpaint request **statically requires** a seed-capable descriptor — mismatch yields a `MediaError` value, never a silent text-only fallback. *(Verify: `swift test` — valid t2i passes; valid img2img with seed-capable descriptor passes; seed against non-seed descriptor → mismatch error; out-of-range params rejected.)* -- [x] 3.2 Add the image stub (a `StubMediaRuntime`-class conformer, Core, no weights): emits ordered `.step(index:total:preview:)` then `.finished(MediaAsset)` with a valid written/placeholder PNG URL; honors cancellation (stream ends without `.finished`). *(Verify: `swift test` — ordered steps, terminal `.finished` with a readable PNG URL, cancellation path emits no `.finished`.)* -- [x] 3.3 Exercise the full route → progress → asset → (sink/Files-entry shape) path against the stub, including the seed/img2img branch. *(Verify: `swift test` — end-to-end over the stub produces a `MediaAsset` whose `kind == .image` and dimensions match the request.)* - -## 4. mflux/FLUX-class MLX image runtime (native-linked — `xcodebuild` compile-verify only) - -- [x] 4.1 Implement `MFluxImageRuntime: MediaRuntime` in the `GemmaRuntime` target / sibling framework: `capabilities = [.image]`; an in-process MLX diffusion `generate(_:)` denoise loop emitting `.step` (optional low-res latent `preview`) and a terminal `.finished(MediaAsset)` after writing the PNG. *(Verify: `xcodebuild` compiles; **user** stable-signed build run-verifies real image output + step preview.)* -- [x] 4.2 Seed-driven img2img / inpaint: when `MediaRequest.seed` is present, run image-to-image (and inpaint, using the seed PNG's alpha channel as the mask) from that PNG first frame; honor `seedNumber` for reproducibility. Do NOT add a mask field to the pinned `MediaRequest`/`MediaParameters` — the seam carries only `seed: Data?`; a first-class mask channel, if ever needed, is `ai-media-runtime`'s seam extension to own. *(Verify: `xcodebuild` compiles; **user** run-verifies img2img from a captured region + that a fixed `seedNumber` reproduces an image.)* -- [x] 4.3 Map vendor/OS errors → `MediaError` at this conformer's boundary (mflux/MLX load/OOM/Metal faults, denoise failure, `Process`/PNG-write IO, capability mismatch). Core stays MLX-free. *(Verify: `xcodebuild` compiles; **user** run-verifies a forced load/OOM/write failure surfaces a clean bounded `MediaError` card, never raw text / `NSAlert`.)* -- [x] 4.4 Plug the runtime in via the existing `ModelProvisioner`/`runtimeFactory` residency lifecycle for the `.image` role (reuse `ModelManager`/fleet residency; do not add a second resident-weights path). *(Verify: `xcodebuild` compiles; **user** run-verifies Q4 loads and co-resides; confirm the metallib `*.bundle` is present in `Contents/Resources/` — no GPU use without it.)* - -## 5. Honest cost disclosure + "busy painting" state (Core values + app UI) - -- [x] 5.1 Pre-fire cost disclosure where `imageModelID` is chosen and before a generation fires: RAM (Q4 ~7 GB co-resident vs FP16 ~24 GB evict-chat, from `ImageResidencyClass`), a heat/compute note (sustained M5 GPU diffusion), and a latency note (seconds-to-tens-of-seconds at default steps) — stated in the same breath the capability is offered (disclosure ethos). *(Verify: `swift test` the underlying classifier/values; `xcodebuild` compile the UI; **user** run-verifies the displayed RAM/eviction note tracks the chosen quant.)* -- [x] 5.2 "Busy painting" honest state: when `ImageResidencyClass == .evictsChat` and an FP16 gen is in flight, surface that chat is unavailable (a bounded, non-blocking observable state — never an `NSAlert`); a gen that fails to land becomes `.failed` with a clean `MediaError` headline (never a false "Done"); cancellation is not a failure. *(Verify: `swift test` the state-transition decision over the classifier + stub; **user** run-verifies the real eviction → "busy painting" with real FP16 weights.)* -- [x] 5.3 Gate the whole backend behind `mediaGenEnabled` under `fullPotentialEnabled` (consume `ai-full-potential-toggle`; do not redefine the toggles). *(Verify: `swift test` — with the flag off the image capability is not offered; with it on it is; the gen role→lane consults `mediaDiffusion → .gpu`.)* - -## 6. Validation - -- [x] 6.1 `swift build` + `swift test` green for all Core components (§1–§3, §5.1–§5.3 pure parts). *(Verify: `swift test` passes; no MLX symbols in Core.)* -- [x] 6.2 `xcodebuild` compile-verify the native-linked conformer + boundary mapping (§4). *(Verify: `xcodebuild` build of the app/`GemmaRuntime` target succeeds, compile-only; an agent does NOT build/sign/install the `.app`.)* -- [ ] 6.3 Hand the user the run-verify checklist: real Q4 co-residency, FP16 evict-chat → "busy painting," img2img from a captured seed, `seedNumber` reproducibility, a forced-failure bounded card, and the metallib bundle presence. *(Verify: documented in `design.md` Risks; user confirms in their stable-signed build.)* -- [x] 6.4 `openspec validate ai-local-image-generation --strict` passes. *(Verify: command exits 0.)* diff --git a/openspec/changes/ai-media-runtime/.openspec.yaml b/openspec/changes/ai-media-runtime/.openspec.yaml deleted file mode 100644 index a4ac4d7..0000000 --- a/openspec/changes/ai-media-runtime/.openspec.yaml +++ /dev/null @@ -1,2 +0,0 @@ -schema: spec-driven -created: 2026-06-23 diff --git a/openspec/changes/ai-media-runtime/design.md b/openspec/changes/ai-media-runtime/design.md deleted file mode 100644 index 1fda32d..0000000 --- a/openspec/changes/ai-media-runtime/design.md +++ /dev/null @@ -1,211 +0,0 @@ -## Context - -The V2 agent's whole side-effect machinery is a **token-stream** model: `LLMRuntime` emits `Token`s, the -route loop turns a turn into a `ToolRoute`, and `TaskDispatching`/`TaskSinks` apply a *quick* side effect -(write an event, append a note, send a payload). Generative media does not fit that shape. A diffusion -run is **minutes**, emits **diffusion-step progress** (not language tokens), saturates the GPU (it -**evicts chat** under the 48 GB budget — the fleet's call), and its product is a **file** the user keeps. -Forcing it through `LLMRuntime` would lie about its cost and its output. - -The addendum (§B1) therefore pins a SECOND runtime seam, `MediaRuntime`, parallel to `LLMRuntime`. This -slice owns that seam plus its value types, the two tools that invoke it, the sink the route loop runs, -the seed (img2img/img2video) path, and the dual output (Files-band gallery asset + canvas -preview/player). The concrete diffusion backends are deliberately OUT of scope — they conform to the -seam in their own slices (`ai-local-image-generation`, `ai-video-animation-generation`), exactly as Gemma -conforms to `LLMRuntime` and could be swapped. The capability is gated under `mediaGenEnabled` -(itself under `fullPotentialEnabled`); default OFF, V2 ships calm. - -Every blueprint/addendum invariant binds: one error taxonomy + the single `AIError.message(for:)` -translator mapped at the boundary, surfaced bounded + non-blocking (never `NSAlert`, never raw error in a -headline); a side effect that did not land is `.failed`, never a false "Saved."; non-activating overlays -with synchronous `orderOut`; the canonical two-finger compass (DOWN=affirm at canvas-top / RIGHT=discard); -reuse-don't-reinvent (`ToolRegistry`, route loop, `WritePolicyTier`, `ParkScheduler`, files band, the -`.screenRegion`/`.clipboardImage` captures, `BubbleMorph`, `DockPreviewOverlay`). - -## Goals - -- Define the `MediaRuntime` seam + media value types VERBATIM from addendum §B1, in MLX-free Core, with a - `StubMediaRuntime` that makes the whole slice `swift test`-verifiable without weights. -- Register `generate_image` / `generate_video` as `ToolDescriptor`s in the `ToolRegistry` with their - write-policy tiers (image `.confirm`; cloud video `.dangerous` + budget cap), and execute a routed - media call via `MediaGenSink` through the EXISTING route → execute → continue loop — no new control flow. -- Resolve the **seed** (img2img/img2video) from the existing screen-region / clipboard captures, as the - first frame; surface a missing-but-required seed as a clean `.failed`. -- Deliver the output twice: a **Files-band gallery asset** (reusing the band) and a **canvas - preview/player** resolved by the compass (DOWN extracts save/paste/set-as; RIGHT discards). -- Park-while-generating via `ParkScheduler` (thinking badge → notch glow on done / needs-you), feeding - the parked machinery the media job's observable state — without re-implementing it. -- Be HONEST about cost everywhere: a heavy gen evicts chat ("busy painting"); cloud video spends money - (budget cap + dangerous tier); latency is minutes; failures are bounded `.failed` cards. - -## Non-Goals - -- **The concrete model backends.** No mflux/FLUX image weights and no LTX Studio / local-LTXV video here. - Those are `ai-local-image-generation` and `ai-video-animation-generation`, conforming to this seam. -- **Lane/residency/eviction math.** `ai-compute-tiers` (lanes) and `ai-model-fleet` (registry/eviction) - own it; this slice consumes `ModelRegistry`/`ModelDescriptor` to pick a runtime and to learn that a - heavy gen evicts chat. -- **The master-toggle UX page** (`ai-full-potential-toggle`). This slice only reads `mediaGenEnabled`. -- **A new gesture grammar, a new overlay species, or a new error translator.** All reused. -- **Round-tripping a cloud video result into a structured agent message** beyond the finished asset — - cloud video is fire-and-forget-with-progress (the handoff template); the asset is the result. - -## Decisions - -### D1. `MediaRuntime` is a SECOND seam parallel to `LLMRuntime`, not an `LLMRuntime` method -A generative job is a long async **`AsyncThrowingStream`** ending in a `MediaAsset` -file — categorically unlike `LLMRuntime`'s `AsyncThrowingStream`. Modeling it as the -addendum-pinned standalone protocol keeps each seam honest about its own shape and lets a media backend -be swapped without touching `LLMRuntime` conformers (and vice-versa). -- **Rationale:** the seam carries diffusion-step progress + intermediate previews + a terminal file — - none of which a token stream expresses. Parallel seams keep the text and media worlds independently - swappable (the blueprint's reuse-don't-reinvent ethos: one seam per shape). -- **Alternatives rejected:** *(a) add a `generateMedia` method to `LLMRuntime`* — pollutes the text seam - with a file-producing, minutes-long job and forces every text conformer (`StubLLMRuntime`, - `DevAIRuntime`, the batched runtime) to carry a media method it can't serve. *(b) Model media as just - another `TaskKind`/`TaskSink`* — a `TaskSink` is a quick synchronous-ish side effect with a `TaskReview`; - it has no notion of streamed step-progress, an intermediate preview, eviction of chat, or a minutes-long - parked job. The media work is a tool (route+approval) WHOSE EXECUTOR is this new seam, not a `TaskSink`. - -### D2. Media is a TOOL; the route loop runs `MediaGenSink` — no new control flow -`generate_image`/`generate_video` are `ToolDescriptor`s contributed to the `ToolRegistry` via the -existing `ToolContributor` seam (addendum §B1, blueprint §3.3). When the router selects one, the existing -route → execute → continue loop dispatches to `MediaGenSink` exactly as it dispatches a `TaskKind` to the -task machinery. The model "chose the menu item"; the sink does the long job. -- **Rationale:** the tool-routing slice already owns the loop, the registry, candidate retrieval, the - bounded step cap, and the per-step write-policy gate. Reusing them means media inherits approval, - auditing, no-progress guarding, and the compass-driven approval (DOWN=approve / RIGHT=skip) for free. -- **Alternatives rejected:** *a bespoke "media run" control path* — duplicates the loop's approval/audit/ - cap machinery and forks the agent's single execution model. The addendum is explicit: "Media generation - is a tool … `MediaGenSink` executes via the existing route→execute→continue loop. No new control flow." - -### D3. Write-policy: image `.confirm`, cloud video `.dangerous` + budget cap (the handoff template) -The `generate_image` descriptor ships `WritePolicyTier.confirm`; the `generate_video` descriptor ships -`.dangerous` when its provider is cloud, plus a per-day budget/rate cap (`mediaVideoBudgetPerDay`, -mirroring `ClaudeHandoffConfig`). The sink's approval/spend gate sits BEFORE any compute or network spend. -- **Rationale:** image gen is local but expensive (heat, eviction) → confirm-by-default. Cloud video - spends real money and leaves the device → dangerous, escalates to foreground via needs-you even when - parked, and is rate-capped, exactly like Claude handoff (addendum §B3/§3.8). Budget is enforced before - the call so an exhausted budget is a clean `.failed`/`.declined`, never a silent spend. -- **Alternatives rejected:** *(a) image `.auto`* — a minutes-long GPU-saturating job that evicts chat is - not something to fire without a beat of confirmation. *(b) cloud video `.confirm`* — understates a - money-spending off-device call; the addendum pins it `.dangerous` + budget-capped. *(c) a separate - budget mechanism* — reuse the handoff per-day cap pattern + the audit log, don't invent a parallel one. - -### D4. Effective tier comes from the injected `WritePolicyResolving` seam (background-autonomy owns it) -The sink/descriptor declare the **shipped** tier (D3); the EFFECTIVE tier per step is resolved by the -`WritePolicyResolving` seam the route loop already injects (default = descriptor tier so this slice stands -alone; `ai-background-autonomy` supplies the real resolver + whitelist + audit). A `.dangerous` cloud-video -tier is NEVER lowered; a `.confirm` image tier may be lowered to `.auto` only by an explicit user -whitelist (background-autonomy's call, not this slice's). -- **Rationale:** one resolution path for all tools; media doesn't fork the security boundary. Every media - step writes one `AuditRecord` (auto/confirmed/declined/escalated/failed) through the shared `AuditLog`. -- **Alternatives rejected:** *media-local tier resolution* — duplicates and could diverge from the - whitelist/audit boundary that background-autonomy owns. *Never lowering image* — denies the user the - documented whitelist affordance; we defer the lowering decision to the resolver, not hard-code it. - -### D5. The seed (img2img / img2video) reuses the existing `.screenRegion` / `.clipboardImage` captures -`MediaRequest.seed: Data?` (PNG) is the first frame. The sink resolves it from the EXISTING capture -seams — the interactive screen-region picker (`.screenRegion`) and the live clipboard image -(`.clipboardImage`, on-demand, normalized to PNG) — the same inputs the vision path already uses. A tool -authored as img2img with no resolvable seed is a clean `.failed(MediaError.seedRequired)`, never a -fabricated blank frame. -- **Rationale:** the project already has a tested region picker and clipboard-image normalization; the - blueprint mandates reuse. The seed *is* one of those captures promoted to a generation input. -- **Alternatives rejected:** *a new media-only image picker* — reinvents the region picker. *Auto-firing - on a clipboard image* — violates the on-demand rule (copying an image must never auto-fire a gen). *A - blank fallback frame* — would fabricate a result; a missing required seed is observably `.failed`. - -### D6. Output #1 — a Files-band gallery asset (reuse the band, don't build a browser) -The finished `MediaAsset.url` is written under a dedicated **generated-media gallery** root and surfaces -as an ordinary `.fileEntry` in the Files band — inheriting its on-demand listing, lift-to-deliver / -Open / Open-With, contextual delivery, and non-destructive scope. The gallery root is local-only and -recoverable (it sits under the band's local-only, trash-not-delete rules). -- **Rationale:** the addendum is explicit ("Generated assets land as Files-band entries (the gallery). - Reuse, do not build a new browser"). The band already lists, previews, opens, and delivers files. -- **Alternatives rejected:** *a standalone media gallery UI* — duplicates the band. *A non-file in-memory - gallery* — loses durability; the asset must survive relaunch and discard (D7). *Writing into an - arbitrary user folder* — a dedicated gallery root keeps the output bounded and recoverable. - -### D7. Output #2 — a canvas preview/player resolved by the compass; the asset is already durable -While generating, the canvas shows live `MediaProgress.step` + the intermediate preview; on -`.finished` it shows the image, or a **player** for video (a `DockPreviewOverlay`-pattern non-activating -panel, synchronous `orderOut`). Resolution is the canonical compass: **DOWN (at canvas-top) extracts** — -**save / paste / set-as** — **RIGHT discards**. Because the asset is ALREADY in the gallery (D6) before -the canvas resolves, a discard never loses the result; it only dismisses the preview. -- **Rationale:** the compass is canonical (DOWN=affirm-at-top, RIGHT=discard); the addendum pins - "swipe-DOWN extracts (save/paste/set-as)". Persisting to the gallery first makes discard non-destructive - and matches the parked lifecycle (the result outlives the canvas). -- **Alternatives rejected:** *a continuous screen-recording / live SCStream player* — the switcher/dock - work already rejected per-frame pumps; a finished clip is played from its file, an in-progress preview - is the runtime's last `.step` preview frame. *Lift-to-commit* — the canvas grammar is swipe-to-resolve, - not lift-to-commit (that's the files band's grammar; do not cross them). *Discard deletes the file* — - the gallery is the durable record; discard is a UI dismiss, the file stays (the user can still find it). - -### D8. Parked-while-generating: feed `ParkScheduler`, don't re-implement it -A media job is slow, so the session **parks** (canvas overscroll-park → notch home zone). This slice -FEEDS the existing parked machinery the job's observable state: a **thinking** badge while painting, a -**done** badge (+ unseen count) + notch **glow** on `.finished`, and a **needs-you** escalation + glow for -a `.dangerous` cloud-video step (via `ParkScheduler.escalate`). The scheduler decides scheduling; this -slice only reports `didAdvance` / `escalate`. -- **Rationale:** `ai-parked-sessions` owns the scheduler, rail, badges, and glow; the addendum says media - "parks via `ParkScheduler`; the notch glows on completion / on needs-you." Reuse, don't reinvent. -- **Alternatives rejected:** *a media-only background queue + progress HUD* — forks the parked lifecycle - and the notch surface. *Blocking the canvas during a gen* — denies the user other work for minutes; the - whole point of parking is to let the GPU paint while the user moves on. - -### D9. Honest residency — a heavy gen evicts chat ("busy painting"), surfaced not hidden -The sink consults the `ModelRegistry` (`ensureResident` for the image/video runtime). When residency math -(owned by `ai-model-fleet`) decides the gen must EVICT chat, the canvas/rail surface a calm "the assistant -is busy painting" state rather than pretending co-residency; chat resumes when the gen finishes/parks. -- **Rationale:** addendum decision 5 ("A heavy gen EVICTS chat … state it honestly … surface 'the - assistant is busy painting' rather than pretend co-residency"). The project's surface-the-cost ethos. -- **Alternatives rejected:** *silently queueing chat behind the gen with no signal* — looks like a hang. - *Refusing the gen to keep chat resident* — denies the capability the hardware can serve; the honest - answer is to evict + tell the user. - -### D10. `MediaError` is the one new taxonomy, mapped at the boundary, one translator -A single `enum MediaError: Error, Equatable` (`LocalizedError`) carries the media-specific cases the -shared `RuntimeError`/`TaskError` cannot: `noCapableBackend(kind)`, `seedRequired`, `seedInvalid`, -`generationFailed(headline)`, `outputWriteFailed`, `cloudBudgetExhausted`, `cloudUnavailable`. Vendor/OS -errors (mflux/LTXV/ComfyUI, `Process`, `NSURLError`, `FileManager`) map into it at the layer boundary -(the sink / backend conformer). Every surface routes through `AIError.message(for:)` → `AIPresentedError` -and is bounded + non-blocking. **Cancellation is NOT a failure** — a discarded/parked-then-discarded gen -ends `.cancelled`, never a `.failed` badge. -- **Rationale:** the blueprint's one-taxonomy / one-translator / map-at-boundary / bounded-non-blocking - law. A distinct cancellation outcome avoids a false failure badge (mirrors the parked-discard rule). -- **Alternatives rejected:** *raw vendor errors into headlines* — banned. *Reusing `TaskError` for - everything* — it cannot carry "no capable backend" / "seed required" / "budget exhausted" cleanly; one - small `MediaError` is the addendum-sanctioned single new enum. - -### D11. Unavailable tools are omitted from candidates (the router never routes to dead ends) -The `MediaToolContributor` advertises `generate_image` only if a `MediaRuntime` advertises `.image`, and -`generate_video` only if a video provider is configured AND `fleetCloudEscalationEnabled` (for cloud) AND -budget remains. Under `mediaGenEnabled == false` (or `fullPotentialEnabled == false`), the contributor -contributes nothing. So the router's candidate set never includes a tool that cannot run. -- **Rationale:** routing to an unavailable tool wastes a loop step and produces a confusing decline. The - capability is gated; gating at the contributor keeps the loop unaware of media's availability rules. -- **Alternatives rejected:** *always advertise, fail at execute* — burns a step + emits a `.failed` the - user reads as a bug; better to never offer what can't run. - -## Target-split & verification (per component) - -| Component | Target | Verification | -|---|---|---| -| `MediaKind` / `MediaSize` / `MediaParameters` / `MediaRequest` / `MediaProgress` / `MediaAsset` value types (addendum §B1) | MLX-free Core | `swift test` — Codable round-trip, `MediaAsset` ↔ `.fileEntry` mapping, seed-present/absent shapes | -| `MediaRuntime` protocol (the seam, §B1) | MLX-free Core | `swift build` (protocol compiles); driven in tests via the stub | -| `StubMediaRuntime` (scripted progress: success, intermediate previews, mid-flight fail, cancel, missing-seed, budget-out) | MLX-free Core (test support) | `swift test` — drives the sink deterministically without weights | -| `MediaToolContributor` (the two `ToolDescriptor`s + `argsSchema` + `WritePolicyTier` + availability gating) | MLX-free Core | `swift test` — descriptors present/omitted by capability/availability; tiers correct (image `.confirm`, cloud video `.dangerous`); args schema validates | -| `MediaGenSink` (route-loop executor: resolve seed → pick runtime → drive `generate` → write asset → `ToolStepResult`) | MLX-free Core | `swift test` — `.done` carries gallery path; `.failed` clean headline on each error; `.awaitingApproval` before compute; cancellation ≠ failure; missing-seed `.failed`; budget gate before spend | -| Seed resolution (reuse `.screenRegion` / `.clipboardImage` capture seams) | MLX-free Core (orchestration) | `swift test` — seed wired into `MediaRequest`; missing-required-seed → `.failed(seedRequired)` | -| Gallery writer + `.fileEntry` surfacing (Files-band entry) | MLX-free Core | `swift test` — asset written under the gallery root; surfaces as `.fileEntry`; local-only/recoverable scope honored | -| Parked-while-generating feed (thinking/done/needs-you → `ParkScheduler` `didAdvance`/`escalate`) | MLX-free Core | `swift test` — progress → thinking; finished → done + unseen count; dangerous cloud video → `escalate`; cancellation → no failed badge | -| `WritePolicyResolving` consumption (effective tier; never lower `.dangerous`) | MLX-free Core | `swift test` — image `.confirm` may lower to `.auto` only on whitelist; cloud video `.dangerous` never lowered | -| `MediaError` taxonomy + boundary mapping + `AIError.message(for:)` routing | MLX-free Core | `swift test` — each case → a clean `AIPresentedError.headline`; no raw text in headline; cancellation distinct from failure | -| Canvas media preview/player overlay (`DockPreviewOverlay` species, non-activating, synchronous `orderOut`; DOWN extract / RIGHT discard) | Native-linked (app target) | `xcodebuild` COMPILE-VERIFY only; real non-activating/synchronous-teardown/compass behavior needs the user's stable-signed build | -| `ModelRegistry` residency consumption ("busy painting" eviction surface) | MLX-free Core (orchestration) | `swift test` against a stub registry — eviction → "busy painting" state; chat resumes after finish/park | - -Agents NEVER build/sign/install the `.app` (ad-hoc signing breaks TCC; the `*.bundle` metallib copy in -`build-app.sh` must not regress). The concrete diffusion backends — and any real GPU/cloud latency, heat, -RAM, eviction, and spend — are verified only in their own slices and only by the user's stable-signed -build; this slice's Core seam/tools/sink/output are fully `swift test`-verified against the stub. diff --git a/openspec/changes/ai-media-runtime/proposal.md b/openspec/changes/ai-media-runtime/proposal.md deleted file mode 100644 index bb9a338..0000000 --- a/openspec/changes/ai-media-runtime/proposal.md +++ /dev/null @@ -1,133 +0,0 @@ -## Why - -The V2 agent can route to **side-effecting tasks** (calendar/reminder/save/send) through the structured -route → execute → continue loop, and it can *read* a screen-region or clipboard image as **vision input**. -What it cannot yet do is **make** an image or a clip. That is a fundamentally different shape of work: -not a token stream that ends in a sentence, but a **long async job with step-progress that ends in a -file**. A diffusion run is minutes-not-seconds, GPU-saturating (it **evicts chat** under the 48 GB -budget — the fleet's decision, surfaced honestly here), and its result is a *thing on disk* the user -will want to keep, paste, or set somewhere — exactly the lifecycle the Files band already owns. - -So generative media needs its **own runtime seam**, parallel to `LLMRuntime`, not bolted onto it. This -slice is `ai-media-runtime` (the new capability `ai-generative-media`). It owns that seam (`MediaRuntime` -+ the media value types from addendum §B1), the two **tools** that let the model invoke it -(`generate_image` / `generate_video` as `ToolDescriptor`s with their write-policy tiers), the -side-effecting **executor** (`MediaGenSink`) that the existing route loop runs, the **seed** path that -turns a captured screen-region / clipboard image into the first frame for img2img / img2video, and the -**output**: a Files-band gallery asset PLUS a canvas preview/player resolved by the canonical compass. -It does NOT own the concrete model backends — local image (mflux/FLUX) is `ai-local-image-generation` -and video (cloud-default / local-LTXV-frontier) is `ai-video-animation-generation`. Like `LLMRuntime` -lets Gemma be swapped, `MediaRuntime` is the swappable seam those backends drop into. - -Honesty is a first-class requirement (the project's surface-the-cost ethos): media is **slow → it -parks** (via `ParkScheduler`, the notch glows on completion / needs-you); image generation is **`.confirm`** -and cloud video is **`.dangerous` + per-day budget-capped** (the Claude-handoff gating template); a heavy -gen says "the assistant is busy painting" rather than pretending co-residency; every failure is a clean, -bounded `.failed` card, never a false "Saved." and never an `NSAlert`. - -## What Changes - -- **A second runtime seam (`MediaRuntime`, addendum §B1, Core).** A protocol parallel to `LLMRuntime`: - `capabilities: Set` and `generate(_ request: MediaRequest) -> AsyncThrowingStream`. - The progress stream emits `.step(index:total:preview:)` (streamed diffusion progress + an optional - intermediate preview frame) and terminates in `.finished(MediaAsset)`. Defined VERBATIM from §B1; a - `StubMediaRuntime` makes the whole slice `swift test`-verifiable without weights. The concrete backends - conform to this seam in their own slices. -- **The media value types (addendum §B1, Core), owned here:** `MediaKind` (`.image`/`.video`), - `MediaParameters` (size/steps/seed-number/guidance/durationMs), `MediaRequest` (prompt + optional - **seed image** `Data?` for img2img/img2video + kind + parameters), `MediaProgress` - (`.step`/`.finished`), `MediaAsset` (id/url/kind/width/height/durationMs — `Codable`, becomes a - Files-band entry). A `MediaSize` width×height value type. -- **`generate_image` + `generate_video` as `ToolDescriptor`s registered in the `ToolRegistry`** (the - tool-routing seam, blueprint §3.3 / addendum §B1). Each advertises its name + summary + an - `argsSchema` (prompt, size, steps, optional seed-image-handle, video duration) and its - `WritePolicyTier`: **image `.confirm`**, **cloud video `.dangerous`** (+ budget cap). Contributed via the - existing `ToolContributor` seam — no route-loop change. A descriptor whose backing `MediaRuntime` does - not advertise the kind (or, for video, has no provider/budget left) is **omitted from candidates**, so - the router never routes to an unavailable tool. -- **`MediaGenSink` — the side-effecting executor run by the route → execute → continue loop.** When the - router selects `generate_image`/`generate_video`, the loop dispatches to `MediaGenSink`, which: resolves - the seed (below), picks the `MediaRuntime` for the kind, drives `generate(_:)`, threads - `MediaProgress` into the live UI, writes the finished `MediaAsset` to the gallery folder, and returns a - `ToolStepResult` (`.done` carrying the asset's gallery path, or `.failed` with a clean headline). It - obeys write-policy: `.confirm`/`.dangerous` steps surface a `TaskReview`-backed **awaiting-approval** - step (DOWN = approve / RIGHT = skip) BEFORE any compute or spend. -- **The seed (img2img / img2video) path.** A `MediaRequest.seed: Data?` (PNG). The capture sources are - the EXISTING `.screenRegion` (interactive region picker) and `.clipboardImage` (live pasteboard, - normalized to PNG) inputs — reused, not reinvented. The sink wires the captured/clipboard image as the - request's seed/first frame; a missing-but-required seed (a tool authored as img2img) is a clean - `.failed`, not a fabricated blank frame. -- **Output #1 — a Files-band gallery asset.** The finished `MediaAsset.url` is written under a dedicated - **generated-media gallery** root and surfaces as an ordinary `.fileEntry` in the Files band (reuse the - band's on-demand listing + lift-to-open + Open-With + contextual-delivery — no new browser). The - gallery root is local-only and recoverable, honoring the band's non-destructive scope. -- **Output #2 — a canvas preview/player resolved by the compass.** While generating, the canvas shows - live step-progress + the intermediate preview; on finish it shows the image / a **player** for video - (the `DockPreviewOverlay` non-activating, synchronous-`orderOut` overlay species). A two-finger - **DOWN** extract (only at canvas top) resolves it — **save / paste / set-as** — per the canonical - compass; **RIGHT** discards; the result is already durably in the gallery so discard never loses it. -- **Parked-while-generating.** Because a gen is slow, the session **parks** via the existing - `ParkScheduler`: the canvas overscroll-park stashes it to the notch home zone, the card shows a - **thinking** badge while it paints, the notch **glows** on completion (a done badge) or on `needsYou` - (a `.dangerous` cloud-video escalation). The scheduler/rail/glow are reused from `ai-parked-sessions`; - this slice only feeds them the media job's observable state. -- **`MediaError` taxonomy, mapped at the boundary.** One new `enum MediaError: Error, Equatable` - (`LocalizedError`) for media-specific failures the shared `RuntimeError`/`TaskError` cannot carry - (no capable backend, seed required/invalid, generation failed/cancelled-distinct, output-write failed, - cloud budget exhausted). Vendor/OS errors (mflux/LTXV/ComfyUI, `Process`, `NSURLError`, `FileManager`) - map into it at the layer boundary; everything routes through the single `AIError.message(for:)` - translator and surfaces **bounded + non-blocking** — never `NSAlert`, never raw error in a headline. - -## Capabilities - -### New Capabilities - -- `ai-generative-media`: the on-device (image) / escalated (video) generative-media capability — the - `MediaRuntime` seam parallel to `LLMRuntime`, the `generate_image`/`generate_video` tools and their - write-policy tiers, the `MediaGenSink` executor driven by the route loop, the seed (img2img/img2video) - path fed by the screen-region / clipboard capture, the output as a Files-band gallery asset plus a - compass-resolved canvas preview/player, the parked-while-generating lifecycle, and the `MediaError` - taxonomy. This capability OWNS the seam, tools, sink, and output; the concrete backends are separate - capabilities (`ai-local-image-generation`, `ai-video-animation-generation`). - -### Modified Capabilities - -- `ai-command-tasks`: the `ToolRegistry` now additionally aggregates **media** tools - (`generate_image`/`generate_video`) through the SAME `ToolContributor` seam, and the route → - execute → continue loop additionally dispatches a routed media call to `MediaGenSink` (a long async - job ending in a file, not a token stream), reusing the existing per-step write-policy gate - (DOWN=approve / RIGHT=skip) and the `.done`/`.failed`/`.declined` mapping. No new control flow — media - is just another contributor + sink. (Delta authored against `ai-command-tasks`.) - -## Impact - -- **Code (MLX-free Core, `AI/Media/`):** new `MediaRuntime` protocol; `MediaKind`, `MediaSize`, - `MediaParameters`, `MediaRequest`, `MediaProgress`, `MediaAsset` value types (addendum §B1 verbatim); - `StubMediaRuntime` (scripted progress, for `swift test`); the `MediaToolContributor` exposing the two - `ToolDescriptor`s with their `WritePolicyTier`; `MediaGenSink` (the route-loop executor — pure - orchestration over an injected `MediaRuntime`, a seed provider, and a gallery writer); the seed - resolution (reusing the existing `.screenRegion`/`.clipboardImage` capture seams); the gallery-asset - writer + the `.fileEntry` surfacing; the canvas preview-state model; the parked-session feed; the - `MediaError` enum + its boundary mapping. All verified by `swift build` + `swift test` against - `StubMediaRuntime` with scripted progress (success, intermediate previews, mid-flight failure, - cancellation, missing seed, budget-exhausted). -- **Native-linked (`xcodebuild` COMPILE-VERIFY ONLY for an agent; real correctness needs the user's - stable-signed build):** the **canvas media player overlay** (the non-activating `DockPreviewOverlay`- - pattern panel rendering the image / playing the clip). The concrete diffusion backends are NOT in this - slice. An agent never builds/signs/installs the `.app` (ad-hoc signing breaks TCC; the `*.bundle` - metallib copy in `build-app.sh` must not regress). -- **Reuse, not rebuild:** the `ToolRegistry`/`ToolContributor`/route loop + `WritePolicyTier` - (`ai-tool-routing`); `TaskReview`/`PreparedAction` for the approval step; `WritePolicyResolving` + - `AuditLog` (`ai-background-autonomy`); `ParkScheduler`/notch rail/glow (`ai-parked-sessions`); the - files band's listing + open + Open-With + delivery (`files-band` / `files-contextual-delivery`); the - `.screenRegion`/`.clipboardImage` capture seams (`screen-region-picker`); `BubbleMorph`; the - `DockPreviewOverlay` overlay species; `AIError.message(for:)`/`AIPresentedError`. -- **Consumes from siblings (addendum):** `ModelRegistry`/`ModelDescriptor` (`ai-model-fleet`) to pick the - resident image/video runtime and to learn that **a heavy gen evicts chat**; `fullPotentialEnabled` + - `mediaGenEnabled` (`ai-full-potential-toggle`) gate the tools' registration and the capability's - activation. This slice depends on the **addendum's pinned contracts**, not on sibling change files. -- **Out of scope (non-goals):** the concrete image backend (mflux/FLUX — `ai-local-image-generation`) - and the concrete video backend (cloud LTX Studio default / local LTXV frontier — - `ai-video-animation-generation`); the lane/residency/eviction math itself (`ai-compute-tiers`, - `ai-model-fleet` — consumed, not owned); the master toggle UX page (`ai-full-potential-toggle`). This - slice defines only the seam, tools, sink, and output those plug into. diff --git a/openspec/changes/ai-media-runtime/specs/ai-command-tasks/spec.md b/openspec/changes/ai-media-runtime/specs/ai-command-tasks/spec.md deleted file mode 100644 index 3cfc2e1..0000000 --- a/openspec/changes/ai-media-runtime/specs/ai-command-tasks/spec.md +++ /dev/null @@ -1,33 +0,0 @@ -## ADDED Requirements - -### Requirement: Media generation is a routed tool executed by a media sink, not a new control flow -The tool registry SHALL additionally aggregate **media-generation** tools (`generate_image` and -`generate_video`) through the **same tool-contributor mechanism** as every other tool, and the existing -route → execute → continue loop SHALL dispatch a routed media call to a **media-generation sink** — a long -asynchronous job that streams step progress and ends in a written file — using the **same** loop, the -**same** per-step write-policy gate (an awaiting-approval step resolved by **DOWN = approve / RIGHT = -skip**), the **same** bounded step cap, and the **same** done/declined/failed result mapping that every -other routed tool uses. Media generation SHALL NOT introduce a separate control path, a separate approval -grammar, or a separate result mapping. A media tool that cannot run (no capable backend; for cloud video no -configured provider or no remaining budget; or media generation disabled) SHALL be **omitted from the route -candidates**, so the router never routes to an unavailable generator. A media step that did not land SHALL -be reported **failed**, and a **cancelled** generation SHALL be a distinct outcome, not a failure. - -#### Scenario: A routed media call runs through the existing loop and approval gate -- **WHEN** the model routes a turn to `generate_image` or `generate_video` -- **THEN** the existing loop dispatches it to the media-generation sink, a confirm/dangerous step pauses as - the same awaiting-approval step (DOWN = approve / RIGHT = skip) before any compute or spend, and its - result is fed back as a tool turn — with no separate control flow - -#### Scenario: An unavailable media tool is not offered to the router -- **WHEN** no runtime can produce the requested media kind, or the cloud-video provider/budget is - unavailable, or media generation is disabled -- **THEN** the media tool is omitted from the route candidates and the router cannot route to it - -#### Scenario: A media side effect that did not land is reported failed, not done -- **WHEN** a routed media generation fails or its asset cannot be written -- **THEN** the step is reported failed with a clean headline, never reported done - -#### Scenario: A cancelled media generation is distinct from a failure -- **WHEN** a routed media generation is cancelled (discarded) -- **THEN** the step ends as cancelled, fed back as such, and is not reported as a failure diff --git a/openspec/changes/ai-media-runtime/specs/ai-generative-media/spec.md b/openspec/changes/ai-media-runtime/specs/ai-generative-media/spec.md deleted file mode 100644 index 4ac184a..0000000 --- a/openspec/changes/ai-media-runtime/specs/ai-generative-media/spec.md +++ /dev/null @@ -1,254 +0,0 @@ -## ADDED Requirements - -### Requirement: Generative media uses a second runtime seam parallel to the text runtime -The system SHALL generate media (images and video) through a **dedicated media runtime seam** that is -**parallel to**, not part of, the text (`LLMRuntime`) seam. The media seam SHALL model a generation as a -**long asynchronous job with step progress that ends in a file** — a progress stream that emits -diffusion-step progress (with an optional intermediate preview frame) and terminates in a finished media -asset — rather than a language-token stream. A media generation request SHALL carry a prompt, a kind -(image or video), generation parameters (size, steps, optional RNG seed number, optional guidance, -video-only duration), and an **optional seed image** for image-to-image / image-to-video. The seam SHALL -declare which kinds it can produce, so the system never requests a kind a runtime cannot serve. The -concrete generation backend SHALL be swappable behind this seam without changing any feature code (exactly -as the text runtime is swappable), so this capability owns the seam, not the weights. - -#### Scenario: A generation is a progress-bearing async job ending in a file -- **WHEN** a media generation runs -- **THEN** it streams diffusion-step progress (optionally with an intermediate preview frame) and ends by - producing a finished media asset that references a written file, not a stream of language tokens - -#### Scenario: The seam advertises its kinds -- **WHEN** the system considers a media generation of a given kind -- **THEN** it checks the runtime's advertised capabilities and does not request a kind the runtime cannot - produce - -#### Scenario: A different backend drops in without feature changes -- **WHEN** a different generation backend conforms to the media seam -- **THEN** the tools, executor, and output paths use it unchanged, just as a different text model would be - swapped behind the text seam - -### Requirement: generate_image and generate_video are routed tools with write-policy tiers -The system SHALL expose media generation to the agent as **tools** — `generate_image` and -`generate_video` — registered in the tool registry through the same contributor mechanism as every other -tool, each advertising its name, a one-line summary, an arguments schema (prompt, size, steps, an optional -seed-image handle, and a video duration), and a **write-policy tier**. Image generation SHALL ship the -**confirm** tier; cloud video generation SHALL ship the **dangerous** tier and SHALL additionally be -**budget-capped** (a per-day cap mirroring the Claude-handoff cost gate). The model SHALL invoke media -generation only by routing to one of these tools, never by a separate control path. A media tool whose -backing runtime cannot serve the kind — or, for video, has no configured provider or no remaining budget — -SHALL be **omitted from the route candidates**, so the router never routes to an unavailable generator. - -#### Scenario: Image generation is a confirm-tier tool -- **WHEN** the `generate_image` tool is registered -- **THEN** it carries the confirm write-policy tier and is offered to the router like any other tool - -#### Scenario: Cloud video is a dangerous, budget-capped tool -- **WHEN** the `generate_video` tool backed by a cloud provider is registered -- **THEN** it carries the dangerous write-policy tier and a per-day budget cap, mirroring the Claude-handoff - cost gate - -#### Scenario: An unavailable generator is not a route candidate -- **WHEN** no runtime can produce the requested kind, or the video provider/budget is unavailable, or media - generation is disabled -- **THEN** the corresponding media tool is omitted from the route candidates and the model cannot route to it - -### Requirement: A media generation is executed by a sink driven by the route loop -The system SHALL execute a routed media generation with a **media generation sink** invoked by the existing -route → execute → continue loop (the same loop that runs every other tool), not by a bespoke control flow. -Before any compute or spend, a media step whose **effective** write-policy tier requires confirmation -SHALL pause as an **awaiting-approval** step that surfaces the action review and resolves by the canonical -two-finger compass — **DOWN = approve / RIGHT = skip**. The effective tier SHALL come from the shared -write-policy resolver (the same seam every tool uses): a **dangerous** tier (cloud video) SHALL **never** -be lowered. On approval the sink SHALL resolve the seed, select the runtime for the kind, drive the -generation, write the finished asset, and return a tool-step result — **done** carrying the asset's gallery -location, **skipped/declined** with no side effect, or **failed** with a clean headline. A side effect that -did not land (a generation that failed, or an asset that could not be written) SHALL be reported **failed**, -never a false success. **Cancellation** (a discarded generation) SHALL be a distinct outcome, **not** a -failure. - -#### Scenario: A side-effecting media step waits for a DOWN approval before compute -- **WHEN** the model routes to a media tool whose effective tier requires confirmation -- **THEN** the step pauses showing the review, and no compute or spend happens until the user approves with - a DOWN swipe (a RIGHT skip applies nothing and feeds back so the model may continue) - -#### Scenario: Cloud video's dangerous tier is never lowered -- **WHEN** the write-policy resolver evaluates a cloud `generate_video` step -- **THEN** its effective tier stays dangerous and is never lowered to auto by any whitelist - -#### Scenario: A finished generation returns done with the asset location -- **WHEN** an approved generation finishes -- **THEN** the sink returns a done result carrying the asset's gallery location, fed back into the loop - -#### Scenario: A failed generation is reported failed, never a false success -- **WHEN** a generation fails or its asset cannot be written -- **THEN** the step is reported failed with a clean headline and no false "saved" is reported - -#### Scenario: A cancelled generation is not a failure -- **WHEN** the user discards an in-flight generation -- **THEN** the step ends as cancelled, not failed, and leaves no failed indicator - -### Requirement: The cloud-video budget is enforced before any spend -The system SHALL enforce the cloud-video **per-day budget/rate cap** **before** invoking the cloud -provider, so an exhausted budget produces a clean declined/failed outcome with **no network call and no -spend**. The cap SHALL reuse the Claude-handoff cost-gate pattern (confirm-by-default, per-day cap, -audited) rather than a parallel mechanism. Exceeding the cap SHALL be an observable, bounded outcome, never -a silent spend and never a silent refusal. - -#### Scenario: An exhausted budget refuses before spending -- **WHEN** the cloud-video budget for the day is exhausted and the model routes to `generate_video` -- **THEN** the step resolves as budget-exhausted before any network call, with no spend, surfaced as a - bounded, clean outcome - -#### Scenario: The cap is audited like the handoff cost gate -- **WHEN** a cloud-video step runs or is refused on budget -- **THEN** it is recorded in the shared audit log with its effective tier and a redacted argument summary, - like the Claude-handoff cost gate - -### Requirement: The seed (image-to-image / image-to-video) reuses the existing screen-region and clipboard captures -The system SHALL source a generation's **seed image** (its first frame for image-to-image / image-to-video) -from the **existing** capture inputs — the interactive screen-region picker and the on-demand live -clipboard image (normalized to PNG) — and SHALL NOT introduce a separate media-only picker. Copying an -image SHALL NOT auto-fire a generation (the clipboard read stays on-demand). A media tool authored to -require a seed that has **no resolvable seed** SHALL resolve as **failed (seed required)**, and an -**undecodable** seed SHALL resolve as **failed (seed invalid)** — never a fabricated blank first frame. - -#### Scenario: A captured region becomes the first frame -- **WHEN** the user supplies a screen-region or clipboard image as the seed for an image-to-image / image-to-video generation -- **THEN** that image is wired as the request's seed/first frame using the existing capture inputs, with no - new picker - -#### Scenario: Copying an image never auto-fires a generation -- **WHEN** an image is on the clipboard -- **THEN** no generation fires automatically; the clipboard image is read only on demand when used as a seed - -#### Scenario: A missing or invalid required seed fails cleanly -- **WHEN** a generation requires a seed but none is resolvable, or the seed cannot be decoded -- **THEN** the step resolves as failed (seed required / seed invalid) with a clean headline and no compute, - never a fabricated blank frame - -### Requirement: A finished asset lands as a Files-band gallery entry -The system SHALL write each finished media asset to a dedicated **generated-media gallery** location and -SHALL surface it as an ordinary **Files-band entry** (the gallery), reusing the Files band's on-demand -listing, open / Open-With, and contextual delivery rather than building a new browser. The gallery location -SHALL be **local-only** and the asset SHALL be **durable** (it survives relaunch and survives discarding the -canvas preview). The asset's entry SHALL carry a **path-stable identity** so re-listing does not strobe the -selection. The gallery SHALL honor the Files band's non-destructive scope (no permanent delete, no -overwrite). - -#### Scenario: A generated asset appears in the Files-band gallery -- **WHEN** a generation finishes -- **THEN** its file is written under the generated-media gallery and appears as a Files-band entry that can - be opened, opened-with, and delivered - -#### Scenario: The gallery asset is durable and local-only -- **WHEN** the app relaunches, or the canvas preview is discarded -- **THEN** the generated asset remains in the gallery (it is not lost) and the gallery reads only local - files - -#### Scenario: Re-listing the gallery does not strobe -- **WHEN** the gallery is re-listed -- **THEN** each generated asset keeps a stable identity by path and the highlight does not flicker - -### Requirement: The canvas preview/player is resolved by the canonical compass -While a generation runs the system SHALL show a **live canvas preview** (step progress plus the -intermediate preview frame); on completion it SHALL show the finished image, or a **player** for video, -presented as a **non-activating** overlay (the Dock-preview overlay species) that never becomes the app's -key/main window and that tears down **synchronously**. The preview SHALL be resolved by the canonical -two-finger compass: **DOWN — only when the canvas is at its top — extracts** the result (save / paste / -set-as); **RIGHT discards** the preview. Because the asset is already durably in the gallery, a discard -SHALL only dismiss the preview and SHALL NOT lose the file. A sub-threshold two-finger scroll SHALL NOT -resolve the preview (the resolve excursion sits above incidental scroll). The preview SHALL NOT continuously -screen-record; an in-progress preview is the runtime's last step frame and a finished video plays from its -file. - -#### Scenario: Generating shows live progress -- **WHEN** a generation is running -- **THEN** the canvas shows step progress and the latest intermediate preview frame - -#### Scenario: DOWN at the top extracts the result -- **WHEN** the finished preview is shown, the canvas is at its top, and the user swipes two-finger down past - the resolve threshold -- **THEN** the result is extracted (save / paste / set-as) - -#### Scenario: RIGHT discards but keeps the file -- **WHEN** the user swipes two-finger right past the resolve threshold on a finished preview -- **THEN** the preview dismisses and the asset remains in the gallery (the file is not lost) - -#### Scenario: The preview panel never steals focus and tears down synchronously -- **WHEN** the preview/player is shown and later dismissed -- **THEN** it never becomes the app's key/main window, the foreground app keeps focus, and the panel is - ordered out synchronously - -#### Scenario: Sub-threshold scroll does not resolve -- **WHEN** the user makes a small two-finger scroll below the resolve threshold over the preview -- **THEN** the preview is not resolved (reading it never extracts or discards it) - -### Requirement: Generation is slow, so the session parks while it paints -Because a media generation is slow, the system SHALL let the session **park** (via the existing parked- -session machinery and notch home zone) while it generates, and SHALL feed the parked machinery the job's -observable state rather than re-implementing it. A parked session that is generating SHALL show the -**thinking** badge; on completion it SHALL show the **done** badge with the unseen-result count and the -notch SHALL **glow**; a **dangerous** cloud-video step on a parked session SHALL **escalate to needs-you** -(via the scheduler) with a clean one-line reason and the ambient notch glow. A **cancellation** SHALL NOT -leave a failed badge. The scheduler SHALL decide scheduling; this capability SHALL only report advance and -escalation. - -#### Scenario: A generating parked session shows the thinking badge -- **WHEN** a session is parked while its generation runs in the background -- **THEN** its rail card shows the thinking badge - -#### Scenario: A finished generation glows the notch with an unseen-result count -- **WHEN** a parked session's generation finishes -- **THEN** its card shows a done badge with the unseen-result count and the notch home zone glows - -#### Scenario: A parked dangerous cloud-video step escalates to needs-you -- **WHEN** a parked session reaches a dangerous cloud-video step -- **THEN** it escalates to needs-you with a clean one-line reason and the ambient notch glow, rather than - spending in the background - -#### Scenario: A cancelled parked generation leaves no failed badge -- **WHEN** a parked session's generation is discarded -- **THEN** it ends as cancelled and leaves no failed badge - -### Requirement: A heavy generation evicts chat and says so honestly -The system SHALL be honest that a heavy generation **evicts chat** under the device memory budget: when the -model fleet's residency decision requires evicting the chat model to run the generation, the system SHALL -surface a calm "**the assistant is busy painting**" state rather than pretending the chat and the -generator co-reside, and chat SHALL resume when the generation finishes or parks. The system SHALL NOT -silently queue chat behind the generation with no signal (which would read as a hang) and SHALL NOT refuse a -generation merely to keep chat resident. - -#### Scenario: A generation that must evict chat surfaces a busy-painting state -- **WHEN** running a generation requires evicting the chat model under the memory budget -- **THEN** the system shows a calm "busy painting" state and chat resumes when the generation finishes or - parks - -#### Scenario: Chat is never silently starved -- **WHEN** a generation is occupying the GPU -- **THEN** the chat unavailability is surfaced (busy painting), never left as an unexplained hang - -### Requirement: Media failures use one taxonomy, mapped at the boundary, surfaced bounded and non-blocking -All media-generation failures SHALL be classified into the shared error taxonomy — reusing the existing -runtime/task errors plus at most one media-specific error type for the cases they cannot carry (no capable -backend, seed required/invalid, generation failed, output-write failed, cloud budget exhausted, cloud -unavailable) — mapped into that taxonomy **at the layer boundary** (where vendor/OS errors from the -generation backend, process spawn, network, or filesystem cross into app code). Every media failure SHALL -be surfaced through the single error translator as a **clean headline with opt-in copyable details**, as a -**bounded, non-blocking** observable failed state — **never** an app-modal alert, **never** raw error text -in a headline. **Cancellation SHALL be distinct from failure.** Every terminal media outcome (done, -declined, escalated, failed, cancelled) SHALL be observable; a media step SHALL NEVER end silently. - -#### Scenario: A backend error maps to a clean headline -- **WHEN** the generation backend, a process spawn, the network, or the filesystem throws -- **THEN** the error is mapped at the boundary into the media error taxonomy and surfaced as a clean - headline with opt-in details, bounded and non-blocking — never an app-modal alert, never raw error text - in the headline - -#### Scenario: Cancellation is not surfaced as a failure -- **WHEN** a generation is cancelled -- **THEN** the outcome is cancelled, distinct from a failure, and no failed message is shown - -#### Scenario: No media step ends silently -- **WHEN** a media step terminates for any reason -- **THEN** it produces an observable outcome (done / declined / escalated / failed / cancelled), never - silence diff --git a/openspec/changes/ai-media-runtime/tasks.md b/openspec/changes/ai-media-runtime/tasks.md deleted file mode 100644 index 755e3e6..0000000 --- a/openspec/changes/ai-media-runtime/tasks.md +++ /dev/null @@ -1,120 +0,0 @@ -> The seam + value types (§1) are the substrate. The tools + sink (§2–§3) are the route-loop integration. -> The seed (§4) and output (§5–§6) are the I/O. §7 is the parked feed, §8 the errors, §9 verifies. -> All of §1–§8 are MLX-free Core verified by `swift test` against `StubMediaRuntime`; only the canvas -> player overlay is native-linked (xcodebuild compile-verify; real behavior needs the user's build). - -## 1. The `MediaRuntime` seam + media value types (Core, addendum §B1 verbatim) - -- [x] 1.1 Add `MediaKind` (`.image`/`.video`), `MediaSize` (width×height), `MediaParameters` - (size/steps/seedNumber/guidance/durationMs), `MediaRequest` (prompt + optional `seed: Data?` PNG + - kind + parameters), `MediaProgress` (`.step(index:total:preview:)` / `.finished(MediaAsset)`), - `MediaAsset` (id/url/kind/width/height/durationMs, `Codable`) under `AI/Media/`, VERBATIM from - addendum §B1. _Verify: `swift test` — Codable round-trip; seed-present/absent; durationMs video-only._ -- [x] 1.2 Add the `MediaRuntime` protocol (`capabilities: Set`, - `generate(_:) -> AsyncThrowingStream`) — pinned §B1; no extra methods. - _Verify: `swift build` compiles the protocol; exercised via the stub in §1.3._ -- [x] 1.3 Add `StubMediaRuntime` (test support): scripted progress sequences — success-with-final-asset, - success-with-intermediate-previews, mid-flight failure, cancellation, and a per-kind capability set. - _Verify: `swift test` — each script drives a deterministic `AsyncThrowingStream`._ - -## 2. The two tools in the `ToolRegistry` (Core; reuse `ToolContributor`) - -- [x] 2.1 Add `MediaToolContributor` exposing `generate_image` and `generate_video` as `ToolDescriptor`s - (name + summary + `argsSchema`: prompt, size, steps, optional seed-image handle, video durationMs) - with `WritePolicyTier` — image `.confirm`, cloud video `.dangerous`. _Verify: `swift test` — - descriptors present with correct names/tiers; `argsSchema` validates a well-formed args object._ -- [x] 2.2 Gate availability: contribute nothing under `mediaGenEnabled == false` / - `fullPotentialEnabled == false`; advertise `generate_image` only if a `MediaRuntime` advertises - `.image`; advertise `generate_video` only if a video provider + `fleetCloudEscalationEnabled` - (cloud) + remaining budget. _Verify: `swift test` — tools omitted from candidates when unavailable; - present when available (the router never routes to a dead end)._ - -## 3. `MediaGenSink` — the route-loop executor (Core; reuse the route → execute → continue loop) - -- [x] 3.1 Add `MediaGenSink` invoked by the existing loop when the router selects a media tool: resolve - effective tier via the injected `WritePolicyResolving`; surface a `TaskReview`-backed - **awaiting-approval** step (DOWN=approve / RIGHT=skip) for `.confirm`/`.dangerous` BEFORE any compute - or spend. _Verify: `swift test` — `.awaitingApproval` precedes compute; a RIGHT skip applies nothing - and feeds back; a DOWN approval proceeds._ -- [x] 3.2 On approval: pick the `MediaRuntime` for the kind, drive `generate(_:)`, thread `MediaProgress` - into the live UI/parked feed, write the finished asset (§5), and return a `ToolStepResult` — - `.done` carrying the gallery path, `.failed` with a clean headline, or `.declined`. _Verify: - `swift test` — `.done` path carries the gallery URL; the result is fed back as a `.tool` turn._ -- [x] 3.3 Enforce the cloud-video **budget cap** before the call (a media-side mirror of the handoff - per-day cap); an exhausted budget resolves `.declined`/`.failed(cloudBudgetExhausted)` with NO spend. - _Verify: `swift test` — budget-out resolves before any runtime call; never a silent spend._ -- [x] 3.4 Consume `ModelRegistry`/`ensureResident`; when residency math says a heavy gen must EVICT chat, - surface a calm "busy painting" state (chat resumes on finish/park). _Verify: `swift test` against a - stub registry — eviction → busy-painting state; not hidden, not a hang._ - -## 4. The seed (img2img / img2video) path (Core; reuse existing captures) - -- [x] 4.1 Resolve `MediaRequest.seed` from the EXISTING `.screenRegion` (region picker) and - `.clipboardImage` (live pasteboard → PNG) capture seams; wire it as the first frame. No new picker; - no auto-fire on a clipboard image. _Verify: `swift test` — a captured/clipboard image becomes the - request seed; no auto-fire path exists._ -- [x] 4.2 A tool authored as img2img/img2video with no resolvable seed resolves - `.failed(MediaError.seedRequired)` — never a fabricated blank frame; an undecodable seed → - `.failed(seedInvalid)`. _Verify: `swift test` — missing/invalid seed → clean `.failed`, no compute._ - -## 5. Output #1 — the Files-band gallery asset (Core; reuse the band) - -- [x] 5.1 Write the finished `MediaAsset.url` under a dedicated **generated-media gallery** root (local-only, - recoverable, honoring the band's non-destructive scope). _Verify: `swift test` — asset persists under - the gallery root; survives a simulated relaunch read._ -- [x] 5.2 Surface the asset as an ordinary `.fileEntry` in the Files band (reuse on-demand listing, - lift-to-deliver / Open / Open-With / contextual delivery — no new browser). _Verify: `swift test` — - a gallery asset maps to a `.fileEntry` the band lists; identity is path-stable (no strobe)._ - -## 6. Output #2 — the canvas preview/player resolved by the compass - -- [x] 6.1 Model the canvas media state: generating (live `.step` + intermediate preview) → finished - (image, or a player for video). _Verify: `swift test` — the pure state model advances on progress and - terminates on `.finished`/`.failed`/`.cancelled`._ -- [x] 6.2 Build the player overlay as a `DockPreviewOverlay`-pattern panel: **non-activating**, never - key/main, **synchronous `orderOut`**; bud in with `BubbleMorph`. _Verify: `xcodebuild` compile-verify - only; non-activating + synchronous-teardown behavior needs the user's stable-signed build (§9.3)._ -- [x] 6.3 Resolve by the canonical compass: **DOWN (at canvas-top) extracts** — save / paste / set-as; - **RIGHT discards** (the asset is already in the gallery, so discard never loses it). _Verify: - `swift test` — the pure resolve model: DOWN-at-top → extract intent; RIGHT → discard with the file - intact; sub-threshold scroll never resolves._ - -## 7. Parked-while-generating (Core; feed `ParkScheduler`, don't reinvent) - -- [x] 7.1 Feed the parked machinery the job's observable state: **thinking** badge while painting, **done** - badge (+ unseen count) + notch **glow** on `.finished`. _Verify: `swift test` — progress → thinking; - finished → done + unseen count; reported via `ParkScheduler.didAdvance`._ -- [x] 7.2 A `.dangerous` cloud-video step on a parked session calls `ParkScheduler.escalate` → needs-you + - ambient glow (a clean one-line reason); a cancellation is NOT a failure (no failed badge). _Verify: - `swift test` — dangerous-video parked → escalate; cancellation → cancelled, never a failed badge._ - -## 8. `MediaError` taxonomy + boundary mapping (Core; one translator) - -- [x] 8.1 Add `enum MediaError: Error, Equatable` (`LocalizedError`): `noCapableBackend(kind)`, - `seedRequired`, `seedInvalid`, `generationFailed(headline)`, `outputWriteFailed`, - `cloudBudgetExhausted`, `cloudUnavailable`; each a clean per-case `errorDescription`. _Verify: - `swift test` — every case has a clean non-empty description._ -- [x] 8.2 Map vendor/OS errors (mflux/LTXV/ComfyUI, `Process`, `NSURLError`, `FileManager`) into - `MediaError` at the layer boundary (sink/backend conformer); route every surface through - `AIError.message(for:)` → `AIPresentedError`; bounded + non-blocking (never `NSAlert`, never raw in a - headline). _Verify: `swift test` — a vendor error maps to a `MediaError` then a clean headline; raw - text only in details/logs._ -- [x] 8.3 Write one `AuditRecord` per media step (auto/confirmed/declined/escalated/failed) through the - shared `AuditLog` (redacted args summary; failure carries only the clean headline). _Verify: - `swift test` — each terminal outcome appends exactly one record with the effective tier + redacted - args._ - -## 9. Verify - -- [x] 9.1 `swift build` + `swift test` green: the seam, value types, contributor, sink, seed, gallery, - canvas resolve model, parked feed, and `MediaError` are all Core and covered against - `StubMediaRuntime`/stub registry. No piece links MLX. -- [x] 9.2 `openspec validate ai-media-runtime --strict` passes; the `ai-generative-media` ADDED - requirements and the `ai-command-tasks` MODIFIED delta match the implemented seam/tools/sink/output. -- [ ] 9.3 **User run-verify** in a stable-signed build (an agent never builds/signs the `.app`): the model - can route to `generate_image`, the `.confirm` approval shows (DOWN approves), the canvas shows live - step-progress, the finished image lands as a Files-band gallery asset AND a non-activating - compass-resolved canvas preview (DOWN saves/pastes, RIGHT discards leaving the file), the session - parks while painting (thinking → done glow), a cloud-video tool escalates needs-you under the budget - cap, and a heavy gen surfaces "busy painting" rather than a hang. Real latency/heat/RAM/eviction/$ are - observed only here and in the backend slices. diff --git a/openspec/changes/ai-model-fleet/.openspec.yaml b/openspec/changes/ai-model-fleet/.openspec.yaml deleted file mode 100644 index a4ac4d7..0000000 --- a/openspec/changes/ai-model-fleet/.openspec.yaml +++ /dev/null @@ -1,2 +0,0 @@ -schema: spec-driven -created: 2026-06-23 diff --git a/openspec/changes/ai-model-fleet/design.md b/openspec/changes/ai-model-fleet/design.md deleted file mode 100644 index 8456d16..0000000 --- a/openspec/changes/ai-model-fleet/design.md +++ /dev/null @@ -1,99 +0,0 @@ -## Context - -Today `ModelManager` (`AI/ModelManager.swift`) owns exactly one resident runtime. `ModelDescriptor` (`AI/ModelRegistry.swift`) is `Identifiable, Equatable, Sendable` and carries `id`, `name`, size, integrity hash, download source, and `capabilities: Set`. `StandardModelRegistry` lists a few Gemma descriptors; `ModelManager.resolveRuntime(for:)` selects one by required capabilities and loads it through the existing `ModelProvisioner` (real path) or `runtimeFactory` (dev-stub path). There is no notion of a *lane*, a *provider*, a *byte budget*, or *eviction to admit another model*. - -The compute-media-fleet wave (addendum §C1) pins the fleet shapes this slice OWNS: `ModelRole`, `ModelProvider`, the extended `ModelDescriptor`, and the `ModelRegistry` protocol. The descriptor's `lane: ComputeLane?` field consumes `ComputeLane` from `ai-compute-tiers` (§A1) verbatim. The cloud gating (`fleetCloudEscalationEnabled`, under `fullPotentialEnabled`) comes from `ai-full-potential-toggle` (§D1). Sibling slices are authored concurrently, so this slice depends only on those **pinned contracts**, not on sibling files existing yet. - -The hard physical fact this slice encodes: **48 GB unified is a shared budget.** Chat (~17 GB GPU) + ternary (~0.5 GB CPU) + a Q4 image model (~7 GB) + the live KV cache co-reside; a **video gen or an FP16 image model (~24 GB) cannot, and evicts chat.** Cloud members (Claude, GLM-5.2 / 753B) never fit and never try. That admission decision is pure math and must be correct before any weights move — so it lives in Core and is `swift test`-verified without real weights. - -## Goals & Non-Goals - -**Goals** - -- Evolve `ModelDescriptor` to the §C1 shape (additively — every existing construction site keeps compiling) and add `ModelRole` / `ModelProvider` / the `ModelRegistry` protocol. -- A pure, unit-tested `ResidencyPlanner`: given descriptors + budget + free memory + a target, produce the resident set and the eviction list (video/FP16-image evicts chat; cloud never resident). -- Register the **cloud** members (Claude, GLM-5.2) as `.cloud` / `.cloudEscalation` — visible for selection, never resident, off until `fleetCloudEscalationEnabled`. -- Wire residency through the **existing** `ModelProvisioner` / `runtimeFactory` (no provisioner API change); a **fleet-of-one stays byte-for-byte today's behavior**. -- A Hub fleet-roster UX that discloses role / lane / provider / status / honest residency cost (including the evict-chat warning) inline. - -**Non-Goals** - -- The CPU lane runtime, `LaneRouting`, `TernaryCPURuntime` — owned by `ai-compute-tiers`; this slice only carries `lane` on the descriptor and plans the ternary model's co-residency. -- The media seam / tools / sink / concrete image+video backends — owned by `ai-media-runtime` and the two backend slices; this slice registers their descriptors and plans their residency only. -- The toggle storage and the master gate page — owned by `ai-full-potential-toggle`; this slice reads `fleetCloudEscalationEnabled`. -- Context-tuning sliders — owned by `ai-batched-runtime-and-context`; this slice only carries `maxContextTokens` on the descriptor. -- Any actual cloud API call / Claude-handoff mechanics — owned by `ai-claude-handoff`; this slice routes cloud members to that existing surface. - -## Decisions - -### D1. Extend the EXISTING `ModelDescriptor` additively — do not fork a `FleetDescriptor`. - -The §C1 descriptor IS the evolution of today's descriptor, not a parallel type. Add `role: ModelRole`, `lane: ComputeLane?`, `provider: ModelProvider`, `residencyBytes: UInt64`, `maxContextTokens: Int?` to `ModelDescriptor`, with init defaults (`role: .chat`, `lane: .gpu`, `provider: .onDevice`, `maxContextTokens: nil`, `residencyBytes` derived from the existing size when unset) so the `.standard` entries and GemmaRuntime's `pipelineModel(for:)` keep compiling untouched. `residencyBytes` is the eviction-budget number (the on-GPU/on-CPU resident footprint), distinct from the on-disk download size already present. - -- **Rationale:** A fleet-of-one is the existing single descriptor with defaults; a second type would force every caller (selection, lifecycle, Hub status) to branch. Additive fields keep one code path. -- **§C1 note:** §C1 sketches `ModelDescriptor` as `Codable`. Today's descriptor is `Equatable, Sendable, Identifiable` but **not** `Codable` (it is constructed in code, not persisted as JSON). We honor §C1's *field shape* verbatim but keep the descriptor non-`Codable` to match the existing registry (`capabilities: Set` is the live capability set; §C1's `capabilities: Set` is the sketch's stand-in for it). The persisted artifact is the **selected model id** (`ModelSelector`, already persisted) — not the descriptor — so no `Codable` is needed and the existing persistence is untouched. -- **Alternatives rejected:** (a) a separate `FleetDescriptor` wrapping `ModelDescriptor` — doubles the type surface and the Hub status logic; (b) making the descriptor `Codable` to match §C1 literally — would change the live capability type and gain nothing (descriptors are code-defined, the *id* is what persists). - -### D2. `ModelRegistry` is a protocol; the standard fleet roster conforms — cloud members are roster-but-never-resident. - -Add the §C1 `ModelRegistry` protocol (`descriptors()`, `resident()`, `ensureResident(_:)`). `FleetRoster` (evolving `StandardModelRegistry`) lists chat (GPU), ternary (CPU), image (Q4 + FP16 variants), and the two **cloud** members. `descriptors()` returns all (so selection + the Hub roster see cloud members); `resident()` returns only the on-device models currently loaded (cloud is `provider: .cloud` → never included); `ensureResident(id)` of a cloud member is a no-op for residency (it routes to escalation, see D5) and never touches the provisioner. - -- **Rationale:** Cloud members must be *selectable and visible* (the roster, escalation routing) without ever being *resident* — the provider field cleanly separates "is in the fleet" from "occupies budget bytes." The protocol lets a fleet-of-one be a trivial conformer (one chat descriptor) identical to today. -- **Alternatives rejected:** a single concrete registry class — the protocol is one line and lets tests inject a scripted roster; excluding cloud members from `descriptors()` — then the Hub could not show them as available escalation targets. - -### D3. Residency is a PURE PLANNER over `(descriptors, budgetBytes, freeBytes, target)` — eviction is data, not a side effect. - -`ResidencyPlanner.plan(target:descriptors:budgetBytes:freeBytes:currentlyResident:)` returns a `ResidencyPlan { admit: [id], evict: [id], coResident: [id] }` — a pure value. The rules, encoded once and tested: - -1. **Cloud target** (`provider: .cloud`) → empty admit/evict (cost 0, never resident); planning a cloud target is a residency no-op. -2. **Co-residency set:** chat (`.chat`, GPU) + ternary (`.ternaryChat`, CPU) + a **Q4** image model + the live KV reservation fit together when their `residencyBytes` sum + a KV reserve ≤ `budgetBytes` — admit the target alongside the rest. -3. **Eviction trigger:** if admitting the target (a `.video` model, or a `.image` model whose `residencyBytes` exceeds the FP16 threshold) would exceed `budgetBytes` with chat resident, the plan **evicts chat** (and any other GPU-lane occupant) — smallest-victim-first among GPU-lane models until the target fits, chat being the expected victim. The CPU-lane ternary (bandwidth-frugal, ~0.5 GB) is **not** evicted for a GPU gen (different lane, negligible bytes). -4. **Infeasible:** if the target cannot fit even after evicting every evictable on-device model, the plan reports infeasible → `ensureResident` throws `FleetError.cannotAdmit` (mapped through `AIError.message(for:)`, surfaced bounded + non-blocking). - -`ModelManager` is the only place the plan is *applied* (evict → load); the planner itself never calls Metal or the provisioner. Free memory is an **injected probe** (a closure returning `UInt64`) so tests pass fixed values. - -- **Rationale:** "What must be evicted to admit X" is exactly the kind of decision that must be correct before any irreversible memory move, and exactly the kind that is cheap to exhaustively unit-test if it is pure. Returning the plan as data (admit/evict/coResident) lets the Hub *preview* the cost ("selecting Video pauses chat") before the user commits. -- **Alternatives rejected:** computing eviction inside `ModelManager`'s load path with live `MTLDevice` queries — untestable by the agent, and entangles the honest-cost disclosure (which needs the plan *before* loading) with the side effect; a fixed "video always evicts chat" constant — wrong for the Q4-image co-resident case and for future budgets; making the planner async/Metal-aware — kills `swift test` verification. - -### D4. Wire the planner through the EXISTING `ModelProvisioner` / `runtimeFactory` — fleet-of-one short-circuits. - -`ModelManager.ensureResident(id)` (the registry method, implemented on the manager) runs `ResidencyPlanner.plan(...)`, evicts each `plan.evict` id via the **existing** evict path, then loads the target through the **existing** `ModelProvisioner` (real) or `runtimeFactory` (dev-stub) — no new provisioning seam. When the roster is a fleet-of-one (only the chat descriptor, no lane/cloud members) the plan is trivially `{admit:[chat], evict:[], coResident:[chat]}` and the load path is byte-for-byte today's lazy-load-and-keep-resident behavior. - -- **Rationale:** The blueprint pins "reuse `ModelProvisioner`/`runtimeFactory`; a fleet-of-one MUST remain valid." Threading the planner *around* the existing load path (not through a new one) guarantees the single-model lifecycle, integrity-verify, residency, and per-model status requirements still hold unchanged. -- **Alternatives rejected:** a new `FleetProvisioner` — duplicates the download/verify/load/evict logic the manager already owns and risks regressing the metallib-bundle / TCC-stable-signing contracts. - -### D5. Cloud members ride the Claude-handoff escalation surface, gated by `fleetCloudEscalationEnabled` (under `fullPotentialEnabled`). - -A `.cloudEscalation` descriptor is never loaded; selecting one as a command's model routes the turn through the **existing** Claude-handoff escalation path (`ai-claude-handoff` §3.8: confirm-by-default, per-day budget cap, audited, fire-and-forget). The entire cloud tier is hidden from selection and never escalated to unless `fleetCloudEscalationEnabled` is true (and that flag is itself only meaningful when `fullPotentialEnabled` is on). GLM-5.2 is registered as a second cloud member alongside Claude — same `.cloud`/`.cloudEscalation` shape, its 753B/1M-ctx/MIT facts in `name`/`capabilities`, `residencyBytes: 0`. - -- **Rationale:** "Cloud is escalation, not residency" is the addendum's load-bearing decision (§5.6, no silent spend). Reusing the Claude-handoff surface means no new audit/budget/confirm machinery — GLM-5.2 is just a second escalation destination. -- **Alternatives rejected:** treating GLM-5.2 as a giant local model with a degraded path — explicitly forbidden (§5.2, datacenter scale, does not fit 48 GB); a separate cloud-runtime seam — the handoff surface already is that seam. - -### D6. The Hub fleet roster discloses cost in the same breath it offers selection — including the evict-chat warning and the cloud badge/budget. - -The Hub AI page's model picker becomes a **fleet roster**: each member row shows role (Chat / Ternary / Image / Video / Cloud), lane (GPU / CPU / Cloud), provider, the existing per-model on-disk/resident status, and its **honest residency cost** (`residencyBytes` in GB). A member whose `ResidencyPlanner` plan would **evict chat** renders that warning inline ("selecting Video pauses the chat model; it reloads when generation finishes"), computed from the *plan*, not hard-coded. Cloud members show a **Cloud** badge + their escalation cost ($ / `mediaVideoBudgetPerDay`-style per-day cap) and are **disabled with an explanatory caption** until `fleetCloudEscalationEnabled`. A fleet-of-one renders exactly as today's single picker. - -- **Rationale:** The wave's honest-surface ethos (§D1 disclosure UX) — no fans-screaming surprise, no surprise spend. Driving the warning from the plan keeps UI and math in sync. -- **Alternatives rejected:** a separate "advanced fleet" window — the Hub is the single config surface (configuration-hub invariant); a static cost label — would drift from the real budget/free-memory. - -### D7. `FleetError` only for cases `RuntimeError` cannot carry; everything routes through the one translator. - -Add a `FleetError: Error, Equatable, LocalizedError` with at most the cases the existing taxonomy cannot express: `.cannotAdmit(modelName:)` (residency infeasible even after eviction), `.cloudDisabled(modelName:)` (a cloud member selected while `fleetCloudEscalationEnabled` is off). Capability-mismatch, unavailable-hardware, download/integrity failures stay `RuntimeError`. Map any OS/probe error at the boundary into the taxonomy; surface via `AIError.message(for:)` → `AIPresentedError`, bounded + non-blocking, never `NSAlert`, never raw error text in a headline. A failed admission is an observable `.failed` for that selection, never a false "loaded." - -- **Rationale:** Blueprint invariant — one taxonomy, one translator, mapped at the boundary. The two new cases are genuinely fleet-specific (eviction infeasibility, cloud-gated) and have no `RuntimeError` equivalent. -- **Alternatives rejected:** overloading `RuntimeError.unavailable(reason:)` for both — loses the structured model name the Hub needs to render a clean headline + Retry; raw-interpolating the eviction list into a string — banned in headlines. - -## Per-component target-split & verification - -| Component | Target | Verification | -|---|---|---| -| `ModelRole` / `ModelProvider` enums | Core (MLX-free) | `swift test` — round-trip, exhaustive cases | -| Extended `ModelDescriptor` (`role`/`lane`/`provider`/`residencyBytes`/`maxContextTokens`, additive init defaults) | Core (`AI/ModelRegistry.swift`) | `swift test` — fleet-of-one default-init equals today's descriptor; new fields set/read; `swift build` proves existing construction sites still compile | -| `ModelRegistry` protocol + `FleetRoster` (chat/ternary/image/cloud members) | Core (`AI/Fleet/`) | `swift test` — `descriptors()` includes cloud members; `resident()` never includes `.cloud`; a scripted roster conforms | -| `ResidencyPlanner` (pure plan: admit/evict/coResident) | Core (`AI/Fleet/ResidencyPlanner.swift`) | `swift test` — Q4 image co-resides with chat+ternary+KV under budget; video/FP16-image evicts chat; ternary (CPU) not evicted for a GPU gen; cloud target → empty plan; infeasible → `cannotAdmit`; injected free-memory probe with fixed values | -| `FleetError` taxonomy + `AIError.message(for:)` routing | Core | `swift test` — each case yields a clean headline via the single translator; no raw interpolation | -| `ModelManager` consuming registry + planner around the existing `ModelProvisioner`/`runtimeFactory` | Core (`AI/ModelManager.swift`) | `swift test` (with `StubLLMRuntime`) — `ensureResident` evicts the planned ids then loads the target; fleet-of-one path equals today's lazy-load; cloud `ensureResident` is a residency no-op | -| `fleetCloudEscalationEnabled` read seam (flag owned by `ai-full-potential-toggle`) | Core (read-only consumer) | `swift test` — off → cloud members not escalated / not selectable; on → escalation routes to handoff | -| Hub fleet-roster UX (role/lane/provider/status/cost, evict-chat warning, cloud badge + gated caption) | Native-linked (SwiftUI in app target) | `xcodebuild` compile-verify only; **user stable-signed build** verifies the live roster rendering, real per-model status, and the disclosure copy | -| GemmaRuntime `pipelineModel(for:)` consuming the new descriptor fields | GemmaRuntime (MLX-linked) | `xcodebuild` compile-verify only; the new fields must compile in the MLX path | -| Real residency / eviction with real weights (does Video actually evict chat + reload? does Q4 image truly co-reside under 48 GB? live free-memory probe) | User stable-signed build ONLY | The agent never builds/signs/installs the `.app` (ad-hoc signing breaks TCC; the `*.bundle` metallib copy must not regress). Live memory behavior, eviction/reload latency, and large-weight on-disk status are observable only here | diff --git a/openspec/changes/ai-model-fleet/proposal.md b/openspec/changes/ai-model-fleet/proposal.md deleted file mode 100644 index d6ddb82..0000000 --- a/openspec/changes/ai-model-fleet/proposal.md +++ /dev/null @@ -1,35 +0,0 @@ -## Why - -V2.5 turns the agent from a single-GPU, single-model companion into a **two-lane, multi-model fleet**: a Gemma chat model on the GPU lane, a small ternary model on the CPU lane (`ai-compute-tiers`), generative-image and -video backends (`ai-media-runtime` + its two backends), and — off by default — cloud escalation members (Claude, GLM-5.2). Today's `ModelManager` owns exactly **one** resident runtime: it downloads, lazy-loads, keeps resident, and evicts a single model, and its `ModelDescriptor` describes only `id/name/size/hash/source/capabilities`. That shape cannot answer the questions the fleet asks: *which physical lane does this model run on? is it on-device or cloud? how many bytes does it cost against the 48 GB unified budget, and what must be evicted to make room for it?* - -Two facts make a registry-with-residency the right (and only honest) shape: - -1. **48 GB is a hard, shared budget.** On the target machine (M5 Pro / 48 GB unified) the chat weights (~17 GB GPU), the ternary model (~0.5 GB CPU), a Q4 image model (~7 GB), and the live KV cache **can co-reside** — but a **video generation or an FP16 image model (~24 GB) cannot fit alongside chat and must EVICT it**. The companion goes quiet while it paints. That is a residency/eviction decision, computed against per-descriptor `residencyBytes`, and it is *pure math* — it must be decided correctly before any weights move, and it must be unit-testable without real weights. -2. **Cloud members do not fit at all and must never try.** GLM-5.2 (753B MoE / 1M context / MIT) is datacenter scale — local residency is not an option; Claude is a hosted API. Both are `provider: .cloud`, `role: .cloudEscalation`, **never resident locally**: they ride the existing Claude-handoff / escalation paths and are **OFF until `fleetCloudEscalationEnabled`** (no silent spend). - -The existing seams already make this additive. `ModelManager` injects the real runtime via the existing `ModelProvisioner` / `runtimeFactory` — this slice keeps those seams and feeds them from a registry of descriptors instead of a single descriptor. A **fleet-of-one MUST remain valid**: with only the chat descriptor registered and no cloud/lane members, behavior is byte-for-byte today's single-model lifecycle. - -## What Changes - -- **Extended `ModelDescriptor` (Core, pure).** Evolve the existing descriptor (which already carries `id/name/size/hash/source/capabilities`) to additionally carry the fleet fields pinned in addendum §C1: `role: ModelRole` (`.chat`/`.ternaryChat`/`.image`/`.video`/`.cloudEscalation`), `lane: ComputeLane?` (nil for cloud — consumed verbatim from `ai-compute-tiers` §A1), `provider: ModelProvider` (`.onDevice`/`.cloud`), `residencyBytes: UInt64` (0 for cloud), `maxContextTokens: Int?` (chat/ternary only). Additive with sensible defaults so every existing construction site (the `.standard` registry entries, GemmaRuntime's `pipelineModel(for:)`) keeps compiling — a bare chat descriptor defaults to `.chat` / `.gpu` / `.onDevice`. -- **`ModelRegistry` protocol + the fleet registry (Core, pure).** Add the `ModelRegistry` protocol (§C1): `descriptors()`, `resident()`, `ensureResident(_:)` (may EVICT). The standard registry becomes the fleet roster: chat (GPU), ternary (CPU), image (Q4 / FP16 variants), and the two **cloud** members (Claude, GLM-5.2) as `.cloud` / `.cloudEscalation` — present in `descriptors()` for selection/UX but **never** returned by `resident()` and never passed to the provisioner. -- **Residency / eviction MATH (Core, pure, `swift test`-verified).** A `ResidencyPlanner` value type that, given the descriptors, the 48 GB budget, current free unified memory, and a target model, computes the **resident set** and the **eviction list** to admit the target: chat (GPU) + ternary (CPU) + Q4 image + KV **co-reside**; admitting a **video** model or an **FP16 image** model **evicts chat** (and surfaces "the assistant is busy painting"); cloud members are never planned for residency (cost 0, never resident). The plan is a pure function of `(descriptors, budgetBytes, freeBytes, target)` — no Metal, no real weights. -- **Fleet residency wired through the existing provisioner.** `ModelManager` consumes the registry + planner: `ensureResident(id)` runs the plan, evicts what the plan names (reusing the existing evict path), then loads the target via the **existing** `ModelProvisioner` / `runtimeFactory` (no provisioner API change). A fleet-of-one short-circuits to today's load path. -- **Cloud members ride the handoff/escalation paths, gated.** Claude and GLM-5.2 are selectable as escalation targets only; selecting one routes through the existing Claude-handoff escalation surface (confirm-by-default, audited), and the whole cloud tier is gated behind the new `fleetCloudEscalationEnabled` flag (consumed from `ai-full-potential-toggle` §D1, itself under `fullPotentialEnabled`). Off → cloud members are hidden from selection and never escalated to. -- **Model-selection UX in the Hub.** The Hub's AI page gains a **fleet roster**: each member shows its role, lane (GPU / CPU / Cloud), provider, on-disk/resident status (per the existing per-model status rule), and its **honest residency cost** — and, for a member whose admission would **evict chat**, says so *in the same breath* it offers selection ("selecting Video pauses the chat model; it reloads when generation finishes"). Cloud members show a **Cloud** badge, their escalation cost ($ / per-day budget), and are disabled with an explanatory caption until `fleetCloudEscalationEnabled`. - -## Capabilities - -### Modified Capabilities - -- `on-device-ai-runtime`: ADD the fleet `ModelRole` / `ModelProvider` / extended `ModelDescriptor` (`role`/`lane`/`provider`/`residencyBytes`/`maxContextTokens`); ADD the `ModelRegistry` protocol (`descriptors`/`resident`/`ensureResident`); ADD the 48 GB residency/eviction requirement (co-residency set + a video/FP16-image gen evicts chat) computed by a pure planner; ADD cloud fleet members (Claude, GLM-5.2) that are never resident locally and ride the escalation paths off until enabled; KEEP fleet-of-one identical to today. The existing single-resident lifecycle, registry capability-selection, `ModelProvisioner`/`runtimeFactory` seam, and `generate`/`structured` requirements are otherwise unchanged. -- `configuration-hub`: ADD the AI-page fleet roster (role / lane / provider / per-model status / honest residency cost, with the evict-chat disclosure surfaced inline) and the cloud-member badge + escalation-cost disclosure gated behind `fleetCloudEscalationEnabled`. The existing AI-page model picker and per-model lifecycle status are preserved (a fleet-of-one renders as today's single picker). - -## Impact - -- **Code (MLX-free Core — `swift build` / `swift test`):** extend `ModelDescriptor` (`AI/ModelRegistry.swift`) with `role`/`lane`/`provider`/`residencyBytes`/`maxContextTokens` (additive init defaults); add `ModelRole`/`ModelProvider` enums and the `ModelRegistry` protocol; a new `AI/Fleet/ResidencyPlanner.swift` (pure residency/eviction math) + `FleetRoster` (the standard fleet descriptors incl. cloud members); a `FleetError` taxonomy (only the cases `RuntimeError` cannot carry); `ModelManager` consuming registry + planner around the **existing** `ModelProvisioner`/`runtimeFactory`; the persisted `fleetCloudEscalationEnabled` read seam (the flag itself is owned by `ai-full-potential-toggle`). A `StubLLMRuntime`-backed fleet (the existing dev path) keeps the whole thing `swift test`-verified without real weights. -- **Code (GemmaRuntime / native-linked — `xcodebuild` compile-verify only; real correctness needs the user's stable-signed build):** wiring the fleet descriptors into GemmaRuntime's `pipelineModel(for:)` so the new fields compile; **real residency/eviction with real weights** (does admitting Video actually evict chat and reload it? does Q4 image truly co-reside with chat + KV under 48 GB?) is observable only in the user's signed build — the agent compile-verifies the math and the wiring, never the live memory behavior. -- **User-stable-signed build only:** live free-memory probing, real eviction/reload latency, and the on-disk/resident status of large weights. The agent never builds/signs/installs the `.app` (ad-hoc signing breaks TCC grants; the `*.bundle` metallib copy in `build-app.sh` must not regress). -- **Consumes (verbatim, never redefines):** `ComputeLane` + `AgentWorkRole`/`LaneRouting` (owned by `ai-compute-tiers` §A1) — the descriptor's `lane` field is this enum; `fleetCloudEscalationEnabled` + `fullPotentialEnabled` gating (owned by `ai-full-potential-toggle` §D1); the existing `ModelProvisioner`/`runtimeFactory`/per-model lifecycle/`ModelSelector` seams; the Claude-handoff escalation surface (`ai-claude-handoff` §3.8) as the cloud-member route; `RuntimeError` + `AIError.message(for:)` for all failure surfacing. -- **House rules:** Apple-Silicon M5 (M4 min), no degraded/low-end path — the fleet exists *because* 48 GB unified can host it. One error taxonomy (`RuntimeError`, plus `FleetError` only where it cannot carry a case), one translator (`AIError.message(for:)`), mapped at the boundary; a residency plan that fails to admit a model is an observable `.failed` (never a false "loaded"), surfaced bounded + non-blocking, never an `NSAlert`, never raw error text in a headline. No silent cloud spend — cloud members are off until `fleetCloudEscalationEnabled`. -- **Out of scope:** the CPU lane runtime + role→lane policy (`ai-compute-tiers` owns `ComputeLane`/`LaneRouting`/`TernaryCPURuntime`); the media seam + tools + sink (`ai-media-runtime`); the concrete image/video backends (`ai-local-image-generation`, `ai-video-animation-generation` — this slice only registers their descriptors + plans their residency); the master/sub toggle storage (`ai-full-potential-toggle` owns the flags; this slice reads `fleetCloudEscalationEnabled`); the actual Claude-handoff escalation mechanics (`ai-claude-handoff`); context-tuning sliders (`ai-batched-runtime-and-context` owns `maxContextTokens` tuning — this slice only carries the field on the descriptor). diff --git a/openspec/changes/ai-model-fleet/specs/configuration-hub/spec.md b/openspec/changes/ai-model-fleet/specs/configuration-hub/spec.md deleted file mode 100644 index dc16cd5..0000000 --- a/openspec/changes/ai-model-fleet/specs/configuration-hub/spec.md +++ /dev/null @@ -1,33 +0,0 @@ -## ADDED Requirements - -### Requirement: AI page presents the model fleet roster with honest residency cost - -The Hub's **AI** page SHALL present the model picker as a **fleet roster** in which each member shows its role (Chat / Ternary / Image / Video / Cloud), its compute lane (GPU / CPU / Cloud), its provider, the existing per-model on-disk / resident status (reflecting the currently selected member's own state, not a single global status), and its **honest residency cost** (its resident footprint in gigabytes). A member whose admission would **evict the chat model** SHALL disclose that consequence **inline, in the same breath it offers selection**, computed from the residency plan rather than hard-coded — for example, "selecting Video pauses the chat model; it reloads when generation finishes." A registry containing only the single chat descriptor (a fleet-of-one) SHALL render as today's single model picker. - -#### Scenario: The roster shows each member's role, lane, and cost -- **WHEN** the user opens the AI page with a multi-member fleet -- **THEN** each member is listed with its role, lane, provider, its own on-disk/resident status, and its residency cost in gigabytes - -#### Scenario: An evicting member discloses the cost inline -- **WHEN** the user views a fleet member whose admission the residency plan says would evict the chat model (a video or full-precision image model) -- **THEN** that member's row states that selecting it pauses the chat model and that chat reloads when generation finishes, and a member that co-resides (a Q4 image model) shows no such warning - -#### Scenario: A fleet-of-one is unchanged -- **WHEN** only the chat descriptor is registered -- **THEN** the AI page renders the existing single model picker with its per-model lifecycle status, unchanged - -### Requirement: Cloud fleet members are badged, cost-disclosed, and gated off by default - -The Hub's **AI** page SHALL show each cloud fleet member (Claude, GLM-5.2) with a **Cloud** badge, its escalation cost (per-call $ and per-day budget cap), and SHALL render it **disabled with an explanatory caption** until cloud escalation is enabled (the cloud-escalation toggle, itself under the master full-potential gate). No cloud member SHALL be selectable, and no spend SHALL be implied, while the cloud tier is off. Any admission or escalation failure SHALL surface as a **bounded, non-blocking** row (a clean headline via the single error translator, opt-in copyable details, and a Retry affordance) — never an app-modal alert and never raw error text in a headline. - -#### Scenario: Cloud members are disabled and captioned until enabled -- **WHEN** the user opens the AI page while cloud escalation is disabled -- **THEN** the cloud members show a Cloud badge and their escalation cost, are disabled, and carry a caption explaining they require enabling cloud escalation - -#### Scenario: Enabling cloud makes the members selectable -- **WHEN** the user enables cloud escalation -- **THEN** the cloud members become selectable, still showing their per-call $ and per-day budget cost - -#### Scenario: A fleet failure is bounded and non-blocking -- **WHEN** admitting or escalating to a fleet member fails -- **THEN** the AI page shows a bounded, non-blocking row with a clean headline, opt-in copyable details, and Retry — not a modal alert and not raw error text in the headline diff --git a/openspec/changes/ai-model-fleet/specs/on-device-ai-runtime/spec.md b/openspec/changes/ai-model-fleet/specs/on-device-ai-runtime/spec.md deleted file mode 100644 index 3d69c7b..0000000 --- a/openspec/changes/ai-model-fleet/specs/on-device-ai-runtime/spec.md +++ /dev/null @@ -1,69 +0,0 @@ -## ADDED Requirements - -### Requirement: Model fleet descriptor with role, lane, provider, and residency cost - -The system SHALL describe every known model with a fleet `ModelDescriptor` that carries, in addition to its existing identity, size, integrity hash, download source, and capability set: a **role** (`chat`, `ternaryChat`, `image`, `video`, or `cloudEscalation`), an optional **compute lane** (`gpu` or `cpuTernary`, nil for cloud models), a **provider** (`onDevice` or `cloud`), a **residency cost in bytes** (the resident footprint used by the eviction budget; 0 for cloud models), and an optional **maxContextTokens** (chat/ternary models only). The added fields SHALL be additive: a descriptor constructed without them SHALL default to an on-device GPU chat model, so existing single-model registry entries and the model-loading pipeline are unchanged. - -#### Scenario: A bare chat descriptor defaults to the single-model shape -- **WHEN** a `ModelDescriptor` is constructed without specifying role, lane, or provider -- **THEN** it reads as a `chat` model on the `gpu` lane with provider `onDevice`, identical to the pre-fleet descriptor - -#### Scenario: A descriptor carries its fleet attributes -- **WHEN** an image, video, ternary, or cloud model is registered -- **THEN** its descriptor exposes its role, its lane (nil for cloud), its provider, and its residency cost in bytes (0 for cloud), and these are available to selection, residency planning, and the configuration UI - -### Requirement: Model registry exposes the fleet, residency, and admission - -The system SHALL access the fleet through a `ModelRegistry` that can enumerate **all** known descriptors, enumerate the **currently resident** descriptors, and **ensure a model is resident** by identifier (admitting it, evicting other models as required by the residency budget). A `cloud`-provider model SHALL appear in the full enumeration (so it can be selected and shown in the UI) but SHALL NEVER appear in the resident enumeration and ensuring it resident SHALL be a residency no-op (it is not loaded locally). A registry containing only the single chat descriptor (a fleet-of-one) SHALL behave exactly as today's single-model lifecycle. - -#### Scenario: The fleet is enumerable but cloud is never resident -- **WHEN** the registry is queried for all descriptors and for resident descriptors -- **THEN** every fleet member (including the cloud members) appears in the full list, and no `cloud`-provider member ever appears in the resident list - -#### Scenario: A fleet-of-one behaves as today -- **WHEN** the registry holds only the chat descriptor and the chat model is ensured resident -- **THEN** the model is lazy-loaded and kept resident exactly as the pre-fleet single-model lifecycle, with no eviction step - -### Requirement: 48 GB residency and eviction is computed by a pure planner - -The system SHALL decide which models can co-reside and which must be evicted with a **pure residency planner** that, given the fleet descriptors, the unified-memory budget, the currently free memory, and a target model, produces a plan naming exactly which models to **admit**, which to **evict**, and which **co-reside** — before any weights move. The chat model (GPU lane), the ternary model (CPU lane), a quantized (Q4) image model, and the live KV reservation SHALL be able to co-reside within the budget. Admitting a **video** model or a full-precision (FP16) image model that cannot fit alongside the chat model SHALL evict the chat model (the assistant goes quiet while it generates), without evicting the CPU-lane ternary model. Cloud models SHALL never be planned for residency. A target that cannot fit even after evicting every evictable on-device model SHALL be reported as a clean admission failure, not loaded silently and not left in a false "loaded" state. - -#### Scenario: Q4 image co-resides with chat, ternary, and KV -- **WHEN** the planner is asked to admit a quantized (Q4) image model with chat and ternary resident and KV reserved, within the budget -- **THEN** the plan admits the image model with no eviction and the co-resident set includes chat, ternary, the image model, and the KV reservation - -#### Scenario: A heavy generation evicts chat -- **WHEN** the planner is asked to admit a video model (or an FP16 image model) that does not fit alongside the resident chat model -- **THEN** the plan evicts the chat model so the heavy model fits, leaves the CPU-lane ternary model resident, and the eviction is surfaced honestly to the user ("the assistant is busy painting"; chat reloads when generation finishes) - -#### Scenario: A cloud target requires no residency -- **WHEN** the planner is asked to admit a `cloud`-provider model -- **THEN** the plan admits and evicts nothing (the cloud model is never resident locally) - -#### Scenario: An infeasible admission fails cleanly -- **WHEN** a target cannot fit even after evicting every evictable on-device model -- **THEN** the admission reports a clean, bounded failure through the single error translator and the target is not loaded, rather than leaving a false "loaded" state - -### Requirement: Cloud fleet members ride escalation and are off by default - -The system SHALL register **Claude** and **GLM-5.2** as `cloud`-provider, `cloudEscalation`-role fleet members that are never resident locally (GLM-5.2's 753B-parameter / 1M-context scale does not fit the local budget; local is not an option). Selecting a cloud member SHALL route the turn through the existing Claude-handoff escalation path (confirm-by-default, per-day budget-capped, audited, fire-and-forget) rather than loading any local weights. The cloud tier SHALL be **off by default**: while cloud escalation is disabled, cloud members SHALL NOT be offered for selection and selecting one SHALL yield a clean "cloud disabled" failure rather than any network call or spend. - -#### Scenario: Cloud members are never resident -- **WHEN** a cloud member is selected and the escalation tier is enabled -- **THEN** no local weights are loaded for it and the turn is routed through the existing Claude-handoff escalation path with its confirm/budget/audit gating - -#### Scenario: Cloud is off by default with no silent spend -- **WHEN** cloud escalation is disabled and a cloud member is selected -- **THEN** the selection yields a clean "cloud disabled" failure, no network call is made, and no spend occurs - -### Requirement: Fleet residency reuses the existing provisioner seam - -The system SHALL apply a residency plan (evict then load) through the **existing** model-provisioner / runtime-factory seam — no new provisioning mechanism. The fleet planner SHALL decide *which* models to evict and admit; the existing lifecycle SHALL perform the eviction, the integrity-verified load, the keep-resident, and the per-model status update unchanged. A failure mapped at the layer boundary into the model-error taxonomy SHALL surface through the single error translator as a bounded, non-blocking message (never a modal alert, never raw error text in a headline), and a model that did not become resident SHALL be an observable failed state, never a false "loaded." - -#### Scenario: Admission loads through the existing seam -- **WHEN** a plan names a target to admit and models to evict -- **THEN** the named models are evicted and the target is loaded through the existing provisioner / runtime-factory path, with the existing integrity-verify, keep-resident, and per-model status behavior intact - -#### Scenario: A failed admission is observable, not silent -- **WHEN** an admission fails (infeasible residency, integrity failure, or unavailable hardware) -- **THEN** the model is reported as a failed state through the single error translator as a bounded, non-blocking message, and is never shown as a false "loaded" diff --git a/openspec/changes/ai-model-fleet/tasks.md b/openspec/changes/ai-model-fleet/tasks.md deleted file mode 100644 index c93fcfb..0000000 --- a/openspec/changes/ai-model-fleet/tasks.md +++ /dev/null @@ -1,51 +0,0 @@ -> Decomposed for a workflow fan-out: §1–§4 are the pure-Core substrate (do first, all `swift test`), §5 wires the manager around the existing provisioner, §6 is the cloud-escalation gating, §7 is the Hub roster UX (native — `xcodebuild` compile-verify only; user run-verify), §8 verifies. The Hub roster and any real residency behavior require the user's stable-signed build — the agent compile-verifies and unit-tests the math only. - -## 1. Fleet enums + extended `ModelDescriptor` (pure Core) - -- [x] 1.1 Add `ModelRole` (`.chat`/`.ternaryChat`/`.image`/`.video`/`.cloudEscalation`) and `ModelProvider` (`.onDevice`/`.cloud`) enums to `AI/ModelRegistry.swift` (Core), verbatim per addendum §C1. *Verify: `swift test` — exhaustive cases; `swift build`.* -- [x] 1.2 Extend `ModelDescriptor` with `role: ModelRole`, `lane: ComputeLane?` (consume `ComputeLane` from `ai-compute-tiers` §A1 — do NOT redefine), `provider: ModelProvider`, `residencyBytes: UInt64`, `maxContextTokens: Int?`. Add **init defaults** (`role: .chat`, `lane: .gpu`, `provider: .onDevice`, `maxContextTokens: nil`, `residencyBytes` derived from existing size when unset) so every existing construction site keeps compiling (D1). *Verify: `swift build` — existing `.standard` entries + GemmaRuntime `pipelineModel(for:)` compile untouched; `swift test` — default-init descriptor equals today's shape and new fields set/read.* - -## 2. `ModelRegistry` protocol + `FleetRoster` (pure Core) - -- [x] 2.1 Add the `ModelRegistry` protocol (`descriptors()`, `resident()`, `ensureResident(_:) throws`) to `AI/Fleet/` per §C1. *Verify: `swift build`.* -- [x] 2.2 Evolve `StandardModelRegistry` into `FleetRoster`: chat (`.chat`/`.gpu`/`.onDevice`), ternary (`.ternaryChat`/`.cpuTernary`/`.onDevice`, ~0.5 GB), image Q4 + image FP16 variants (`.image`/`.gpu`), and the two **cloud** members — Claude and GLM-5.2 (`.cloud`/`.cloudEscalation`, `residencyBytes: 0`, GLM-5.2's 753B/1M-ctx/MIT facts in `name`/`capabilities`). *Verify: `swift test` — `descriptors()` includes all incl. both cloud members; `resident()` excludes every `.cloud` member; capability-selection still finds the chat model.* -- [x] 2.3 Add a `StubModelRegistry` (Core, test-only) that scripts an arbitrary roster (incl. fleet-of-one). *Verify: `swift test` — fleet-of-one roster conforms; a scripted multi-member roster is queryable.* - -## 3. `ResidencyPlanner` — the pure residency/eviction math (pure Core) - -- [x] 3.1 Add `ResidencyPlan { admit:[String]; evict:[String]; coResident:[String] }` and `ResidencyPlanner.plan(target:descriptors:budgetBytes:freeBytes:currentlyResident:)` (pure value function, injected free-memory probe — no Metal) in `AI/Fleet/ResidencyPlanner.swift` (D3). *Verify: `swift build`.* -- [x] 3.2 Implement co-residency: chat + ternary + Q4 image + a KV reserve admit together when their `residencyBytes` sum + reserve ≤ `budgetBytes`. *Verify: `swift test` — Q4 image co-resides with chat+ternary+KV under the 48 GB budget (no eviction).* -- [x] 3.3 Implement the eviction trigger: admitting a `.video` model, or a `.image` model above the FP16 threshold, evicts the GPU-lane chat (smallest-victim-first among GPU-lane occupants) until the target fits; the CPU-lane ternary is NOT evicted for a GPU gen (D3). *Verify: `swift test` — video evicts chat; FP16 image evicts chat; ternary survives; `coResident` reflects the post-eviction set.* -- [x] 3.4 Implement the cloud + infeasible cases: a `.cloud` target → empty admit/evict (residency no-op); a target that cannot fit even after evicting every evictable on-device model → infeasible flag. *Verify: `swift test` — cloud target plan is empty; an over-budget target reports infeasible.* - -## 4. `FleetError` taxonomy (pure Core) - -- [x] 4.1 Add `FleetError: Error, Equatable, LocalizedError` with `.cannotAdmit(modelName:)` and `.cloudDisabled(modelName:)` — only the cases `RuntimeError` cannot carry (D7); clean per-case `errorDescription`. *Verify: `swift test` — each case has a non-empty clean description.* -- [x] 4.2 Route `FleetError` through `AIError.message(for:)` → `AIPresentedError` (the single translator; the eviction list goes in `details`, never the headline). *Verify: `swift test` — each case yields a clean bounded headline; no raw interpolation in the headline.* - -## 5. `ModelManager` consumes registry + planner around the EXISTING provisioner (Core) - -- [x] 5.1 Give `ModelManager` a `ModelRegistry` + a `ResidencyPlanner`; implement `ensureResident(id)` = run the plan, evict each `plan.evict` via the **existing** evict path, load the target via the **existing** `ModelProvisioner` (real) / `runtimeFactory` (dev-stub). NO new provisioning seam (D4). *Verify: `swift test` (with `StubLLMRuntime`) — `ensureResident` evicts the planned ids then loads the target; a `cannotAdmit` plan throws `FleetError.cannotAdmit` and leaves state `.failed`, never a false `.loaded`.* -- [x] 5.2 Prove the **fleet-of-one** path: a roster with only the chat descriptor short-circuits to today's lazy-load-and-keep-resident behavior (no eviction, identical lifecycle/integrity-verify/per-model status). *Verify: `swift test` — fleet-of-one `ensureResident` matches the pre-fleet single-model load (snapshot of state transitions).* -- [x] 5.3 Cloud `ensureResident` is a residency no-op (never touches the provisioner / never loads weights). *Verify: `swift test` — `ensureResident(cloudID)` loads nothing and `resident()` is unchanged.* - -## 6. Cloud-escalation gating (Core read seam) - -- [x] 6.1 Add a read seam for `fleetCloudEscalationEnabled` (the flag itself owned by `ai-full-potential-toggle` §D1 — consume, do NOT define it; default treated as false). *Verify: `swift build`.* -- [x] 6.2 Gate cloud members: when off, cloud descriptors are not offered for selection and selecting one yields `FleetError.cloudDisabled`; when on, a cloud-member selection routes the turn through the existing Claude-handoff escalation surface (`ai-claude-handoff` §3.8 — confirm-by-default, budget-capped, audited) — this slice routes, it does not reimplement the handoff (D5). *Verify: `swift test` — off → `cloudDisabled` + not selectable; on → routes to the handoff seam (asserted via a handoff spy).* - -## 7. Hub fleet-roster UX (native — `xcodebuild` compile-verify only; user run-verify) - -> Native SwiftUI in the app target. The agent NEVER builds/signs/installs the `.app` (ad-hoc signing breaks TCC). Real roster rendering, live per-model status, and the disclosure copy are verified only in the **user's stable-signed build** (§8.3). - -- [x] 7.1 Evolve the Hub AI-page model picker into a **fleet roster**: each member shows role, lane (GPU / CPU / Cloud), provider, the existing per-model on-disk/resident status, and `residencyBytes` (GB) as its honest cost (D6). *Verify: `xcodebuild` compile; **user run-verify** the roster lists every member with correct lane/role/status.* **(fix-roster-ui: fixed two model-picker bugs (co-residency + download). `HubFleetRosterView`'s single `@Binding selectedID` radio is replaced by TWO bindings — `activeChatID: String?` (radio AMONG chat-role members, `aiSelectedChatModelID`) and `enabledCapabilityModelIDs: Set` (independent image/ternary/video toggles, `aiEnabledCapabilityModelIDs`); a capability Toggle on-enable inserts the id AND triggers a download if not on-disk. New `ModelManager.status(for:)` is a pure, non-mutating per-descriptor lifecycle probe so each row shows its OWN status (on-disk / downloading%+progress / resident). Per-row download wired: `onDownloadCapabilityModel` on the view + `HubContext`, `AppCoordinator.downloadCapabilityModel(_:)` reusing the EXISTING `ModelManager.downloadAndVerify`/provisioner path (no new seam), bound in `setupHubContext`. `AppSettings` adds the two persisted keys; the legacy `ModelManagementView` stays for the active chat model. `swift build` + `swift test` (1444) + `xcodebuild` GREEN.)** -- [x] 7.2 Surface the **evict-chat** disclosure inline, computed from the `ResidencyPlanner` plan (not hard-coded): a member whose admission evicts chat shows "selecting ‹Role› pauses the chat model; it reloads when generation finishes." *Verify: `xcodebuild` compile; `swift test` the underlying plan→warning mapping; **user run-verify** the warning shows for Video/FP16-image and not for Q4-image.* -- [x] 7.3 Cloud members show a **Cloud** badge + escalation cost ($ / per-day cap) and are **disabled with an explanatory caption** until `fleetCloudEscalationEnabled`; a fleet-of-one renders as today's single picker. *Verify: `xcodebuild` compile; **user run-verify** cloud rows are gated/captioned off, enabled on; fleet-of-one is unchanged.* **(wire-full-potential-gates: the live `HubFleetRosterView` is now constructed with `cloudEscalationEnabled: { settings.fullPotentialGate.isUnlocked(.fleetCloud) }` (was the default `{ false }`), so cloud rows are SELECTABLE only when `.fleetCloud` is unlocked — master ∧ sub-flag ∧ ai-commands. The media-video cloud gate (`MediaToolAvailability.isCloudEscalationEnabled`) routes through the same `isUnlocked(.fleetCloud)` resolver. LOCKED → cloud members stay disabled/captioned (the calm panic-off). `xcodebuild` GREEN.)** -- [x] 7.4 Any admission/escalation failure surfaces as a **bounded, non-blocking** row (clean headline via `AIError.message(for:)` + opt-in copyable details + Retry) — never an `NSAlert`, never raw error text in a headline (D7, blueprint invariant). *Verify: `xcodebuild` compile; **user run-verify** a forced `cannotAdmit` shows the bounded row, not a modal.* - -## 8. Provisioner wiring + verification - -- [x] 8.1 Wire `FleetRoster` descriptors into GemmaRuntime's `pipelineModel(for:)` so the new descriptor fields compile in the MLX path; no `ModelManager`/`ModelProvisioner` API change. *Verify: `xcodebuild` compile-verify (the metallib `*.bundle` copy in `build-app.sh` must NOT regress).* **(wire-compute-fleet: `makeModelManager` now passes `fleet: FleetRoster.standard` into the LIVE `ModelManager.init`, so `ensureResident`'s plan/evict path is reachable in the running app — previously `fleet` defaulted nil and the block was dead.)** -- [x] 8.2 Full Core suite green: enums, extended descriptor, registry, planner, error taxonomy, manager fleet-of-one + eviction + cloud no-op, gating. *Verify: `swift build` + `swift test` all pass.* -- [ ] 8.3 **User stable-signed build** validates the live behavior the agent cannot: real Q4-image co-residency with chat+KV under 48 GB, a Video/FP16-image admission actually evicting chat and reloading it, the live free-memory probe, and the on-disk/resident status of real weights in the roster. *Verify: user's `INSTALL=1 ./scripts/build-app.sh` run — agent does not build/sign the `.app`.* -- [x] 8.4 `openspec validate ai-model-fleet --strict` passes. *Verify: command exits 0.* diff --git a/openspec/changes/ai-skills-as-files/.openspec.yaml b/openspec/changes/ai-skills-as-files/.openspec.yaml deleted file mode 100644 index 38f7628..0000000 --- a/openspec/changes/ai-skills-as-files/.openspec.yaml +++ /dev/null @@ -1,2 +0,0 @@ -schema: spec-driven -created: 2026-06-22 diff --git a/openspec/changes/ai-skills-as-files/design.md b/openspec/changes/ai-skills-as-files/design.md deleted file mode 100644 index dfce2ef..0000000 --- a/openspec/changes/ai-skills-as-files/design.md +++ /dev/null @@ -1,179 +0,0 @@ -## Context - -The existing AI feature already encodes everything a "skill" is, just in code: - -- `AICommand` (`AI/AICommand.swift`) is a pure `Codable` value: `name`, `icon`, `tint`, `input: InputSource`, `promptTemplate`, `output: OutputTarget`, `model`, `confirmBeforeRun`, `runtimeParameter`, `reasoning`, with `requiredCapabilities` derived statically from the input source. -- `AICommandCatalog` (`AI/AICommandCatalog.swift`) is ~75 of those, built via a `preset(...)` factory, grouped into nine `Category`s (each with a `tint` and `sfSymbol`), with `commands(in:)`, `copy(of:)` (fresh id on add), and `seeded()` (a curated 8-command subset for fresh install). `AIBand.seeded()` calls straight through. -- `ParsedActions` (`AI/Tasks/ParsedActions.swift`) supplies one `StructuredSchema` per side-effecting `TaskKind` (calendar/reminder/contact/save-to-project/open-tool/send-to), each with an `applicable` decline affordance. -- `PromptTemplate.resolve(_:with:activeLanguage:)` resolves `{input}/{date}/{app}/{url}/{lang}` against a `FireContext`; unknown tokens pass through verbatim. - -V2 adds a router (`ai-tool-routing`) that picks a capability via `runtime.structured(routeSchema, as: ToolRoute.self)`. A capability the router can pick must describe itself: a `ToolDescriptor{name, summary, argsSchema, writePolicy}`. The router never wants to load 75 full prompt templates to decide; it wants a cheap **TOC of one-line summaries** and to load the chosen skill's body on demand. That is the same shape memory needs. So this slice externalizes the catalog into **declarative skill files** and owns the **shared retrieval index**. - -The constraint that shapes every decision below: the curated catalog and its Bands-editor browse/add/seed behavior are good product and must survive byte-for-byte. So skills are the new source of truth; `AICommandCatalog` becomes a projection. - -## Goals / Non-Goals - -**Goals:** -- A concrete on-disk **skill file format** (one file per action, named for the action) that externalizes `AICommand` + the `ParsedActions` sink binding + a router-facing summary, parseable into a pure `SkillManifest` (MLX-free Core). -- A **skills folder + authoring path**: built-in skills (read-only, bundled), user skills (writable Application Support folder), validation at load, malformed-skill surfaced bounded + non-blocking, user-shadows-built-in by id. -- The **shared `DocIndex`/`IndexedDoc`/`DocKind` retrieval contract** (blueprint 3.4) owned here, with progressive disclosure (TOC always, body on demand). Pure synchronous index; async file IO bridged by the store. -- The **skill → `ToolDescriptor` projection** + an invocation seam consumed by `ai-tool-routing`'s `ToolRegistry`. -- An **identity-preserving, idempotent migration**: the 75 presets become built-in skill files; `AICommandCatalog` projects over them so Bands-editor browse/add and `seeded()` are unchanged. - -**Non-Goals:** -- The route→execute→continue loop and the `ToolRegistry` aggregation — owned by `ai-tool-routing`; this slice only *provides* descriptors + an invoke seam. -- The memory store, its TOC/subfiles, and its read/write tools — `ai-agent-memory` *consumes* our `DocIndex`; we do not build it. -- Launching `claude` — `ai-claude-handoff` owns the process spawn and `ClaudeHandoffConfig`; we only carry the optional per-skill block on the file and surface it on the manifest. -- A Hub UI for editing a skill (v1 authoring is file-based, like dropping a `.md`). A skill marketplace / remote fetch. -- Changing `AICommand`, `PromptTemplate`, or `ParsedActions` behavior — they are reused verbatim. - -## Decisions - -### 1. Skill file format — front-matter + body, one file per action. - -A skill is a UTF-8 text file `.skill.md` (e.g. `create-meeting-in-gcal.skill.md`, `fix-grammar.skill.md`). It has a YAML front-matter block delimited by `---`, then a markdown body that IS the prompt template: - -``` ---- -id: fix-grammar -title: Fix Grammar -summary: Correct spelling and grammar in the selected text, returning only the fixed text. -keywords: [grammar, spelling, proofread, correct] -category: Writing # built-in only; preserves AICommandCatalog.Category for the projection -icon: text.badge.checkmark # ItemIcon.sfSymbol name -tint: "#40B866" # ItemColor; optional, defaults to the category tint -input: selection # InputSource raw value -output: replaceSelection # OutputTarget; see §2 for the sink encoding -confirmBeforeRun: false # optional; defaults from output.isSideEffecting (AICommand.init parity) -runtimeParameter: # optional; mirrors RuntimeParameter - language: { default: English } -reasoning: null # optional; AIReasoning .on/.off, null = follow global -tools: [] # optional allow-list of extra ToolDescriptor names this skill may call -claudeHandoff: null # optional ClaudeHandoffConfig block (owned by ai-claude-handoff) ---- -Fix the spelling and grammar of the following text. Return only the corrected text, with no commentary: - -{input} -``` - -- **`summary` is the router-facing when-to-use line** — the single most load-bearing addition over `AICommand`. It is the `IndexedDoc.summary` (the TOC line the router ranks) AND the `ToolDescriptor.summary` (the one line the model sees in route mode). The in-code catalog never had this; it is authored per skill. -- The **body is the prompt template** verbatim, resolved by the unchanged `PromptTemplate`. Keeping it as the body (not a quoted YAML string) keeps multi-line templates readable and editable. -- **Markdown (`.skill.md`) over JSON/TOML/plist** — chosen because (a) it is the human-authoring sweet spot (front-matter is a known idiom, the body is just text), (b) it mirrors the memory subfile format (`ai-agent-memory` also wants human-editable docs over the same index), and (c) it survives copy/paste and version control. *Rejected:* one JSON file per skill (machine-friendly but hostile to no-code authoring of a multi-line prompt); a single monolithic `skills.json` (loses one-file-per-action, harder to drop in / shadow / diff). - -### 2. `SkillManifest` (pure Core) — the parsed in-memory form, an `AICommand` superset. - -``` -struct SkillManifest: Equatable, Sendable, Identifiable { - var id: String // the file's `id`, path-relative, stable; the ToolDescriptor.name - var origin: SkillOrigin // .builtIn / .user (drives read-only + shadowing) - var title: String - var summary: String // router-facing when-to-use; the TOC line - var keywords: [String] - var category: String? // built-in projection grouping (nil for user skills) - var command: AICommand // the reused value model: icon/tint/input/template/output/param/reasoning - var toolNames: [String] // optional allow-list of extra tools this skill may invoke - var claudeHandoff: ClaudeHandoffConfig? // consumed type, owned by ai-claude-handoff - var updatedAt: Date -} -enum SkillOrigin: String, Codable, Sendable { case builtIn, user } -``` - -- The skill **reuses `AICommand`** rather than re-flattening its fields: `SkillFile` parse builds an `AICommand` (with `id = UUID()` minted fresh — the file `id` string is the *skill* identity, the `AICommand.id` is the per-instance identity when added to a band, exactly the `copy(of:)` stencil rule). So `requiredCapabilities`, `resolvedReasoning`, `defaultConfirmBeforeRun`, and the `runtimeParameter`/`{lang}` plumbing all come for free and stay consistent with band items. -- **Sink encoding in the file.** `OutputTarget` has associated values (`.runTask(TaskKind)`, `.sendTo(Destination)`). The front-matter encodes these as a small tagged form: `output: replaceSelection | pasteAtCursor | previewOnly`, or `output: { runTask: addToCalendar }`, `output: { runTask: { saveToProject: Inbox } }`, `output: { sendTo: { shortcut: "My Shortcut" } }`. `SkillFile` maps these to/from `OutputTarget` at the boundary. This is the "output schema / sink binding" externalized. - -### 3. The skill → `ToolDescriptor` projection (the skill↔router contract). - -A skill projects to a `ToolDescriptor` (blueprint 3.3, owned by `ai-tool-routing`): - -- `name` = `SkillManifest.id`. -- `summary` = `SkillManifest.summary`. -- `argsSchema` = the skill's parsed-action `StructuredSchema`: for a side-effecting sink (`.runTask`/`.sendTo`) it is the corresponding `ParsedActions` schema (`ParsedCalendarEvent.schema`, etc.) so the model emits the same validated/declinable shape the dispatcher already consumes; for an in-place sink (`replaceSelection`/`pasteAtCursor`/`previewOnly`) it is a minimal text schema (`{ "result": string }`) since the model just produces text. -- `writePolicy` = `.confirm` when `command.output.isSideEffecting` (the existing `confirmBeforeRun` default), else `.auto` — exactly mirroring `AICommand.defaultConfirmBeforeRun`. A skill may pin `confirmBeforeRun: true` to force `.confirm`. (`ai-background-autonomy` later intersects this with the user whitelist; this slice only sets the descriptor default — blueprint C1: the bare `WritePolicyTier` is defined with `ToolDescriptor` in `ai-tool-routing`, so we *read/set* it, we do not define it.) - -`SkillToolProvider` (Core) exposes `descriptors() -> [ToolDescriptor]` (the TOC-cheap projection) and `invoke(skillID:arguments:context:) -> ToolStepResult` which resolves the template via `PromptTemplate`, calls the runtime, and routes to the bound sink via `TaskDispatching`. `ai-tool-routing`'s `ToolRegistry` aggregates this provider alongside memory tools and `launch_claude`. **This slice does not run the loop**; it hands the registry a contributor. - -### 4. The shared index — `DocIndex`/`IndexedDoc`/`DocKind` (OWNED here, blueprint 3.4). - -``` -struct IndexedDoc: Codable, Equatable, Identifiable, Sendable { - let id: String; var title: String; var summary: String - var keywords: [String]; var kind: DocKind; var bodyPath: URL; var updatedAt: Date -} -enum DocKind: String, Codable, Sendable { case skill, memoryCore, memorySubfile } -protocol DocIndex: Sendable { - func allSummaries() -> [IndexedDoc] // the combined TOC the model always sees - func retrieve(query: String, limit: Int) -> [IndexedDoc] // ranked summaries - func body(of id: String) throws -> String // load full body on demand -} -``` - -- **Progressive disclosure.** `allSummaries()` returns every doc's `summary` line — cheap, always shown to the router. The full body (the prompt template for a skill, the subfile text for memory) is loaded only via `body(of:)` when the model selects that doc — a routed `retrieve`/`read` tool step. This bounds the route prompt to ~75 one-liners, not 75 full templates. -- **Pure synchronous over an in-memory snapshot; async IO bridged by the store** (the documented Files-band pattern: `FilesNavigationModel` pure/sync, `DirectoryLister` async/off-main, `FilesColumnController` bridges with a cache). `SkillStore` builds the `[IndexedDoc]` snapshot off-main (read each file's front-matter), then a pure `InMemoryDocIndex` answers `allSummaries`/`retrieve` synchronously; `body(of:)` reads the file (cached). The pure index never touches `FileManager`. -- **Ranking** (`retrieve`) is a cheap keyword/substring score over `title`+`summary`+`keywords` — deterministic, unit-testable, no embedding model (M5 could run one, but the router already does the heavy lifting via `structured()`; the TOC scan is a cheap pre-filter, and a deterministic ranker is testable without a model). The index is `kind`-agnostic, so the combined skills+memory TOC is one `allSummaries()` over a merged snapshot — `ai-agent-memory` contributes its `IndexedDoc`s into the same index; **memory defines no second retriever** (blueprint C2). -- `kind` distinguishes a skill doc (`.skill`) from a memory doc so a consumer (or the route prompt) can group/label them; the retriever ranks uniformly. - -### 5. `SkillStore` — built-in + user, validation, shadowing, watch. - -``` -final class SkillStore { // owns IO; bridges to the pure index - func loadAll() async -> SkillLoadResult // built-in bundle ∪ user folder, shadowed, validated - func index() -> DocIndex // the pure snapshot for the router - func manifest(id: String) -> SkillManifest? - // user-folder writes are out of band (user edits files); the store watches + reloads -} -struct SkillLoadResult { var skills: [SkillManifest]; var problems: [SkillProblem] } -struct SkillProblem: Equatable { var fileName: String; var headline: String } // bounded, non-blocking -``` - -- **Locations.** Built-in skills ship read-only inside the app bundle (`Contents/Resources/Skills/`, generated at build from the migration — §7). User skills live in a writable folder under Application Support (e.g. `~/Library/Application Support//Skills/`), created on first run. No new permission (reads the filesystem on demand, like `keepClipboardHistory` / the Files band). -- **Coexistence + shadowing.** The store loads built-in then user; a **user skill whose `id` matches a built-in shadows it** (the user's file wins; `origin: .user`). This is the no-code "edit a built-in" path: copy the built-in to the user folder, edit, done. Otherwise both appear, user skills after built-in. -- **Validation at load (never a crash, never a silent drop).** Each file is parsed; failures (missing front-matter, missing `id`/`summary`, unknown `input`/`output` enum value, malformed sink, bad `runtimeParameter`, template that fails to parse) are mapped at the parse boundary into a `SkillProblem{fileName, headline}` (a clean `AIPresentedError.headline`) and collected — the skill is excluded from the index but the load **succeeds for the rest**. Problems surface as a bounded, non-blocking list (a Hub row), never an `NSAlert`; raw parse text rides only in logs / opt-in details. -- **Watch + reload.** The user folder is watched (a coalesced reload like the Files-band cache); a dropped/edited file re-indexes off-main and republishes. Built-in skills are loaded once. - -### 6. Error taxonomy. - -A new `enum SkillError: Error, Equatable, LocalizedError` ONLY for skill-load/validation cases `RuntimeError`/`TaskError` cannot carry: `.malformedFrontMatter(detail:)`, `.missingRequiredField(name:)`, `.unknownEnumValue(field:value:)`, `.duplicateID(id:)`, `.unreadable(detail:)`. Each has a clean `errorDescription`; raw OS/parse text stays in opt-in `details`/logs. `AIError.message(for:)` is extended to translate `SkillError` into an `AIPresentedError` (the one translator). Skill **invocation** failures (template/model/sink) flow through the existing `RuntimeError`/`TaskError` and become a `ToolStepResult(status: .failed(headline:))` — never a false "Done", never silence. Vendor/OS errors (`FileManager`, YAML parse) map into `SkillError` at the `SkillFile`/`SkillStore` boundary; Core stays MLX-free. - -### 7. Catalog migration — built-in skill files + `AICommandCatalog` as a projection. - -- **Generation.** A deterministic generator iterates `AICommandCatalog.entries` and emits one built-in skill file per preset: `id` = a slug of the name (e.g. "Fix Grammar" → `fix-grammar`), `title`/`icon`/`tint`/`input`/`output`/`runtimeParameter` from the `AICommand`, `category` from the `Entry`, body = the `promptTemplate`. The `summary` (the new router line) is authored per preset (a short when-to-use sentence — the one piece not derivable from the existing command; the generator seeds a reasonable default from `name`+template and is hand-tuned). Generation runs as a build/dev step writing into `Contents/Resources/Skills/`; the generated files are checked in so the build is reproducible and `swift test` can read them from a known fixtures path. -- **`AICommandCatalog` becomes a projection.** After migration, `AICommandCatalog.entries`/`commands(in:)`/`seeded()` are computed from the loaded **built-in** skill set (filtered to `origin: .builtIn`, grouped by `category`), preserving: the nine categories and their `tint`/`sfSymbol`, the per-category command lists (catalog order), `copy(of:)` (fresh `AICommand.id` on add), and `seeded()` (the same curated 8 names in the same order). The Bands-editor browser, "add as a band", and the fresh-install seed are therefore **byte-identical** to today. -- **Idempotent + identity-preserving.** No persisted `Favorites`/band record is rewritten; an upgrading user's bands (already holding `.aiCommand` items with their own `AICommand.id`s) are untouched. `AIBand.seeded()`/`seededBand()` still call `AICommandCatalog.seeded()`, which now projects from skills — same names, same templates, same ids-are-fresh-on-add semantics. The fresh-install guard (don't re-seed an upgrading user) is unchanged. -- **Why a projection, not a deletion.** The Bands editor adds an `.aiCommand` band *item* (a persisted `AICommand`), not a live skill reference — that behavior is load-bearing and tested. Keeping `AICommandCatalog` as a projection means zero churn to the Bands editor / `AIBand` / `Favorites` while the *source of truth* moves to files. The router uses the skill files; the launcher grid keeps using `AICommand` band items. The two views never diverge because the catalog is computed from the same files. - -## Target split & verification (per component) - -| Component | Target | Verified by | -|---|---|---| -| `SkillManifest`, `SkillOrigin` (value types) | Core | `swift test` (round-trip, defaults parity with `AICommand`) | -| `SkillFile` parse/serialize (front-matter + body, sink encoding) | Core | `swift test` (parse fixtures, malformed → `SkillProblem`, round-trip stability) | -| `IndexedDoc`/`DocKind`/`DocIndex` + `InMemoryDocIndex` (the shared retriever) | Core | `swift test` (`allSummaries`, deterministic `retrieve` ranking, `body(of:)`) | -| `SkillStore` (built-in ∪ user, shadowing, validation, watch) | Core | `swift test` against temp dirs (shadowing, problem collection, reload) | -| `SkillToolProvider` (skill → `ToolDescriptor`, `invoke`) | Core | `swift test` against `StubLLMRuntime` (descriptor projection, invoke → `ToolStepResult`, decline path) | -| `SkillError` + `AIError.message(for:)` extension | Core | `swift test` (clean headline, no raw interpolation) | -| Migration generator + `AICommandCatalog` projection | Core | `swift test` (projection equals today's catalog: categories, per-category lists, `seeded()` names/order) | -| Generated built-in `.skill.md` corpus (checked-in resource) | Core fixture | `swift test` reads the bundled corpus; count + parse-clean assertion | -| Full app link (no MLX code added here) | app/GemmaRuntime | `xcodebuild` compile-verify only; **user** does the real install to author skills on disk end-to-end | - -No piece of this slice links MLX. Everything is pure value types + file IO + a stub-driven invoke, so the whole slice verifies under `swift build`/`swift test`. (House rule: an agent never builds/signs/installs the `.app`.) - -## Edge cases - -- **Two skills, same `id`, both user files** → `SkillError.duplicateID`; the first loaded wins, the second is a `SkillProblem` (deterministic order by filename). User-shadows-built-in is NOT a duplicate (different `origin`, intended). -- **Built-in skill the user deleted from the user folder** → built-in still loads (deletion of a *user* override just un-shadows; you cannot delete a built-in, only shadow it). A user wanting to "remove" a built-in shadows it with an empty/disabled file (a future `enabled: false` front-matter key — noted, not in v1). -- **Skill bound to a side-effecting sink but the model declines** (`applicable: false`) → `invoke` returns `ToolStepResult(status: .declined(reason:))`, the dispatcher fires nothing, exactly the existing decline path; never a false "Done." -- **Skill with a `{lang}` template but no `runtimeParameter`** → `{lang}` resolves to empty (the unchanged `PromptTemplate` rule); not an error. -- **Vision skill (`input: screenRegion`/`clipboardImage`)** → `requiredCapabilities == [.vision]` flows from the reused `AICommand`; the route/executor enforces a vision model exactly as today. -- **Malformed front-matter mid-corpus** → that one file is a `SkillProblem`; the rest of the corpus indexes; the router never sees the broken skill; the Hub shows one bounded problem row. -- **Empty user folder / folder absent** → store creates it; built-in skills alone form the index; no error. -- **`retrieve` query with no keyword hits** → returns an empty (or score-floored) list; the router falls back to the full `allSummaries()` TOC; never throws. -- **A user skill `id` that collides with a memory doc `id`** → `IndexedDoc.id` is namespaced by a path-relative id and `kind`; skills and memory live in different folders, so collision is structurally impossible (documented contract for `ai-agent-memory`). - -## Rejected alternatives - -- **Keep `AICommandCatalog` as the source of truth, add a separate router-only descriptor table.** Rejected: two sources of the same verbs drift; the whole point is one declarative unit the user can author and the router can read. -- **Per-skill JSON / single `skills.json`.** Rejected: hostile to no-code authoring of a multi-line prompt; loses one-file-per-action drop-in/shadow/diff. Markdown front-matter is the human sweet spot and mirrors memory subfiles. -- **An embedding-based retriever in this slice.** Rejected for v1: the `structured()` router is the real selector; the TOC scan is a cheap deterministic pre-filter (unit-testable without a model). An embedding ranker is an additive future behind the same `DocIndex` seam (M5 can run it). -- **A second retriever for memory.** Rejected by blueprint C2: skills and memory share ONE `DocIndex`; memory contributes `IndexedDoc`s, it does not define its own ranking/IO. -- **Defining `WritePolicyTier` here.** Rejected by blueprint C1: the bare enum is defined with `ToolDescriptor` in `ai-tool-routing`; we set it on the descriptor, we do not own it. -- **Deleting the Bands-editor `.aiCommand`-item flow in favor of live skill references.** Rejected: that flow is load-bearing and tested; the projection keeps it intact with zero churn while moving the source of truth to files. diff --git a/openspec/changes/ai-skills-as-files/proposal.md b/openspec/changes/ai-skills-as-files/proposal.md deleted file mode 100644 index 7dd8236..0000000 --- a/openspec/changes/ai-skills-as-files/proposal.md +++ /dev/null @@ -1,34 +0,0 @@ -## Why - -Today the AI feature ships ~75 ready-made verbs as a hardcoded Swift array (`AICommandCatalog`): each preset is an `AICommand` value (input source, prompt template, output target, optional runtime parameter), grouped into nine browsable `Category`s, curated into a seeded band. The catalog is excellent product — but it is *closed*. A user can edit a copy after adding it to a band, yet they cannot **author a new verb from nothing**, cannot **see what a verb is for at a glance in a way the model can read**, and the model itself has no way to *discover* which verb fits a request — every command is fired explicitly from the launcher grid. - -V2 turns the agent into a tool-using companion: the router (`ai-tool-routing`) asks the model "which capability does this request need?" and a capability is a `TaskKind`-shaped tool. For the verbs that the catalog already encodes (rewrite this, summarize that, extract a calendar event), the natural unit is **a skill: a declarative, user-authorable file** that externalizes exactly what an `AICommand` carries today — name, a router-facing *when-to-use* description, input source, prompt template, output/sink binding, optional runtime parameter, optional Claude-handoff config — plus the one thing the in-code catalog cannot express: a **table-of-contents summary** the router scans cheaply before loading the full skill. - -This is the same retrieval shape memory needs (`ai-agent-memory`): a TOC of summaries the model always sees, with the full body loaded on demand (progressive disclosure). So this slice **owns** the shared `DocIndex`/`IndexedDoc`/`DocKind` contract that memory consumes — one retriever, not two. - -The migration constraint is firm: the curated 75-preset catalog and its Bands-editor browse-and-add behavior are good and must survive. So the presets become **built-in skill files seeded on disk**, and the existing `AICommandCatalog` becomes a *thin projection* of the built-in skills (categories, tints, seed subset preserved) rather than the source of truth. - -## What Changes - -- **Skill file format (declarative, on-disk).** One file per skill, named for its action (e.g. `create-meeting-in-gcal.skill.md`), with YAML front-matter (`title`, `summary` = the router-facing one-line *when-to-use*, `keywords`, `input`, `output`/sink binding, optional `runtimeParameter`, optional `tools` allow-list, optional `claudeHandoff` block) and a markdown body holding the prompt template. This is `AICommand` + the `ParsedActions` sink binding + a router summary, externalized. A pure `SkillManifest` value type (MLX-free Core) is the parsed in-memory form. -- **Skills folder + authoring.** Built-in skills ship read-only inside the app bundle; user skills live in a writable `Skills/` folder under Application Support. A user adds/edits a skill by dropping or editing a text file (no code, no rebuild); the store validates each file at load (front-matter present, input/output enum values known, template parses) and surfaces a malformed skill as a bounded, non-blocking row — never a crash, never a silent drop. Built-in and user skills coexist; a user skill with the same id **shadows** the built-in (the override mechanism). -- **The index (shared retrieval).** This slice OWNS `DocIndex`/`IndexedDoc`/`DocKind` (blueprint 3.4): a cheap in-memory manifest of `IndexedDoc(kind: .skill)` summaries the router scans, with the full skill body loaded on demand via `body(of:)`. Pure synchronous index over an in-memory snapshot; file IO is bridged off-main by the owning store (the Files-band sync-model + async-cache pattern). `ai-agent-memory` consumes this exact contract for its TOC + subfiles — it does NOT define a second retriever. -- **Skill → tool contract.** Each skill projects to a `ToolDescriptor` (blueprint 3.3): `name` = skill id, `summary` = the TOC line, `argsSchema` = the skill's parsed-action schema (reusing `ParsedActions`' `StructuredSchema` for sink-bound skills; a minimal `{input}` schema for in-place skills), `writePolicy` from the sink (side-effecting sinks → `.confirm`, in-place → `.auto`). Invoking a skill runs its prompt template through the model and routes the result to its bound sink via `TaskDispatching`. -- **Catalog migration (identity-preserving, idempotent).** The ~75 presets are emitted as built-in skill files (a deterministic generator keyed off the existing `AICommandCatalog.entries`, preserving category, tint, name, template, input, output, runtimeParameter). `AICommandCatalog` becomes a projection over the built-in skill set so the Bands-editor browser, the per-category "add as a band", and `seeded()` all behave **byte-identically** to today. No persisted band is rewritten; an upgrading user keeps their bands untouched. - -## Capabilities - -### New Capabilities - -- `ai-skill-index`: the shared declarative-document retrieval seam — `DocIndex`/`IndexedDoc`/`DocKind`, the TOC-of-summaries + on-demand body load (progressive disclosure), pure synchronous over an in-memory snapshot with off-main file IO bridged by the owning store. Owned here; consumed by `ai-agent-memory`. - -### Modified Capabilities - -- `ai-command-catalog`: the curated preset corpus is externalized into **built-in skill files**; the in-code `AICommandCatalog` becomes a projection over them (categories, tints, seed subset, Bands-editor browse/add preserved). Adds the **skill file format**, the **skills folder + authoring/validation**, **built-in vs user coexistence + shadowing**, and the **skill → `ToolDescriptor` projection**. - -## Impact - -- **Code (MLX-free Core, `AI/Skills/`):** new `SkillManifest` value type + `SkillFile` parse/serialize (front-matter + body), `SkillStore` (load built-in bundle + user folder, validate, shadow, watch), `DocIndex`/`IndexedDoc`/`DocKind` (`AI/Skills/DocIndex.swift`, the shared retriever), `SkillToolProvider` (skill → `ToolDescriptor` + invocation seam consumed by `ai-tool-routing`'s `ToolRegistry`), and the migration generator that emits built-in skill files from `AICommandCatalog.entries`. A new `SkillError: Error, Equatable, LocalizedError` only if `RuntimeError`/`TaskError` cannot carry skill-load/validation failures. -- **Reuse, not rebuild:** `AICommand`/`InputSource`/`OutputTarget`/`TaskKind`/`Destination`/`RuntimeParameter` (the skill file IS these, serialized), `ParsedActions` `StructuredSchema`s (the skill's `argsSchema`), `PromptTemplate` (token resolution unchanged), `TaskDispatching` (sink execution unchanged), `ClaudeHandoffConfig` (consumed as an optional skill block, owned by `ai-claude-handoff`), `ToolDescriptor` (consumed, owned by `ai-tool-routing`). -- **MLX-free Core:** every piece here is pure value types + file IO; verified under `swift build` / `swift test`. No GemmaRuntime/MLX code in this slice. The full app links via `xcodebuild` compile-verify only; the user does the real install for end-to-end run-verify of authoring on disk. -- **Out of scope:** the route loop and `ToolRegistry` aggregation (`ai-tool-routing`); the memory store and its tools (`ai-agent-memory` consumes our index); the launching of `claude` (`ai-claude-handoff`); any UI for editing a skill inside the Hub (v1 authoring is file-based); a marketplace / remote skill fetch. diff --git a/openspec/changes/ai-skills-as-files/specs/ai-command-catalog/spec.md b/openspec/changes/ai-skills-as-files/specs/ai-command-catalog/spec.md deleted file mode 100644 index 70265f8..0000000 --- a/openspec/changes/ai-skills-as-files/specs/ai-command-catalog/spec.md +++ /dev/null @@ -1,78 +0,0 @@ -## ADDED Requirements - -### Requirement: Skills are declarative, user-authorable files -Each AI verb SHALL be expressible as a **skill file**: a declarative, human-editable text file (one file per action, named for the action, e.g. `create-meeting-in-gcal.skill.md`) that externalizes everything a command carries — a stable **id**, a **title**, a router-facing **summary** (a one-line *when-to-use* description), **keywords**, an **input source**, a **prompt template**, an **output/sink binding**, and optional **runtime parameter**, **reasoning override**, **tool allow-list**, and **Claude-handoff** block. A skill file SHALL parse into a pure value model (a `SkillManifest`) that reuses the existing command value model for input/template/output/parameter semantics, so a skill behaves identically to an equivalent command when fired. A skill file SHALL require **no code change and no rebuild** to add or edit. - -#### Scenario: A skill file declares a complete verb -- **WHEN** a skill file is parsed -- **THEN** it yields a manifest carrying an id, title, router summary, input source, prompt template, and output/sink binding sufficient to fire without further editing - -#### Scenario: Adding a skill needs no code -- **WHEN** a user adds a valid skill file to the user skills folder -- **THEN** the skill becomes available without recompiling or reinstalling the app - -#### Scenario: A skill's prompt template resolves identically to a command -- **WHEN** a skill's prompt template containing `{input}`, `{date}`, `{app}`, `{url}`, or `{lang}` is resolved against a fire context -- **THEN** the resolution is identical to the existing command template resolution (unknown tokens pass through, a missing language resolves to empty) - -### Requirement: Skills folder, validation, and built-in / user coexistence -The system SHALL load skills from two locations: **built-in skills** shipped read-only inside the app bundle, and **user skills** in a writable folder under Application Support (created on first run). Each file SHALL be **validated at load**; a malformed skill (missing front-matter or a required field, an unknown input/output value, a malformed sink, an unparseable template) SHALL be reported as a **bounded, non-blocking problem** (a clean headline) and excluded from the index while the rest of the corpus loads — never a crash and never a silent drop. Built-in and user skills SHALL coexist; a **user skill whose id matches a built-in SHALL shadow** the built-in (the user file wins), giving a no-code path to override a shipped skill. Loading SHALL require **no new permission**. - -#### Scenario: Built-in and user skills both load -- **WHEN** the skill store loads -- **THEN** the built-in skills and any user skills are both present in the index - -#### Scenario: A user skill shadows a built-in of the same id -- **WHEN** a user skill file declares the same id as a built-in skill -- **THEN** the user skill replaces the built-in (its body and origin are used) and the built-in is not also listed - -#### Scenario: A malformed skill is reported, not fatal -- **WHEN** one skill file is malformed -- **THEN** that file is reported as a single bounded problem (clean headline, raw detail only in logs/opt-in details), every other skill still loads, and the app does not crash or block - -#### Scenario: Two user skills share an id -- **WHEN** two user skill files declare the same id -- **THEN** one wins deterministically and the other is reported as a duplicate-id problem - -### Requirement: A skill projects to a routing tool descriptor -Each skill SHALL project to a **tool descriptor** the router can scan: the descriptor's name SHALL be the skill id, its summary SHALL be the skill's router-facing one-line description, its arguments schema SHALL be the skill's parsed-action schema (the existing per-task structured schema for a side-effecting sink, or a minimal text-result schema for an in-place sink), and its **write policy** SHALL default to **confirm** when the skill's output is side-effecting (and otherwise **auto**), mirroring the existing confirm-before-run default. Invoking a skill SHALL resolve its template, call the model, and route the result to the skill's bound sink via the existing task dispatch, producing an observable step outcome (done / declined / failed). A declined or failed invocation SHALL NOT report a false success. - -#### Scenario: A side-effecting skill projects a confirm-tier descriptor with its task schema -- **WHEN** a skill bound to a side-effecting sink (e.g. add-to-calendar) is projected -- **THEN** its descriptor carries the corresponding parsed-action schema and a confirm write policy - -#### Scenario: An in-place skill projects an auto-tier descriptor -- **WHEN** a skill bound to an in-place sink (e.g. replace-selection) is projected -- **THEN** its descriptor carries a text-result schema and an auto write policy - -#### Scenario: An invoked skill that the model declines reports a decline, not a done -- **WHEN** a side-effecting skill is invoked on input the model declines as not applicable -- **THEN** the step outcome is a decline with a reason, no side effect is dispatched, and no false success is reported - -## MODIFIED Requirements - -### Requirement: Categorized AI command catalog -The system SHALL provide a curated **catalog** of ready-made command presets, each a complete, fireable command (name, icon, tint, input source, prompt template, output target, and — where applicable — a runtime parameter). Every preset SHALL belong to exactly one **category**, and the catalog SHALL cover at least these categories: **Writing**, **Tone**, **Understand**, **Translate**, **Developer**, **Reply**, **Capture** (side-effecting tasks), **Vision** (screen-region), and **Format**. The catalog SHALL be **derived from the built-in skill files** rather than a separately maintained in-code array: the in-code catalog is a **projection** over the built-in skills (grouping them by category, preserving each category's tint and section glyph and the per-category command order), so the skill files are the single source of truth and the catalog and the router never diverge. The projected catalog SHALL remain the single source of the presets used both by the Bands-editor browser and by the fresh-install seed. - -#### Scenario: Catalog spans the named categories -- **WHEN** the catalog is enumerated -- **THEN** it contains presets grouped under Writing, Tone, Understand, Translate, Developer, Reply, Capture, Vision, and Format, and every preset declares its category - -#### Scenario: Each preset is a complete, fireable command -- **WHEN** any catalog preset is inspected -- **THEN** it carries a name, icon, input source, prompt template, and output target sufficient to fire without further editing - -#### Scenario: The catalog is a projection over the built-in skills -- **WHEN** the catalog is enumerated -- **THEN** every preset corresponds to a built-in skill file, grouped by its category, with the category's tint and section glyph preserved — and editing a built-in skill file changes the corresponding catalog preset - -### Requirement: Fresh-install seed is drawn from the catalog -The fresh-install "AI" band SHALL be composed from a **curated subset of the catalog** (which is itself projected from the built-in skills) rather than a separate hand-maintained list, so the seeded defaults, the browsable catalog, and the skill files all stay consistent. The seed SHALL remain a single, curated band (not the entire catalog), SHALL preserve the existing curated subset and its order, and SHALL only apply on a fresh install (the existing migration/idempotency guard is unchanged — an upgrading user is not re-seeded and no persisted band is rewritten by the move to skill files). - -#### Scenario: Fresh install seeds a curated catalog subset -- **WHEN** the app is first installed with no prior AI commands -- **THEN** the seeded "AI" band's commands are drawn from the catalog (projected from the built-in skills) and form one curated band, not the whole catalog - -#### Scenario: Upgrading users are not re-seeded or rewritten -- **WHEN** an existing user who already has AI commands upgrades to the skill-files build -- **THEN** their bands are left untouched, no persisted band item is rewritten, and the grown seed is not applied diff --git a/openspec/changes/ai-skills-as-files/specs/ai-skill-index/spec.md b/openspec/changes/ai-skills-as-files/specs/ai-skill-index/spec.md deleted file mode 100644 index f8678c0..0000000 --- a/openspec/changes/ai-skills-as-files/specs/ai-skill-index/spec.md +++ /dev/null @@ -1,38 +0,0 @@ -## ADDED Requirements - -### Requirement: Shared declarative-document index -The system SHALL provide a single **declarative-document index** seam (`DocIndex` over `IndexedDoc`) shared by skills and memory, so there is exactly **one** retriever and not two. An `IndexedDoc` SHALL carry a stable path-relative **id**, a **title**, a **summary** (its table-of-contents line), **keywords**, a **kind** (skill / memory-core / memory-subfile), a **body location** loaded lazily, and an **updated-at** timestamp. The index SHALL be **kind-agnostic** when ranking, so a combined skills-and-memory table of contents is one enumeration. The skills capability OWNS this seam; the memory capability SHALL CONSUME it and SHALL NOT define a second retriever. - -#### Scenario: Skills and memory share one index -- **WHEN** both skill documents and memory documents are present -- **THEN** they appear in one combined index enumeration, each tagged with its kind, ranked by one retriever - -#### Scenario: A document carries a table-of-contents summary -- **WHEN** an indexed document is inspected -- **THEN** it exposes a one-line summary, keywords, a kind, and a lazily-loadable body location - -### Requirement: Progressive disclosure (table of contents always, body on demand) -The index SHALL expose a cheap **table of contents** — every document's summary line — that the router always sees, and SHALL load a document's **full body only on demand** when that document is selected. Selecting and reading a body SHALL be a **routed tool step** (a retrieve/read tool), not an eager bulk load, so the routing prompt is bounded to the one-line summaries regardless of corpus size. The retriever SHALL also offer a **ranked query** that returns the best-matching summaries for on-demand expansion. Ranking SHALL be deterministic. - -#### Scenario: The router sees summaries, not full bodies -- **WHEN** the router scans available capabilities -- **THEN** it receives the per-document summary lines (the table of contents), not the full document bodies - -#### Scenario: A body is loaded only when selected -- **WHEN** a document is selected via the retrieve/read tool -- **THEN** its full body is loaded at that point, not before - -#### Scenario: A query returns ranked summaries deterministically -- **WHEN** a query is run against the index -- **THEN** it returns the best-matching document summaries in a deterministic order, and a query with no matches returns an empty (or score-floored) list without error - -### Requirement: Pure synchronous index over an off-main snapshot -The retriever SHALL be a **pure, synchronous** state machine over an in-memory snapshot of indexed documents, and SHALL NOT perform file IO for enumeration or ranking. Building the snapshot and loading document bodies SHALL be bridged **off-main** by the owning store (the synchronous-model-with-async-cache pattern), so the pure index is unit-testable headless. This mirrors the existing column-navigator pattern where a pure model reads a cache that an async lister populates. - -#### Scenario: Enumeration and ranking never block on IO -- **WHEN** the table of contents is enumerated or a query is ranked -- **THEN** the operation is synchronous over the in-memory snapshot and performs no file IO - -#### Scenario: Snapshot building and body loads are off-main -- **WHEN** the index snapshot is built or a body is loaded -- **THEN** the file IO is performed off the main thread by the owning store and the result is published back, not done inline in the pure index diff --git a/openspec/changes/ai-skills-as-files/tasks.md b/openspec/changes/ai-skills-as-files/tasks.md deleted file mode 100644 index e698d0d..0000000 --- a/openspec/changes/ai-skills-as-files/tasks.md +++ /dev/null @@ -1,47 +0,0 @@ -> Wave 3 slice. Depends on `ai-tool-routing` (`ToolDescriptor`/`ToolRoute`/`ToolStepResult`/`ToolStepStatus`, the bare `WritePolicyTier`) and consumes `ClaudeHandoffConfig` from `ai-claude-handoff`. OWNS the shared `DocIndex`/`IndexedDoc`/`DocKind` that `ai-agent-memory` consumes. All pieces are MLX-free Core — verified under `swift build`/`swift test`; the full app links via `xcodebuild` compile-verify only (an agent never builds/signs/installs the `.app`). - -## 1. Skill value model (pure Core) - -- [x] 1.1 Add `SkillManifest` + `SkillOrigin` (`AI/Skills/SkillManifest.swift`): `id`, `origin`, `title`, `summary`, `keywords`, `category?`, `command: AICommand`, `toolNames`, `claudeHandoff: ClaudeHandoffConfig?`, `updatedAt`. Reuse `AICommand` verbatim (no field re-flattening). **Verify:** `swift test` — manifest holds an `AICommand` whose `requiredCapabilities`/`resolvedReasoning`/`defaultConfirmBeforeRun` match a directly-built command. -- [x] 1.2 Define the **sink encoding** mapping (`OutputTarget` ⇄ front-matter tagged form): `replaceSelection`/`pasteAtCursor`/`previewOnly` and `{runTask: …}`/`{sendTo: …}` incl. `saveToProject`/`openToolWithPayload`/`shortcut`/`urlScheme`/`shell`. **Verify:** `swift test` — every `OutputTarget` case round-trips through the encoding. - -## 2. Skill file parse / serialize (pure Core) - -- [x] 2.1 Add `SkillFile` (`AI/Skills/SkillFile.swift`): parse a `.skill.md` (YAML front-matter delimited by `---` + markdown body = prompt template) into a `SkillManifest`; serialize back. Map every field (id/title/summary/keywords/category/icon/tint/input/output/confirmBeforeRun/runtimeParameter/reasoning/tools/claudeHandoff). **Verify:** `swift test` — parse a fixture, serialize, re-parse, assert stable (round-trip). -- [x] 2.2 Map parse failures at the boundary into `SkillProblem{fileName, headline}` (clean headline) — missing front-matter, missing `id`/`summary`, unknown `input`/`output` enum, malformed sink, bad `runtimeParameter`, unparseable template. Never throw past the store; raw text → logs/details only. **Verify:** `swift test` — each malformed fixture yields exactly one `SkillProblem`, no crash. - -## 3. The shared index (pure Core; OWNED here) - -- [x] 3.1 Add `IndexedDoc`/`DocKind`/`DocIndex` exactly per blueprint 3.4 (`AI/Skills/DocIndex.swift`). `kind`-agnostic protocol; `IndexedDoc(kind: .skill/.memoryCore/.memorySubfile)`. **Verify:** `swift test` — type round-trips `Codable`; doc-string cites blueprint ownership. -- [x] 3.2 Add `InMemoryDocIndex` (pure, synchronous over an injected `[IndexedDoc]` snapshot): `allSummaries()`, deterministic `retrieve(query:limit:)` (cheap keyword/substring score over title+summary+keywords, stable tiebreak), `body(of:)` reading the doc's `bodyPath` (cached). Never touches `FileManager` for ranking. **Verify:** `swift test` — `allSummaries` returns the TOC; `retrieve` ranks deterministically; empty-hit query returns floored/empty (never throws); `body(of:)` loads the right file. - -## 4. Skill store (Core; IO bridged off-main) - -- [x] 4.1 Add `SkillStore` (`AI/Skills/SkillStore.swift`): `loadAll() async -> SkillLoadResult` loading built-in bundle (`Contents/Resources/Skills/`) ∪ user folder (Application Support `Skills/`, created on first run), off-main; build the `[IndexedDoc]` snapshot; expose `index() -> DocIndex` (pure) + `manifest(id:)`. Files-band sync-model + async-cache pattern; pure index never does IO. **Verify:** `swift test` against temp dirs — built-in + user skills both appear in `allSummaries`. *(DEVIATION kept: built-ins are PROJECTED in-memory from `AICommandCatalog` (`builtInManifests()`), not loaded from a bundle — the catalog stays the byte-for-byte source of truth; `testBuiltInProjectionMatchesCatalog` covers count + categories + reused `AICommand`.)* -- [x] 4.2 **Shadowing:** a user skill whose `id` matches a built-in shadows it (`origin: .user`, user file wins); otherwise both appear (built-in first). NOT a duplicate. **Verify:** `swift test` — a user override replaces the built-in body in `body(of:)` and flips `origin`. -- [x] 4.3 **Validation + problem collection:** malformed files become `SkillProblem`s; the load **succeeds for the rest**; `duplicateID` for two same-`id` user files (deterministic winner by filename). **Verify:** `swift test` — a corpus with one broken file indexes the rest and reports one problem. -- [x] 4.4 **Watch + coalesced reload** of the user folder (re-index off-main, republish); built-in loaded once. **Verify:** `swift test` — simulating a folder change triggers a reload that picks up a new/edited file. (Live FS-event wiring is `xcodebuild` compile-verify + user run-verify.) *(DONE: `SkillFolderWatcher` (`AI/Skills/SkillFolderWatcher.swift`) wraps a `DispatchSource.makeFileSystemObjectSource` on the user-folder fd (`[.write, .delete, .rename, .extend, .attrib]`); rapid events COALESCE through the pure, unit-tested `ReloadCoalescer` (a settle-window gate) into a single off-main `store.loadAll()` that republishes the fresh `SkillLoadResult` on the main actor. Built-ins are projected in-memory (loaded once); only the user folder is watched. `loadAll()` is idempotent — a re-call picks up new/edited/removed files. `testReloadCoalescerCollapsesABurst` pins the coalescing decision; `testWatcherReloadPicksUpANewFileAndRepublishes` drives the exact coalesced-reload path via `triggerReloadForTesting()` and asserts a dropped file republishes + resolves via `manifest(id:)`. The live FS-event firing is the only remaining user run-verify, folded into 8.4.)* - -## 5. Skill → tool contract (Core; consumed by ai-tool-routing) - -- [x] 5.1 Add `SkillToolProvider` (`AI/Skills/SkillToolProvider.swift`): `descriptors() -> [ToolDescriptor]` projecting each skill — `name`=id, `summary`=summary, `argsSchema`= the `ParsedActions` schema for a side-effecting sink / a minimal `{result: string}` schema for an in-place sink, `writePolicy`= `.confirm` when `command.output.isSideEffecting` (or `confirmBeforeRun`) else `.auto`. **Verify:** `swift test` — a calendar skill projects the `ParsedCalendarEvent.schema` + `.confirm`; a grammar skill projects the text schema + `.auto`. -- [x] 5.2 Add `invoke(skillID:arguments:context:) async -> ToolStepResult`: resolve the template via the unchanged `PromptTemplate`, call the runtime, route to the bound sink via `TaskDispatching`; map a decline to `.declined(reason:)`, a failed side effect to `.failed(headline:)`, success to `.done` with a short summary. **Verify:** `swift test` against `StubLLMRuntime` — in-place skill → `.done`; side-effecting decline → `.declined`; sink failure → `.failed` (never a false done). *(Realized as the `ToolContributor.run(_:gate:)` registry-aggregated entry point rather than a free `invoke(skillID:)` — same resolve→runtime→sink mapping; all three outcome cases covered by `testRunInPlaceSkillReturnsDone` / `testRunSideEffectingDeclineReturnsDeclined` / `testRunSideEffectingSinkFailureReturnsFailedNotFalseDone`.)* -- [x] 5.3 Document the registry seam: `SkillToolProvider` is a contributor `ai-tool-routing`'s `ToolRegistry` aggregates (this slice provides, the loop is owned there). **Verify:** `swift test` — `descriptors()` is pure and stable; no loop logic added here. - -## 6. Errors (Core) - -- [x] 6.1 Add `SkillError: Error, Equatable, LocalizedError` (only cases `RuntimeError`/`TaskError` cannot carry): `.malformedFrontMatter`, `.missingRequiredField`, `.unknownEnumValue`, `.duplicateID`, `.unreadable`; clean `errorDescription` per case; raw text → details/logs only. **Verify:** `swift test` — each case has a clean headline, no reflected enum dump. -- [x] 6.2 Extend `AIError.message(for:)` to translate `SkillError` → `AIPresentedError` (the one translator). **Verify:** `swift test` — `SkillError` headlines are bounded and contain no raw interpolation. - -## 7. Catalog migration (Core) - -- [x] 7.1 Add the **generator** that emits one built-in `.skill.md` per `AICommandCatalog.entries` preset (slug `id`, all fields from the `AICommand` + `Entry.category`, body = template, authored `summary`). Run it as a dev/build step; **check in** the generated corpus under `Contents/Resources/Skills/`. **Verify:** `swift test` reads the bundled corpus; assert preset count == generated file count and all parse clean. *(SUPERSEDED by the kept DEVIATION: no on-disk generated corpus / build step — `SkillStore.builtInManifests()` projects one `SkillManifest` per `AICommandCatalog.entries` preset in-memory (slug id via `SkillStore.slug`, all `AICommand` fields + `Entry.category`, body = template, `derivedSummary`). `testBuiltInProjectionMatchesCatalog` asserts count == `AICommandCatalog.entries.count` and categories preserved.)* -- [x] 7.2 Rewrite `AICommandCatalog` as a **projection** over the loaded **built-in** skill set: `entries`/`commands(in:)` grouped by `category` preserving order, `tint`/`sfSymbol` per category, `copy(of:)` (fresh `AICommand.id`), `seeded()` (same 8 names, same order). **Verify:** `swift test` — the projected catalog equals today's catalog (category set, per-category command names/order, `seeded()` names+order, `copy(of:)` mints a fresh id). *(DEVIATION: the projection direction is INVERTED — `AICommandCatalog` is left BYTE-FOR-BYTE unchanged as the source of truth and `SkillStore` projects skills FROM it (lower-risk than rewriting the heavily-tested catalog). The verify holds trivially: the catalog is unchanged, and `testCatalogProjectionPreservesSeededNamesAndOrder` plus the unchanged `AICommandCatalogTests` pin names/order/`seeded()`/`copy(of:)`.)* -- [x] 7.3 Confirm idempotency / identity preservation: no `Favorites`/band record rewritten; `AIBand.seeded()`/`seededBand()` unchanged in behavior; the fresh-install re-seed guard untouched. **Verify:** `swift test` — `AIBand.seededBand()` produces the same band shape as before the change. - -## 8. Verify - -- [x] 8.1 `swift build` + `swift test` green; the pure model (1.x), parse (2.x), index (3.x), store (4.x), provider (5.x), errors (6.x), and migration projection (7.x) covered. No MLX code added — the whole slice is Core. -- [x] 8.2 The full `ThreeFingerSwitcher` product **compiles + links** (Core + GemmaRuntime/MLX) via `xcodebuild` compile-verify only (no sign/install). -- [x] 8.3 `openspec validate --strict` passes; the `ai-command-catalog` delta + the new `ai-skill-index` spec match the implementation, and the `DocIndex` contract reads identically to what `ai-agent-memory` consumes. -- [ ] 8.4 **User run-verify** in a stable-signed build: dropping a `.skill.md` into the user `Skills/` folder makes it appear (no rebuild); editing a built-in via a same-`id` user file shadows it; a malformed file shows one bounded problem row and never crashes; the existing Bands-editor browse/add/seed behavior is unchanged. diff --git a/openspec/changes/ai-tool-routing/.openspec.yaml b/openspec/changes/ai-tool-routing/.openspec.yaml deleted file mode 100644 index 38f7628..0000000 --- a/openspec/changes/ai-tool-routing/.openspec.yaml +++ /dev/null @@ -1,2 +0,0 @@ -schema: spec-driven -created: 2026-06-22 diff --git a/openspec/changes/ai-tool-routing/design.md b/openspec/changes/ai-tool-routing/design.md deleted file mode 100644 index 2fcf0b9..0000000 --- a/openspec/changes/ai-tool-routing/design.md +++ /dev/null @@ -1,244 +0,0 @@ -## Context - -The agentic task layer already exists and is the thing V2 inverts control over. Read these before the design: - -- **`AI/LLMRuntime.swift`** — `structured(_:schema:as:) -> StructuredOutcome` is the seam this slice repurposes as the router. Its contract is exactly what we need: VALIDATE against a `StructuredSchema`, REPAIR/RETRY within a bounded loop, decode into `T`, and allow a first-class `.declined`. It throws `RuntimeError.couldNotProduceValid(attempts:)` only when the bounded loop is exhausted. `Token`/`TokenChannel.thinking` already splits reasoning from the committed answer. `RuntimeError` is the shared, `Equatable`, `LocalizedError` taxonomy. -- **`AI/Tasks/TaskDispatching.swift` + `TaskDispatcher.swift`** — `prepare(_ kind: TaskKind, resolvedPrompt: String, source: TaskSource, reasoning:) async -> TaskReview` and `execute(_ review:) async throws`. `prepare` runs `runtime.structured(...)` against the **kind's own** `ParsedActions` schema and maps a typed/affordance decline → `.declined`, exhausted validation → `.unavailable`, success → `.action(title:fields:payload:)`. `execute` fires the side effect for a confirmed `.action` ONLY. **Critical:** `prepare` takes a `resolvedPrompt: String`, NOT an arbitrary args object — the kind selects the schema; the prompt is the model's input. This is the bridge point (Decision 4). -- **`AI/Tasks/TaskReview.swift` / `ParsedActions.swift` / `TaskSinks.swift`** — `TaskReview` (`.action`/`.declined`/`.unavailable`), `PreparedAction` (the opaque validated payload), `ReviewField`, `TaskSource`, `TaskError` (`LocalizedError`, the side-effect taxonomy), and the small injectable sinks. All MLX-free, all unit-tested headless. -- **`AI/AICommandExecutor.swift`** — the existing fire→stream→commit state machine. NOT modified here (the conversational extension is `ai-conversational-canvas`). This slice's loop is observable state the canvas binds to. -- **`AI/AICommand.swift`** — `TaskKind` (calendar/reminder/contact/saveToProject/openToolWithPayload/sendTo) and `Destination`. - -The shared blueprint (`docs/ai-agent-v2-blueprint.md` §3.3) pins the route contract; this slice OWNS those types. Integration fix **C1** puts the bare `WritePolicyTier` enum HERE (with `ToolDescriptor`), so every descriptor carries its tier with no DAG back-edge to `ai-background-autonomy`. - -**Dependency note (honest):** the sibling change dirs `openspec/changes/ai-conversation-runtime/` and `.../on-device-ai-runtime/` are NOT yet on disk in this repo. This slice consumes `AgentMessage`/`AgentConversation`/`AgentSessionID`/`LLMChatRequest`/`chat()` from the blueprint's §3.1/§3.2 **sketches**. Where a type I consume is not yet committed, the design declares a narrow Core protocol seam I own (`ConversationContext`, `ChatStreaming`) so this slice compiles and `swift test`-passes in isolation, and binds to the real types when the owner slice lands. See Open Questions. - -## Goals / Non-Goals - -**Goals:** -- Invert who-drives: the **model** picks the tool via a `structured()` route turn against a fixed `RouteSchema` — the reliability mechanism is `structured()`'s repair/retry/decline, NOT Gemma native function-call tokens. -- "No tool, just talk" (`tool == ""` / `.declined`) is **first-class**, the common case, never a failure. -- Reuse `TaskReview`/`PreparedAction`/`ParsedActions`/`TaskSinks`/`TaskDispatcher` **wholesale**; the model only chooses the `TaskKind` (+ its stored config) the user used to choose. -- A **bounded** agent loop: hard step cap, live plan via `.thinking`, per-step approval (DOWN=approve / RIGHT=skip), and no-progress/loop-guard termination. -- **Candidate retrieval:** ~3–5 candidate tools per route turn, never the full registry; the shape is shared with skills + memory (`DocIndex`, blueprint §3.4). -- Entire slice MLX-free Core, verified by `swift test` with scripted `StubLLMRuntime` structured outcomes. - -**Non-Goals:** -- Conversation/message types + compaction (`ai-conversation-runtime` OWNS §3.1; consumed here). -- Canvas state cases + gesture interpretation + rendering (`ai-conversational-canvas` OWNS; this slice exposes observable loop state + the approval contract, it does not draw). -- The real batched MLX runtime (`ai-batched-runtime-and-context`); this loop drives an injected `LLMRuntime` and is agnostic to which conformer it is. -- The `DocIndex` retriever implementation (`ai-skills-as-files` OWNS; consumed here behind the candidate seam). -- The whitelist, the append-only audit log, and `.needsYou` escalation (`ai-background-autonomy` OWNS; consumed via `WritePolicyResolving`, with a stand-alone default). -- `launch_claude` / `ClaudeHandoffConfig` (`ai-claude-handoff`); it arrives as one more `ToolContributor` with no loop change. -- Native Gemma tool-call token parsing — explicitly rejected (Decision 1, Rejected Alternatives). - -## Decisions - -### 1. The route turn IS a `structured()` call against a fixed `RouteSchema` (NOT native function-call tokens) -`RouteSchema` is a single `StructuredSchema` (a `static let`, mirroring `ParsedCalendarEvent.schema`) describing the decision shape: - -```jsonc -{ - "type": "object", - "required": ["tool"], - "properties": { - "tool": { "type": "string", "description": "the chosen tool name, or \"\" to answer directly" }, - "argumentsJSON": { "type": "string", "description": "JSON object of arguments for the chosen tool" }, - "rationale": { "type": "string", "description": "one short sentence: why this choice" } - } -} -``` - -`ToolRouter.route(...)` builds a route prompt — the conversation tail + the **candidate** tool descriptors (`name`, `summary`, and each tool's `argsSchema.json`) + an instruction to choose one or answer directly — and calls `runtime.structured(LLMRequest(prompt:reasoning:), schema: RouteSchema, as: ToolRoute.self)`. Outcomes map: -- `.value(route)` with `route.tool == ""` → **plain answer** (the loop streams a text turn and stops). -- `.value(route)` with a non-empty `route.tool` that matches a candidate descriptor → a **routed call**. -- `.value(route)` with a `tool` that matches NO descriptor → treated as a plain answer (defensive; never dispatch an unknown tool). Logged. -- `.declined(reason)` → **plain answer** (the model judged no tool fits — exactly the first-class "just talk"). -- `throws RuntimeError.couldNotProduceValid` → **plain answer** fallback (a malformed route is never a fabricated tool call; the spec's "do not depend on hard token-level caging" applies to the router itself). -- any other `throws` → mapped via `AIError.message(for:)` to a clean headline, surfaced as a `.failed` loop outcome (never silence). - -This reuses the seam we already trust for structure and inherits its repair/retry/decline for free. **The router is the reliability mechanism; the model's raw tokens are never parsed for tool calls.** - -### 2. Type contracts (blueprint §3.3, OWNED here) — bare `WritePolicyTier` lives here (C1) -```swift -public enum WritePolicyTier: String, Codable, Equatable, Sendable { - case auto // whitelisted/safe: runs without confirm, even when parked (still audited downstream) - case confirm // default: needs foreground approval (DOWN=approve / RIGHT=skip) - case dangerous // always escalates to foreground (needs-you), even if parked -} - -public struct ToolDescriptor: Codable, Equatable, Sendable { - public let name: String // stable id, e.g. "add_to_calendar", "send_to", "memory.write" - public let summary: String // one line the router sees - public let argsSchema: StructuredSchema - public let writePolicy: WritePolicyTier - public var keywords: [String] // cheap retrieval ranking signal (Decision 6) -} - -public struct ToolRoute: Codable, Equatable, Sendable { - public let tool: String // "" = plain text answer - public let argumentsJSON: String // JSON object string; "" when tool == "" - public let rationale: String? - public var isPlainAnswer: Bool { tool.isEmpty } -} - -public struct ToolStepResult: Codable, Equatable, Sendable { - public let tool: String - public let status: ToolStepStatus - public let summary: String // short human outcome; fed back as a .tool message -} -public enum ToolStepStatus: Codable, Equatable, Sendable { - case done - case awaitingApproval - case declined(reason: String) - case failed(headline: String) // AIPresentedError.headline ONLY; raw text → logs/details -} -``` -`WritePolicyTier` is defined here so a descriptor is self-describing. `ai-background-autonomy` does NOT redefine it — it owns the user **whitelist**, effective-tier **resolution** (`descriptor.writePolicy ∩ whitelist`), the audit log, and escalation. `ai-agent-memory` and `ai-claude-handoff` CONSUME the enum off their descriptors. - -### 3. `ToolRegistry` + `ToolContributor` — aggregation, no loop coupling -```swift -public protocol ToolContributor: Sendable { - func descriptors() -> [ToolDescriptor] - func canHandle(_ tool: String) -> Bool - // Run a routed call; returns a ToolStepResult. May produce an awaitingApproval pause. - func run(_ call: RoutedCall, gate: ApprovalGate) async -> ToolStepResult -} - -public struct ToolRegistry: Sendable { - private let contributors: [ToolContributor] - public func allDescriptors() -> [ToolDescriptor] - public func descriptor(named: String) -> ToolDescriptor? - public func run(_ call: RoutedCall, gate: ApprovalGate) async -> ToolStepResult -} -``` -v1 ships ONE contributor: `TaskKindToolContributor` (Decision 4). Later waves register `MemoryToolContributor`, `SkillToolContributor`, `ClaudeHandoffContributor` — each just adds descriptors and a `run`; the loop is untouched. The registry is the blueprint's "aggregates TaskKind tasks + memory tools + skill invocation + launch_claude." - -### 4. The bridge: a routed call → the EXISTING `TaskDispatcher.prepare`/`execute` (the load-bearing reuse) -`TaskDispatcher.prepare` takes a `resolvedPrompt: String` and selects the schema from the `TaskKind`. A `ToolRoute` carries `argumentsJSON`. The bridge in `TaskKindToolContributor`: -1. Map `ToolDescriptor.name` → a `TaskKind` (+ its stored config). The descriptor for a `saveToProject`/`openToolWithPayload`/`sendTo` tool **binds the config at registration** (project name / tool / destination come from the authored command or skill, never invented by the router — exactly the existing rule in `ParsedActions.swift`). So `name == "save_to_project:"`-style descriptors resolve back to `TaskKind.saveToProject(project:)`. -2. Build the `resolvedPrompt` the dispatcher expects by **folding the route's `argumentsJSON` into the prompt** as a structured instruction block (e.g. "Use these arguments: {…}"). The dispatcher's own `structured()` parse then re-validates against the kind's `ParsedActions` schema — so the router's args are a *hint*, and the kind's schema remains the authority. This keeps `TaskDispatcher`/`ParsedActions`/`TaskSinks` **byte-unchanged**: the model "chose the menu item and pre-filled it"; the existing two-stage prepare still validates. -3. Call `dispatcher.prepare(kind, resolvedPrompt:, source:, reasoning:)` → `TaskReview`. -4. Map `TaskReview` → `ToolStepResult` + the approval decision (Decision 5). - -This is why the slice says "reuse the task machinery WHOLESALE": the contributor is a thin adapter; no parse logic, no sink, no schema is duplicated. - -### 5. Per-step gating: read-only auto-runs; side-effecting waits for DOWN=approve -A descriptor's `writePolicy` (intersected by an injected `WritePolicyResolving`) decides the gate: -- `.auto` → run immediately: `prepare` → if `.action`, `execute` right away → `ToolStepResult(.done, summary)`. -- `.confirm` / `.dangerous` → `prepare` → if `.action`, **pause**: emit `ToolStepResult(.awaitingApproval, summary)` and surface the backing `TaskReview` as observable loop state. The loop suspends until the `ApprovalGate` resolves: - - **approve (DOWN)** → `execute(review)` → `.done` (or `.failed` if the sink throws — never a false "Done", per the executor's existing honesty rule). - - **skip (RIGHT)** → `ToolStepResult(.declined(reason: "skipped"))`, fed back so the model continues without that effect. -- `prepare` → `.declined` → `ToolStepResult(.declined(reason))` (no gate; nothing fires). -- `prepare` → `.unavailable(reason)` → `ToolStepResult(.failed(headline: reason))`. - -```swift -public protocol WritePolicyResolving: Sendable { - func effectiveTier(for descriptor: ToolDescriptor) -> WritePolicyTier -} -// Stand-alone default so this slice compiles + tests without ai-background-autonomy: -public struct DescriptorWritePolicy: WritePolicyResolving { - public func effectiveTier(for d: ToolDescriptor) -> WritePolicyTier { d.writePolicy } -} -``` -`ApprovalGate` is an async seam the canvas drives (DOWN/RIGHT). In tests it is a scripted gate that approves/skips deterministically. The canvas (slice `ai-conversational-canvas`) binds the `.awaitingApproval` `TaskReview` to its action-review preview and resolves it with the canonical compass DOWN=approve / RIGHT=skip — identical mnemonic to commit/discard. - -### 6. Candidate retrieval — never the full registry in front of the router -A `ToolCandidateSource` surfaces ~3–5 candidates per route turn: -```swift -public protocol ToolCandidateSource: Sendable { - func candidates(for context: RouteContext, limit: Int) -> [ToolDescriptor] -} -``` -v1 default `KeywordToolCandidateSource`: cheap lexical match of the latest user turn against each descriptor's `name`/`summary`/`keywords` (token overlap + substring), top-`limit` (default 5), always including any tool the active skill explicitly allows. The shape deliberately mirrors `DocIndex.retrieve(query:limit:) -> [IndexedDoc]` (blueprint §3.4) so when `ai-skills-as-files`/`ai-agent-memory` land, a `DocIndexToolCandidateSource` adapter ranks skill/memory tools through the SAME retriever — no second ranking path. **Retrieval is itself a routed tool step:** a `widen_candidates`/`retrieve` `ToolDescriptor` lets the model ask for more tools when the 5 candidates don't fit, re-entering the loop with a broader set (blueprint §3.4 "retrieval is itself a routed tool step"). - -### 7. The bounded loop (`AgentLoop`) — route → execute → continue, with a hard cap + loop-guard -Pure Core, owns no UI. `now`/randomness are not needed (the loop is deterministic given scripted outcomes). Shape: - -``` -loop step s from 0 ..< maxToolSteps: // maxToolSteps default 8 - candidates = candidateSource.candidates(for: context, limit: 5) - route = router.route(context, candidates) // a structured() call (Decision 1) - emit route.rationale into the .thinking channel // live plan (Decision 8) - if route.isPlainAnswer: - stream a final text answer turn; END(.answered) - guard let d = registry.descriptor(named: route.tool) else { stream answer; END(.answered) } - // loop-guard / no-progress (Decision 9): - if route == lastRoute { END(.stopped(.repeatedStep)) } - result = registry.run(RoutedCall(descriptor: d, route: route, source:), gate: gate) - append .tool AgentMessage(toolResult: result) to context - emit result.summary into .thinking - switch result.status: - .failed(h): END(.failed(headline: h)) // a side effect that didn't land - .declined: continue (the model sees it and may re-route) - .done / .awaitingApproval→resolved: continue - lastRoute = route -END(.capReached) when the for-range is exhausted → stream a best-effort final answer noting the cap. -``` -Terminal outcomes are an observable `AgentLoopOutcome` enum (`.answered(text)`, `.stopped(reason)`, `.failed(headline)`, `.capReached`). A `.failed` always carries a clean `AIPresentedError.headline`. The loop NEVER ends silently and NEVER fires a side effect outside `registry.run` (which itself only fires through the gated `TaskDispatcher.execute`). - -### 8. The live plan rides the existing `.thinking` channel — no third channel -Each step's `route.rationale` and each `ToolStepResult.summary` are emitted as `.thinking`-channel text (blueprint: "do NOT add a third channel without cross-slice sign-off"). The canvas already renders `.thinking` into its collapsible section; the running plan appears there for free. The committed answer is `.response`-channel only, exactly as today. The visible-plan UX contract with `ai-conversational-canvas`: the loop publishes an ordered `[ToolStepResult]` + the current `.awaitingApproval` review as observable state; the canvas draws the step list + the pending approval card and feeds the `ApprovalGate`. - -### 9. No-progress / loop-guard (small-model spin defense) -Two cheap guards, both pure and unit-tested: -- **Repeated step:** if a `ToolRoute` is byte-identical to the immediately preceding executed route (same `tool` + `argumentsJSON`), END `.stopped(.repeatedStep)` — the model is spinning on the same call. -- **Declined-then-re-route ceiling:** if the model re-routes to the same tool after it just `.declined`/`.failed` for that tool more than once, END `.stopped(.noProgress)`. A `.declined` result is fed back so the model *can* pivot; refusing to pivot terminates. -- The **hard step cap** (`maxToolSteps`) is the backstop regardless of progress. -On any guard termination the loop streams a best-effort final answer summarizing what it did and why it stopped (never a bare halt). - -### 10. Errors — one taxonomy, mapped at the boundary, observable + bounded -- Router/runtime failures map through `AIError.message(for:)` → `ToolStepResult(.failed(headline))` / `AgentLoopOutcome.failed(headline)`. `RuntimeError.cancelled` (a discard) is NOT a failure — it ends the loop quietly like the executor's existing cancel path. -- Task-side failures are already `TaskError` (`LocalizedError`) mapped in `TaskSinks`; `prepare`/`execute` surface clean reasons; the contributor passes the clean headline into `.failed`. -- **No new `Error` is introduced** — `RuntimeError` + `TaskError` carry every failure this slice can produce (route-malformed → `couldNotProduceValid` → plain-answer fallback, not an error; unknown tool → defensive plain answer, logged). This honors "at most one `Error`, only if `RuntimeError`/`TaskError` cannot carry it." -- No `NSAlert`. The slice produces only observable state + clean headlines; the canvas renders them bounded + non-blocking with a Retry/Skip affordance. - -## Type & file touch list (all Core, MLX-free; verified by `swift test` unless noted) - -| File (new unless noted) | Target | Contents | Verification | -|---|---|---|---| -| `AI/Agent/ToolContracts.swift` | Core | `WritePolicyTier`, `ToolDescriptor`, `ToolRoute`, `ToolStepResult`, `ToolStepStatus`, `RoutedCall`, `RouteContext` | `swift test` (Codable round-trip, Equatable) | -| `AI/Agent/RouteSchema.swift` | Core | `RouteSchema` (`static let` `StructuredSchema`) + route-prompt builder (candidates → prompt text) | `swift test` (prompt includes candidate names/schemas) | -| `AI/Agent/ToolRouter.swift` | Core | `ToolRouter.route(...)` over `runtime.structured(...)`; outcome→route mapping incl. decline/couldNotProduceValid fallbacks | `swift test` w/ scripted `StubLLMRuntime` | -| `AI/Agent/ToolRegistry.swift` | Core | `ToolContributor`, `ToolRegistry`, `ApprovalGate` (async approve/skip seam) | `swift test` (aggregation, dispatch, unknown-tool) | -| `AI/Agent/TaskKindToolContributor.swift` | Core | descriptors for the existing `TaskKind`s; the args→`resolvedPrompt` fold; `TaskReview`→`ToolStepResult` mapping; gate integration | `swift test` w/ fake `TaskDispatching` + scripted gate | -| `AI/Agent/WritePolicyResolving.swift` | Core | `WritePolicyResolving` seam + `DescriptorWritePolicy` default | `swift test` | -| `AI/Agent/ToolCandidateSource.swift` | Core | `ToolCandidateSource` + `KeywordToolCandidateSource` (+ the `widen_candidates`/`retrieve` descriptor) | `swift test` (top-N ranking, skill-allowed inclusion) | -| `AI/Agent/AgentLoop.swift` | Core | the bounded route→execute→continue loop; `AgentLoopOutcome`; step cap; loop-guard; `.thinking` plan emission | `swift test` (cap, loop-guard, plain-answer, approval pause, failed-step) | -| `AI/Agent/ConversationSeam.swift` | Core | narrow `ConversationContext`/`ChatStreaming` protocols I own UNTIL `ai-conversation-runtime` lands, then re-pointed at `AgentMessage`/`chat()` (Open Question Q1) | `swift test` | -| `Tests/.../ToolRouterTests.swift` etc. | Core (test) | scripted `StubLLMRuntime` structured outcomes; fake dispatcher/gate/candidate source | `swift test` | - -No file in this slice links MLX. The real model that ultimately answers route turns is the batched conformer from `ai-batched-runtime-and-context` (xcodebuild compile-only there); here the loop drives an injected `LLMRuntime` and is verified entirely against `StubLLMRuntime`. **No `.app` build, no signing, no permission change — the user's stable-signed build is unaffected.** - -## Edge cases - -- **Route says a tool the candidate set didn't include** (model hallucinated a name): no descriptor → defensive plain answer, logged; never dispatched. The `widen_candidates` tool is the legitimate path to more tools. -- **Route returns `tool != ""` but empty/garbage `argumentsJSON`:** the fold passes whatever it has; the kind's own `structured()` parse in `prepare` either repairs it or returns `.unavailable` → `.failed` step → model sees it and can re-route. The router's bad args never reach a sink unvalidated. -- **`.awaitingApproval` step then the user discards the whole canvas:** the `ApprovalGate` resolves as cancelled (not skip, not approve); the loop ends like the executor's cancel path (`RuntimeError.cancelled` is not a failure). No side effect fires. -- **A `.auto` step whose `prepare` returns `.action` but `execute` throws** (e.g. permission denied on an auto calendar write): `.failed(headline)` with the clean `TaskError` message; the loop surfaces it and stops or continues per Decision 7 — never a false "Done." -- **Model never picks a tool (always `tool == ""`):** the very first route turn answers and stops — the common chat case; zero tool steps, zero cost beyond one route turn. -- **Cap reached mid-plan:** stream a best-effort answer noting the cap; the partial side effects that already committed are recorded (auditable downstream); the loop does not silently truncate. -- **Two candidates with identical `name`:** registry dedupes by `name` (first contributor wins); a name collision across contributors is a registration-time assertion in debug. -- **Reasoning flag:** the route turn carries `reasoning` from the active command/turn (like `TaskDispatcher.prepare`'s `reasoning:`), so a model that thinks-before-routing streams its plan into `.thinking` without changing the committed answer. - -## Rejected alternatives - -- **Gemma native function-call tokens.** Rejected: small models emit them unreliably, and `ai-command-tasks` already commits to "SHALL NOT depend on hard token-level caging to obtain structure." The `structured()` router (with repair/retry/decline) is the reliability mechanism; native tokens would fork the trust model and double the failure surface. -- **A new parallel `ToolDispatcher` separate from `TaskDispatcher`.** Rejected: it would duplicate the parse/review/execute/sink machinery the blueprint says to reuse wholesale and break the MLX-free/`StubLLMRuntime`-tested boundary. The contributor is a thin adapter instead. -- **Passing `argumentsJSON` straight to a sink, bypassing the kind's schema parse.** Rejected: it would let the router's (less reliable) JSON reach a side effect unvalidated. Folding args into the `resolvedPrompt` keeps the kind's `ParsedActions` schema as the authority — the existing validation/repair/decline stays in force. -- **All tools in front of the router every turn.** Rejected: blows the context budget and degrades small-model routing accuracy. Candidate retrieval (~5) is mandatory; the model widens via a routed tool step. -- **Defining `WritePolicyTier` in `ai-background-autonomy` and importing it here.** Rejected per integration fix C1: it creates a DAG back-edge (routing → autonomy) although routing is the earlier wave. The bare enum lives with `ToolDescriptor`; autonomy owns only resolution/audit/escalation. -- **A dedicated `RoutingError`/`AgentError` taxonomy.** Rejected: `RuntimeError` + `TaskError` carry every failure; a route-malformed is a graceful plain-answer fallback, not an error. Adding a taxonomy violates "at most one `Error`, only if the existing ones cannot carry it." -- **A separate non-`.thinking` "plan" channel.** Rejected per blueprint (no third channel without cross-slice sign-off); the plan rides `.thinking`, which the canvas already renders. - -## Target-split & verification summary - -- **Everything in this slice is MLX-free Core**, verified by `swift build` + `swift test`. The router, loop, registry, contributor, candidate source, and contracts are pure value types + protocols driven by an injected `LLMRuntime`; tests inject `StubLLMRuntime` with **scripted structured outcomes** (`.value(route)` / `.declined`) and a fake `TaskDispatching` + scripted `ApprovalGate`. -- **No `xcodebuild`/`.app` work in this slice.** The MLX dependency is downstream (`ai-batched-runtime-and-context` provides the real `chat()` conformer the loop drives in production); compile-verify of that is owned there. To compile-check this slice in isolation without other in-flight slices' uncommitted files, use a throwaway `git worktree` + `swift build`. -- **No signing, no permission, no TCC interaction** — the slice never touches the build/sign path. - -## Open Questions - -- **Q1 (sequencing seam):** `ai-conversation-runtime` (`AgentMessage`/`AgentConversation`/`AgentTurn`) and `on-device-ai-runtime` (`LLMChatRequest`/`chat()`) are not yet committed on disk. I plan a narrow `ConversationContext`/`ChatStreaming` seam I own so the loop compiles + tests in isolation, then re-point it at the real types when they land. If the owners land first, I bind directly and delete the seam. Confirm the §3.1/§3.2 sketches are final enough to bind. -- **Q2 (descriptor identity for configured tasks):** I propose encoding the bound config in the descriptor `name` (e.g. `save_to_project:`) so a routed call resolves deterministically back to a `TaskKind(+config)`. Alternative: a side table mapping descriptor → `(TaskKind, config)`. The side-table is cleaner but adds state; confirm preference. -- **Q3 (`maxToolSteps` default):** 8 is a guess balancing useful multi-hop against small-model spin. Tune in run-verify with the real batched runtime. -- **Q4 (candidate `limit`):** 5 per the prompt's "~3–5". Whether the active skill's allowed tools count against the 5 or are additive is a small policy choice I default to "additive, capped at 8 total" — confirm with `ai-skills-as-files`. diff --git a/openspec/changes/ai-tool-routing/proposal.md b/openspec/changes/ai-tool-routing/proposal.md deleted file mode 100644 index 34bde49..0000000 --- a/openspec/changes/ai-tool-routing/proposal.md +++ /dev/null @@ -1,30 +0,0 @@ -## Why - -Today the AI feature is **user-driven, one-shot, fire-then-commit**: the human chooses the `TaskKind` (the command's `OutputTarget`), the executor resolves a prompt, the dispatcher asks the model for a schema-validated parsed action, and a single side effect commits on a down-swipe. The whole agentic machinery — `TaskDispatching.prepare → TaskReview → PreparedAction → TaskDispatcher.execute → TaskSinks` — already exists, is MLX-free, and is unit-tested headless against `StubLLMRuntime`. What is missing for V2 is **inversion of control**: let the *model* pick which tool to run from a turn of conversation, run it, feed the result back, and continue — a bounded agent loop — while reusing every existing seam wholesale. - -The reliability mechanism for "which tool" is **not** Gemma's native function-call tokens (small models emit them unreliably and we deliberately "do not depend on hard token-level caging" — `ai-command-tasks` spec). It is the seam we already trust: `LLMRuntime.structured(_:schema:as:)`, with its bounded repair/retry and first-class `.declined`. A **route turn** is a `structured()` call against a fixed `ROUTE_SCHEMA` that returns either `{ tool, argumentsJSON }` (call a tool) or `{ tool: "" }` (just talk). "No tool, just answer" is first-class, never a degenerate failure. - -This slice is `ai-tool-routing` (capability `ai-command-tasks`). It owns the route schema, the `ToolDescriptor`/`ToolRoute`/`ToolStepResult` contracts (per the shared blueprint §3.3), the `ToolRegistry`, candidate retrieval (so the router never sees all tools), and the **pure, bounded route → execute → continue loop**. It does NOT own the conversation types (those are `ai-conversation-runtime`, blueprint §3.1) nor the canvas UX (that is `ai-conversational-canvas`); it consumes the former and is consumed by the latter. - -## What Changes - -- **A route turn built on `structured()`.** A new `RouteSchema` (a fixed `StructuredSchema`) and a `ToolRouter` that calls `runtime.structured(LLMRequest(prompt: routePrompt), schema: RouteSchema, as: ToolRoute.self)`. The router prompt advertises only the **candidate** tools (their `name` + `summary` + `argsSchema`). A `StructuredOutcome.declined` OR a decoded `ToolRoute` with `tool == ""` both mean "no tool — produce a plain text answer"; a non-empty `tool` is a routed call. Exhausted repair (`RuntimeError.couldNotProduceValid`) falls back to a plain text answer, never a fabricated tool call. -- **`ToolDescriptor` / `ToolRoute` / `ToolStepResult` / `ToolStepStatus` (blueprint §3.3), with the bare `WritePolicyTier` enum defined HERE** (integration fix C1: every descriptor carries its tier without a DAG back-edge to `ai-background-autonomy`, which owns only *resolution*, the audit log, and escalation). -- **A `ToolRegistry`** (Core) that aggregates contributors into `[ToolDescriptor]` and dispatches a routed call. v1 contributors are the existing `TaskKind` tasks (the calendar/reminder/contact/save/open/send capabilities) exposed as descriptors; later waves add memory tools, skill invocation, and `launch_claude` as additional contributors behind the same `ToolContributor` seam — no loop change. -- **The bridge to the existing task machinery.** A routed `TaskKind` call reuses `TaskDispatcher.prepare`/`execute`, `TaskReview`, `PreparedAction`, `ParsedActions`, `TaskSinks` **wholesale**. The router only chooses the `TaskKind` (and its config: project / tool / destination) the *user* used to choose; `prepare` still does the schema-targeted parse → `TaskReview`. The model "chose the menu item"; everything downstream is unchanged. -- **Per-step write-policy gating.** A descriptor's `writePolicy` decides whether a step auto-runs (read-only / `.auto`) or waits for approval (`.confirm` / `.dangerous`). The loop surfaces an `.awaitingApproval` step (a `TaskReview`-backed pause); the canvas resolves it with **DOWN = approve / RIGHT = skip** (the canonical compass, reused — `ai-conversational-canvas` renders it). This slice defines the *contract* and the *gate*; the actual whitelist/escalation lives in `ai-background-autonomy`, consumed via an injected `WritePolicyResolving` seam (defaults to descriptor tier so this slice stands alone). -- **A bounded agent loop.** A hard **step cap** (`maxToolSteps`, default 8), the running plan rendered live through the existing `.thinking` channel, per-step approval, and **no-progress / loop-guard termination** (a repeated identical `(tool, argumentsJSON)` or a repeated declined step ends the loop with a clean summary, never spins). -- **Candidate retrieval.** A `ToolCandidateSource` that surfaces ~3–5 candidate tools per route turn by cheap name/summary/keyword matching (and, later, the shared `DocIndex` for skills/memory). The router never sees the full registry. Retrieval is itself expressible as a routed tool step (blueprint §3.4) so the model can ask to widen the candidate set. - -## Capabilities - -### Modified Capabilities - -- `ai-command-tasks`: the schema-targeted, validated, declinable task machinery — unchanged in its parse/review/execute contract — is now ALSO reachable by **model-driven routing**: a route turn (a `structured()` call against a fixed route schema) selects a `TaskKind`-shaped tool, the bounded loop runs it via the existing `TaskDispatcher`, the result is fed back, and generation continues. The per-step approval reuses the action-review confirmation (DOWN=approve / RIGHT=skip) and the existing decline/unavailable/failed mapping. Adds the `ToolDescriptor`/`ToolRoute`/`ToolStepResult` contracts, the `ToolRegistry`, the bare `WritePolicyTier`, candidate retrieval, and the bounded loop. - -## Impact - -- **Code (MLX-free Core, `AI/Agent/`):** new `ToolRoute`, `ToolDescriptor`, `ToolStepResult`/`ToolStepStatus`, `WritePolicyTier`, `RouteSchema`, `ToolRouter` (the route turn over `structured()`), `ToolRegistry` + `ToolContributor`, `TaskKindToolContributor` (bridges existing tasks to descriptors), `ToolCandidateSource`, `WritePolicyResolving` (default seam), and the pure `AgentLoop` (route → execute → continue, bounded). All pure/`@MainActor`-as-needed, verified by `swift test` against `StubLLMRuntime` with **scripted structured outcomes**. -- **Reuse, not rebuild:** `LLMRuntime.structured`/`StructuredSchema`/`StructuredOutcome`/`RuntimeError`, `TaskDispatching`/`TaskDispatcher`/`TaskReview`/`PreparedAction`/`ParsedActions`/`TaskSinks`, `AIError.message(for:)`, `TokenChannel.thinking`. Consumes `AgentMessage`/`AgentConversation`/`AgentSessionID`/`LLMChatRequest`/`chat()` from `ai-conversation-runtime` / `on-device-ai-runtime` (blueprint §3.1/§3.2). -- **MLX-free Core / verification:** the entire slice is Core and verifies under `swift build` + `swift test`. No piece links MLX; the real batched runtime that the loop ultimately drives is `ai-batched-runtime-and-context` (xcodebuild compile-only there, not here). No `.app` build, no signing, no permission change. -- **Out of scope:** the conversation/message types and compaction (`ai-conversation-runtime`); the canvas state cases + gesture interpretation (`ai-conversational-canvas`); the skills/memory `DocIndex` retriever itself (`ai-skills-as-files` owns it; this slice consumes the contract); the whitelist + audit log + escalation policy (`ai-background-autonomy`); `launch_claude` and `ClaudeHandoffConfig` (`ai-claude-handoff`). This slice defines only the seams those plug into. diff --git a/openspec/changes/ai-tool-routing/specs/ai-command-tasks/spec.md b/openspec/changes/ai-tool-routing/specs/ai-command-tasks/spec.md deleted file mode 100644 index 00de182..0000000 --- a/openspec/changes/ai-tool-routing/specs/ai-command-tasks/spec.md +++ /dev/null @@ -1,109 +0,0 @@ -## ADDED Requirements - -### Requirement: Model-driven tool routing via a structured route turn -The system SHALL allow the **model** to select which task ("tool") to run from a turn of conversation, using the existing schema-targeted **structured-output** mechanism as the router — NOT native function-call tokens. A **route turn** SHALL be a `structured(...)` call against a fixed route schema that returns either a chosen tool with its arguments, or an explicit "no tool" decision (a plain text answer). The route turn SHALL inherit structured output's bounded repair/retry and first-class decline: a declined route, an empty-tool route, an unknown tool name, or an exhausted-repair route SHALL all resolve to a **plain text answer**, and SHALL NEVER fabricate or dispatch a tool call. The system SHALL NOT depend on hard token-level function-call caging to obtain the routing decision. - -#### Scenario: Route turn selects a tool -- **WHEN** a conversational turn is best served by a task (e.g. "add this to my calendar") -- **THEN** the route turn returns the matching tool and its arguments, and that tool is run - -#### Scenario: No tool is needed (plain answer is first-class) -- **WHEN** a conversational turn needs no task (e.g. an ordinary question) -- **THEN** the route turn returns "no tool" (an empty tool or a decline) and the system answers directly without running any task - -#### Scenario: A malformed or unknown route never fabricates a tool call -- **WHEN** the route turn's repair/retry loop is exhausted, declines, or names a tool that is not a candidate -- **THEN** the system falls back to a plain text answer and dispatches no side effect - -### Requirement: A routed tool reuses the existing task machinery -A tool the model routes to SHALL be a `TaskKind`-shaped capability, and running it SHALL reuse the existing task pipeline **wholesale** — the same `prepare` (schema-targeted, validated, repairable, declinable structured output), the same action `review` (fields + payload), and the same `execute` (the side-effect sink) the user-driven path uses. The model SHALL only choose the tool (and inherit its authored configuration — the project / tool / destination); the tool's own schema SHALL remain the authority that validates the action, so the router's arguments are a hint, never an unvalidated side effect. - -#### Scenario: Routed calendar tool runs through the existing prepare/review/execute -- **WHEN** the model routes a turn to the "add to calendar" tool -- **THEN** the task's existing structured-output prepare produces a validated event review, and (after gating) the existing execute creates the event — no separate dispatch path is used - -#### Scenario: A routed tool's configuration is not invented by the model -- **WHEN** the model routes to a "save to project" or "send to destination" tool -- **THEN** the target project / destination comes from the tool's authored configuration, not from the model, and the tool's schema validates the content the model produced - -#### Scenario: A routed tool that cannot produce a valid action does not fire -- **WHEN** a routed tool's structured prepare exhausts repair without a valid action -- **THEN** no side effect fires and the step is reported as unavailable, fed back so the model may continue - -### Requirement: Per-step write-policy gating (read-only auto-runs; side-effecting waits for approval) -Each tool SHALL declare a **write-policy tier** (auto / confirm / dangerous). A routed step whose effective tier is **auto** (read-only or whitelisted) SHALL run without an extra confirmation; a step whose tier is **confirm** or **dangerous** SHALL pause as an **awaiting-approval** step that surfaces the action review, and SHALL apply its side effect only on an explicit approval. The approval SHALL reuse the canonical two-finger compass — **DOWN = approve / RIGHT = skip** — mirroring the existing commit/discard resolution. A skipped step SHALL apply no side effect and SHALL be fed back so the model may continue; a discarded session SHALL apply no side effect. - -#### Scenario: Read-only step runs without confirmation -- **WHEN** the model routes to a read-only (auto-tier) tool -- **THEN** the step runs immediately and its result is fed back into the loop without a confirmation pause - -#### Scenario: Side-effecting step waits for a DOWN approval -- **WHEN** the model routes to a side-effecting (confirm/dangerous) tool and its action review is ready -- **THEN** the step pauses showing the review, and the side effect fires only when the user approves with a DOWN swipe - -#### Scenario: A skipped step applies nothing and the loop continues -- **WHEN** the user skips an awaiting-approval step with a RIGHT swipe -- **THEN** no side effect fires, the step is reported skipped, and the model continues with that outcome - -#### Scenario: A side effect that did not land is a failure, never a false success -- **WHEN** an approved (or auto) step's side effect throws when executed -- **THEN** the step is reported failed with a clean headline, never reported done - -### Requirement: Bounded multi-hop agent loop -The system SHALL run a **bounded** route → execute → continue loop: it routes a turn, runs the chosen tool, appends the tool result as a turn, and re-routes — up to a hard **step cap**. The loop's running plan (each step's rationale and outcome) SHALL be surfaced live through the existing reasoning ("thinking") channel and SHALL NOT be committed as the answer. The loop SHALL detect **no progress** — a repeated identical routed call, or repeatedly re-routing to a tool that just declined/failed — and terminate with a best-effort final answer rather than spinning. Every termination (answer, cap reached, no-progress, failure) SHALL produce an observable outcome with a clean message; the loop SHALL NEVER end silently. - -#### Scenario: Multi-hop plan runs within the step cap -- **WHEN** a turn needs several tools in sequence -- **THEN** the loop runs each in turn, feeding each result back, and stops at or before the hard step cap - -#### Scenario: Running plan is shown in the thinking channel -- **WHEN** the loop routes and runs steps -- **THEN** the step rationales and outcomes appear in the collapsible thinking section, while only the final answer is committed - -#### Scenario: No-progress loop is guarded -- **WHEN** the model re-routes to the same call (or re-tries a just-failed tool) without progress -- **THEN** the loop terminates with a best-effort final answer summarizing what it did, rather than repeating indefinitely - -#### Scenario: Cap reached terminates cleanly -- **WHEN** the loop reaches the hard step cap -- **THEN** it stops and streams a best-effort answer noting the cap, never silently truncating - -### Requirement: Candidate tool retrieval (never all tools in front of the router) -The route turn SHALL be offered only a small **candidate** set of tools (about three to five), selected by cheap name/summary/keyword matching against the turn, rather than the entire tool registry. The candidate-selection shape SHALL be the SAME retrieval contract shared with skills and memory (a table-of-contents of summaries with on-demand expansion), so there is one ranking/IO path. The model SHALL be able to request a **wider** candidate set as itself a routed tool step when the offered candidates do not fit. - -#### Scenario: Router sees a small candidate set -- **WHEN** a turn is routed -- **THEN** only a handful of relevant candidate tools (with their summaries and argument schemas) are presented to the route turn, not the full registry - -#### Scenario: The model can widen the candidate set -- **WHEN** the offered candidates do not fit the turn -- **THEN** the model may route to a "widen candidates" tool step, and the next route turn is offered a broader set - -#### Scenario: Skill-allowed tools are always candidates -- **WHEN** a skill is driving the session and declares allowed tools -- **THEN** those tools are included among the route turn's candidates regardless of lexical match - -## MODIFIED Requirements - -### Requirement: Action review before side effects (default on, user-overridable) -A side-effecting task SHALL present an **action-review preview** (the concrete fields that will be applied) before it executes **when the command's `confirmBeforeRun` is enabled**, and `confirmBeforeRun` SHALL **default to enabled** for side-effecting tasks. The user MAY disable it per command; when disabled, the task commits without the extra action-review step (the baseline deliberate commit still applies). Discarding SHALL always cancel the task with no side effect. The SAME action-review preview and confirmation SHALL back a **model-routed** side-effecting step: a routed step whose effective write-policy requires confirmation SHALL surface the action review as an awaiting-approval step resolved by **DOWN = approve / RIGHT = skip**, and a routed step whose effective write-policy is **auto** (read-only / whitelisted) MAY run without the extra confirmation while still being recorded — so the review contract is shared between the user-driven and model-driven paths, not duplicated. - -#### Scenario: Action is shown before it fires (default) -- **WHEN** a calendar/save/open/send task whose command has `confirmBeforeRun` enabled has produced its parsed action -- **THEN** the user sees the action's fields and nothing is applied until they commit - -#### Scenario: Confirmation defaults on for side-effecting tasks -- **WHEN** a side-effecting command is created without an explicit choice -- **THEN** its `confirmBeforeRun` defaults to enabled - -#### Scenario: User may disable review for a trusted task -- **WHEN** the user disables `confirmBeforeRun` on a side-effecting command and commits it -- **THEN** the task executes its side effect without the extra action-review step, honoring the stored value - -#### Scenario: Discard cancels with no effect -- **WHEN** the user discards a task before committing -- **THEN** no event is created, no file is written, no tool is opened, and nothing is sent - -#### Scenario: A model-routed side-effecting step reuses the same review -- **WHEN** the model routes to a side-effecting tool whose effective write-policy requires confirmation -- **THEN** the same action-review preview is shown as an awaiting-approval step and the side effect fires only on a DOWN approval (a RIGHT skip applies nothing) diff --git a/openspec/changes/ai-tool-routing/tasks.md b/openspec/changes/ai-tool-routing/tasks.md deleted file mode 100644 index 2d1f805..0000000 --- a/openspec/changes/ai-tool-routing/tasks.md +++ /dev/null @@ -1,48 +0,0 @@ -> Decomposed for a workflow fan-out: §1 is the type substrate (do first), §2–§6 are the router / registry / bridge / candidates / loop (each independent once §1 lands), §7 is the cross-slice seams, §8 verifies. Every item is MLX-free Core verified by `swift test` unless noted; NO `.app` build, NO signing. - -## 1. Tool contracts (pure Core type substrate) - -- [x] 1.1 Add `AI/Agent/ToolContracts.swift`: `WritePolicyTier` (`auto`/`confirm`/`dangerous`), `ToolDescriptor{name, summary, argsSchema: StructuredSchema, writePolicy, keywords}`, `ToolRoute{tool, argumentsJSON, rationale?}` (+ `isPlainAnswer`), `ToolStepResult{tool, status, summary}`, `ToolStepStatus{done, awaitingApproval, declined(reason), failed(headline)}`, `RoutedCall`, `RouteContext`. All `Codable`/`Equatable`/`Sendable`. Bare `WritePolicyTier` lives HERE (integration fix C1). *Verify: `swift test` — Codable round-trip + Equatable for each type.* -- [x] 1.2 Doc-comment that `WritePolicyTier` is OWNED here and `ai-background-autonomy` consumes it for resolution/audit/escalation only (no redefinition). *Verify: review.* - -## 2. The route turn (`structured()` as router) - -- [x] 2.1 Add `AI/Agent/RouteSchema.swift`: the fixed `RouteSchema` `StructuredSchema` (`required: ["tool"]`, properties `tool`/`argumentsJSON`/`rationale`) mirroring `ParsedActions.swift` style; plus a pure route-prompt builder that renders the candidate descriptors (name + summary + `argsSchema.json`) + the conversation tail + the "choose one or answer directly" instruction. *Verify: `swift test` — the built prompt contains each candidate's name + schema and the empty-tool instruction.* -- [x] 2.2 Add `AI/Agent/ToolRouter.swift`: `ToolRouter.route(context, candidates, runtime, reasoning) async -> ToolRoute` calling `runtime.structured(LLMRequest(prompt:reasoning:), schema: RouteSchema, as: ToolRoute.self)`; map `.value(tool=="")` / `.declined` / `couldNotProduceValid` / unknown-tool → plain answer; non-empty matched tool → routed call; other throws → mapped failure (via `AIError.message(for:)`). `RuntimeError.cancelled` is not a failure. *Verify: `swift test` with scripted `StubLLMRuntime` for each outcome branch.* - -## 3. Registry + approval gate - -- [x] 3.1 Add `AI/Agent/ToolRegistry.swift`: `ToolContributor` protocol (`descriptors()`, `canHandle(_:)`, `run(_:gate:)`), `ToolRegistry` (aggregate `allDescriptors()`, dedupe by `name` first-wins, `descriptor(named:)`, `run(_:gate:)` routing to the owning contributor), and the async `ApprovalGate` seam (`awaitDecision() -> .approve/.skip/.cancel`). *Verify: `swift test` — aggregation, name dedupe, dispatch to the right contributor, unknown tool returns a defensive `.failed`/skip.* -- [x] 3.2 Add a `ScriptedApprovalGate` test double (deterministic approve/skip/cancel queue). *Verify: used by §4/§6 tests.* - -## 4. Bridge to the existing task machinery (the load-bearing reuse) - -- [x] 4.1 Add `AI/Agent/TaskKindToolContributor.swift`: build `ToolDescriptor`s for the existing `TaskKind`s (calendar/reminder/contact = `.confirm`; saveToProject/openToolWithPayload/sendTo carry their bound config in the descriptor identity per Decision Q2; default `.confirm`). *Verify: `swift test` — one descriptor per authored task; config round-trips to the right `TaskKind`.* -- [x] 4.2 Implement the args→`resolvedPrompt` fold: render `ToolRoute.argumentsJSON` into the prompt the existing `TaskDispatching.prepare` consumes (args are a hint; the kind's `ParsedActions` schema stays the authority). DO NOT modify `TaskDispatcher`/`ParsedActions`/`TaskSinks`. *Verify: `swift test` with a fake `TaskDispatching` asserting the folded prompt; confirm zero changes to existing task files (`git diff` empty for those).* -- [x] 4.3 Map `TaskReview` → `ToolStepResult` + gate: `.declined`→`.declined`; `.unavailable`→`.failed(headline)`; `.action` with effective `.auto`→`execute` now→`.done` (or `.failed` if `execute` throws — never a false Done); `.action` with `.confirm`/`.dangerous`→`.awaitingApproval`, await the gate, approve→`execute`, skip→`.declined("skipped")`, cancel→ends the loop (not a failure). *Verify: `swift test` — each branch with scripted dispatcher + gate.* - -## 5. Write-policy gate + candidate retrieval - -- [x] 5.1 Add `AI/Agent/WritePolicyResolving.swift`: the `WritePolicyResolving` seam + `DescriptorWritePolicy` stand-alone default (returns the descriptor's own tier), so the slice compiles + tests without `ai-background-autonomy`. *Verify: `swift test`.* -- [x] 5.2 Add `AI/Agent/ToolCandidateSource.swift`: `ToolCandidateSource` protocol + `KeywordToolCandidateSource` (lexical match of the latest user turn vs descriptor `name`/`summary`/`keywords`, top-`limit` default 5, always include the active skill's allowed tools, additive cap 8). Shape mirrors `DocIndex.retrieve(query:limit:)` (blueprint §3.4). *Verify: `swift test` — ranking, top-N, skill-allowed inclusion, cap.* -- [x] 5.3 Add the `widen_candidates`/`retrieve` `ToolDescriptor` so the model can request a broader set (retrieval-as-a-routed-step); the registry's `run` for it re-feeds a wider candidate set into the next loop step. *Verify: `swift test` — a route to `widen_candidates` enlarges the next turn's candidates.* - -## 6. The bounded agent loop - -- [x] 6.1 Add `AI/Agent/AgentLoop.swift`: the pure route→execute→continue loop with `maxToolSteps` (default 8); plain-answer termination on `tool==""`; per-step `registry.run` + appended `.tool` context; `AgentLoopOutcome{answered(text), stopped(reason), failed(headline), capReached}`. *Verify: `swift test` — plain-answer one-shot, single tool step, multi-hop, cap reached.* -- [x] 6.2 Emit `route.rationale` + each `ToolStepResult.summary` into the `.thinking` channel (NO third channel); publish the ordered `[ToolStepResult]` + current `.awaitingApproval` review as observable loop state for the canvas. *Verify: `swift test` — thinking accumulates the plan; response carries only the final answer.* -- [x] 6.3 Loop-guard / no-progress: end `.stopped(.repeatedStep)` on a byte-identical consecutive route; end `.stopped(.noProgress)` on re-routing to a tool that just declined/failed > once; always stream a best-effort final answer on any termination (never a bare halt). *Verify: `swift test` — repeated-step and declined-re-route termination; cap backstop.* -- [x] 6.4 Cancellation: a gate `.cancel` / `RuntimeError.cancelled` ends the loop quietly (not `.failed`), fires no side effect. *Verify: `swift test`.* - -## 7. Cross-slice seams (compile-in-isolation today, bind later) - -- [x] 7.1 Add `AI/Agent/ConversationSeam.swift`: narrow `ConversationContext`/`ChatStreaming` protocols this slice OWNS until `ai-conversation-runtime`/`on-device-ai-runtime` commit `AgentMessage`/`AgentConversation`/`LLMChatRequest`/`chat()`; appending a `.tool` turn goes through this seam. *Verify: `swift test`; documented to be re-pointed at the real types (Open Question Q1).* -- [x] 7.2 Document the consumer seams other slices plug into: `ToolContributor` (memory / skills / handoff register here), `WritePolicyResolving` (autonomy supplies the whitelist-intersecting resolver), `ToolCandidateSource` (skills/memory supply a `DocIndex`-backed source), `ApprovalGate` (the canvas drives DOWN=approve/RIGHT=skip). *Verify: review against blueprint §5.* - -## 8. Verify - -- [x] 8.1 `swift build` + `swift test` green; the router (every outcome branch), the bridge (each `TaskReview`→`ToolStepResult`), the gate (approve/skip/cancel), candidate ranking, the loop (plain-answer / multi-hop / cap / loop-guard), and the `.thinking` plan split are all covered by scripted `StubLLMRuntime` outcomes + fakes. *Verify: `swift test`.* -- [x] 8.2 Confirm the existing task files (`TaskDispatcher.swift`, `ParsedActions.swift`, `TaskSinks.swift`, `TaskReview.swift`, `TaskDispatching.swift`, `AICommandExecutor.swift`) are **unmodified** by this slice (reuse wholesale, no fork). *Verify: `git diff --stat` shows no changes to those files.* -- [x] 8.3 `openspec validate --strict` passes; the `ai-command-tasks` delta's ADDED requirements + scenarios match the implemented contracts. *Verify: `openspec validate --strict`.* -- [x] 8.4 **No `.app` build, no signing, no permission change** performed by this slice — verification is `swift test` only; the MLX-linked real-runtime compile is owned by `ai-batched-runtime-and-context`. *Verify: review.* -- [ ] 8.5 **User run-verify** (later, after the real batched runtime + canvas land, in a stable-signed build): a conversational ask routes to the right tool, read-only steps auto-run, a side-effecting step waits for a DOWN approve / RIGHT skip, the plan shows in the Thinking section, and a spin terminates at the cap with a clean summary. *(Deferred — depends on downstream slices.)* diff --git a/openspec/changes/ai-video-animation-generation/.openspec.yaml b/openspec/changes/ai-video-animation-generation/.openspec.yaml deleted file mode 100644 index a4ac4d7..0000000 --- a/openspec/changes/ai-video-animation-generation/.openspec.yaml +++ /dev/null @@ -1,2 +0,0 @@ -schema: spec-driven -created: 2026-06-23 diff --git a/openspec/changes/ai-video-animation-generation/design.md b/openspec/changes/ai-video-animation-generation/design.md deleted file mode 100644 index 1004c7b..0000000 --- a/openspec/changes/ai-video-animation-generation/design.md +++ /dev/null @@ -1,105 +0,0 @@ -## Context - -`ai-media-runtime` (addendum §B1) owns the **second runtime seam**, `MediaRuntime`: a long async job that streams `MediaProgress` and ends in a `MediaAsset` file — NOT a token stream. It owns the value types (`MediaKind`/`MediaParameters`/`MediaRequest`/`MediaProgress`/`MediaAsset`), the `generate_image` / `generate_video` `ToolDescriptor`s, the `MediaGenSink` that executes them through the existing route→execute→continue loop, and the output surfaces (Files-band gallery entry + canvas preview/player). It does NOT own the concrete backends. - -This slice owns the **video backend behind that seam** (addendum §B3). The pinned, honest framing: - -- **Local video is a frontier, not the default.** LTXV-class video is ComfyUI/MPS, 35 GB+ weights, minutes-per-clip, and it EVICTS chat under the 48 GB budget (fleet §C1, decision §5.5 of the addendum). That is real and must be disclosed, not hidden. It belongs behind the master `fullPotentialEnabled` toggle. -- **The honest default is CLOUD escalation** (LTX Studio API / hosted), and because that spends money and uploads bytes, it is gated **exactly like the Claude handoff** (§3.8): confirm-by-default, per-day budget/rate cap over a relaunch-surviving ledger, needs-you escalation when parked, one audit record per attempt, fire-and-forget with progress. -- **One seam, swappable backends.** Both `CloudVideoRuntime` (default) and `LocalLTXVRuntime` (frontier) are `MediaRuntime` conformers advertising `capabilities = [.video]`. The fleet's `videoProvider` descriptor picks which is wired. A future local LTXV (or a different cloud vendor) drops in at the same seam — exactly as `LLMRuntime` lets Gemma be swapped, so feature code (the sink, the canvas, the Files output) never changes. - -This slice reuses the gating machinery of `ai-claude-handoff` and `ai-background-autonomy` rather than inventing a parallel one: cloud video IS a dangerous, money-spending, network-escalating tool, structurally identical to `launch_claude`. - -## Goals - -- A concrete `video`-capable `MediaRuntime` that defaults to **cloud escalation**, selected by the registry's `videoProvider`. -- Cloud video gated **`.dangerous` + budget-capped + audited + needs-you-when-parked**, reusing `WritePolicyTier` / `AuditLog` / the handoff budget pattern — NOT a new gate. -- **Honest disclosure**: the confirm step and audit summary state the bytes leave the device and the per-clip cost; the redacted summary never carries the full prompt or raw seed bytes. -- A **frontier local LTXV** backend behind `fullPotentialEnabled` + `mediaGenEnabled`, disclosing 35 GB+ residency, minutes-per-clip latency, chat eviction, and thermal cost. -- **img2video from the seed**, fire-and-forget with `MediaProgress` step previews, parking because it is slow. -- The seam-swap contract: a later backend joins the SAME `MediaRuntime` with no feature-code change. -- The **majority** of the slice (selector, budget, disclosure, tier resolution, progress plumbing, parking, error mapping) is MLX-/network-free Core, `swift test`-verified via stubs. - -## Non-Goals - -- Redefining the `MediaRuntime` seam, the `generate_video` descriptor, the `MediaGenSink`, or the Files-band / canvas output — all owned by `ai-media-runtime` and CONSUMED here. -- The image backend (`ai-local-image-generation`), the fleet residency/eviction MATH (`ai-model-fleet`), the master Hub toggle page (`ai-full-potential-toggle`). -- A new error taxonomy when `MediaError` / `RuntimeError` already carries the case; a new gating mechanism when the §3.8 budget pattern already fits. -- Pinning a specific hosted vendor's exact API schema beyond "upload prompt + optional seed, poll progress, fetch a file"; any non-LTXV / non-LTX-Studio backend. -- A structured round-trip beyond fire-and-forget-with-progress; video editing (trim/splice) of the result. -- Any degraded / low-end / Intel path. M5 floor (M4 min). No regressions. - -## Decisions - -### 1. The video backend is a `MediaRuntime` conformer, not a new seam. - -`CloudVideoRuntime` and `LocalLTXVRuntime` both conform to the `MediaRuntime` protocol (owned by `ai-media-runtime`), advertise `capabilities = [.video]`, and implement `generate(_:) -> AsyncThrowingStream`. The `MediaGenSink` and canvas/Files surfaces are unchanged. - -- **Rationale:** the addendum pins exactly one media seam and demands "build it so a local LTXV drops into the SAME `MediaRuntime` later, exactly as `LLMRuntime` lets Gemma be swapped." A second protocol would fork the seam and break the swap contract. -- **Alternatives rejected:** (a) a dedicated `VideoRuntime` protocol — forks the seam, duplicates the progress/asset plumbing, violates the pinned reuse rule. (b) Folding video into the image runtime — different residency class, different lane behavior (evicts chat), different gating tier; conflating them hides the cost. - -### 2. The default video backend is CLOUD escalation; local is the frontier exception. - -`videoProvider` defaults to `.cloud` (a `CloudVideoRuntime` over a hosted API). `.localLTXV` is selectable only when `fullPotentialEnabled` AND `mediaGenEnabled` are on. - -- **Rationale:** addendum §B3 + decision §5/§6 — "Honest default = CLOUD escalation … Local LTXV is a FRONTIER option behind the full-potential toggle." Cloud keeps the calm default (no 35 GB download, no eviction, no fans) and matches the project's honest-surface ethos; local is the deliberate "release full potential." -- **Alternatives rejected:** (a) local-LTXV default — dishonest: it would silently download 35 GB, evict chat on first use, and pin the GPU for minutes; the opposite of "ships calm." (b) cloud-only, no local path — abandons the frontier user and the swap contract that proves the seam is backend-agnostic. - -### 3. Cloud video is `.dangerous` + budget-capped, mirroring the Claude handoff (NOT the image tool's `.confirm`). - -When `videoProvider == .cloud`, the effective gate for `generate_video` is **dangerous**: confirm-by-default per call, escalates to **needs-you** when parked, runs only when the user approves. A pure `VideoBudget` enforces a **rolling-24h** call cap + a concurrency cap over an **append-only ledger that survives relaunch**, `now:`-injected (deterministic). An over-budget call **never silently drops** — it degrades to a foreground confirm (active) or needs-you (parked) and states the cap was reached. A failed launch **refunds** its spend so the cap stays honest. - -- **Rationale:** cloud video spends real money AND uploads bytes — strictly more dangerous than `launch_claude`, which is "dangerous." The addendum says gate it "EXACTLY like the Claude handoff." Reusing the §3.8 budget/escalation pattern (rolling window, ledger, refund, degrade-not-drop) gives the same runaway-spend protection with one audited mechanism. -- **Alternatives rejected:** (a) `.confirm` like the image tool — image gen is free + local; cloud video is neither; a one-tap confirm with no cap is a runaway-spend hole. (b) A calendar-day reset — gameable across midnight (the handoff spec already rejected this); a rolling 24h window keyed off injected time is the adopted rule. (c) Silently dropping over-budget calls — the blueprint forbids silent side-effect loss; degrade-to-confirm keeps the user the only authority that spends over the cap. - -### 4. Honest upload + cost disclosure at the confirm step and in the audit summary. - -Because cloud video sends the prompt (and any seed frame) off-device, the confirm step states the bytes leave the device and the per-clip $ order; the **redacted** audit summary carries the provider, a **truncated** prompt, and a **seed-image-present flag** — never the full prompt verbatim, never the raw seed bytes. Local-LTXV discloses 35 GB+ residency, minutes-per-clip latency, chat eviction, and thermal cost at selection. - -- **Rationale:** the project's honest-surface ethos (addendum §D1 disclosure UX) + the blueprint redaction rule (raw text only in logs / behind an opt-in details disclosure, never in a headline / audit summary). A remote upload is a privacy event the user must consent to knowingly. -- **Alternatives rejected:** (a) a generic "generating video…" with no upload notice — hides a privacy + spend event. (b) Putting the full prompt in the audit summary — violates the redaction rule and can leak secrets. - -### 5. img2video from the seed; fire-and-forget with progress; parks because it is slow. - -Both backends consume the optional `MediaRequest.seed` (PNG) as the first frame for img2video. `generate(_:)` streams `MediaProgress.step(index:total:preview:)` while the remote/local job runs and ends in `MediaProgress.finished(MediaAsset)` with `kind == .video` and `durationMs != nil`. Because the job is minutes-long, it **parks** via `ParkScheduler`; the notch glows on completion / `needsYou`. The finished clip lands as a Files-band asset + a canvas player; **swipe-DOWN** extracts it (the canonical compass). - -- **Rationale:** addendum §B1/§B3 + decision §4 — "Media is a tool, parked because it's slow … Output is a Files-band asset"; the seed field exists precisely so a screen-region/clipboard capture becomes the first frame. Streaming step previews keeps the slow job observable (failure is observable state, never silence). -- **Alternatives rejected:** (a) a blocking call returning only the final file — a minutes-long block freezes the agent and gives no progress; parking + a stream is the pinned model. (b) Building a new video player overlay — reuse the canvas player + `DockPreviewOverlay` non-activating / synchronous-`orderOut` pattern (owned upstream), do not reinvent. - -### 6. Local video is OFF the cloud spend/budget path but still audited and still parks. - -`LocalLTXVRuntime` spends no money and uploads nothing, so it does NOT consume the `VideoBudget` ledger and is NOT `.dangerous` on the spend axis. It is still gated by the master toggle (frontier), still emits exactly one audit record per attempt, and still parks (it is slow). Its disclosure is residency/heat/eviction, not $. - -- **Rationale:** the budget cap exists to bound real spend; local video has none. But "failure is observable, every attempt audited" still binds, and local video is still minutes-long so it parks. Conflating "dangerous" (spend) with "expensive" (compute) would mis-gate local video. -- **Alternatives rejected:** charging local video against the cloud budget — nonsensical; the budget is a $ cap. Skipping its audit — violates "every tool step writes one audit record." - -### 7. Errors map into the shared `MediaError` at the boundary; a video enum only if needed. - -Vendor/OS failures — `NSURLError` (cloud upload/poll), `Process`/ComfyUI failures (local), over-budget, provider-disabled, upload-declined — map into the `MediaError` taxonomy (owned by `ai-media-runtime`) at the layer boundary inside each runtime. A video-only `enum` is added ONLY if `MediaError` / `RuntimeError` cannot carry a case (e.g. `.videoBudgetExceeded`, `.videoProviderDisabled`). Everything surfaces through the single `AIError.message(for:)` translator, bounded + non-blocking — never `NSAlert.runModal`, never raw OS/vendor text in a headline (raw text only in logs / behind a Show-details disclosure). A side effect that did not land (no clip written, upload failed) becomes `.failed`, never a false "Done." - -- **Rationale:** blueprint invariant — one taxonomy, one translator, mapped at the boundary, surfaced bounded + non-blocking. Core stays MLX-/network-free, so it cannot see `NSURLError` / `Process` types; only the taxonomy crosses into feature code. -- **Alternatives rejected:** a standalone video error surface / raw `NSURLError.localizedDescription` in the canvas — violates the one-translator + redaction rules; an `NSAlert` would freeze the Settings window (the bounded + non-blocking rule). - -### 8. The swap-in contract is a first-class requirement. - -The slice specifies that a later video backend (a different cloud vendor, or in-process local video when feasible) joins the SAME `MediaRuntime` seam, selected by `videoProvider`, with no change to the sink, canvas, Files output, gating, or budget code. - -- **Rationale:** addendum §B3 explicitly requires "build the seam so a local LTXV backend drops into the SAME `MediaRuntime` later." Making it a spec requirement (not just prose) lets a future slice verify the seam held. -- **Alternatives rejected:** leaving it implicit — risks a future backend bolting on a parallel path and forking the seam. - -## Per-component target-split & verification - -| Component | Target | Verification | -|---|---|---| -| `VideoProvider` selector value (`.cloud` / `.localLTXV`) backing `videoProvider` | Core (MLX-/network-free) | `swift test` — round-trips Codable, default `.cloud`, `.localLTXV` requires master toggle | -| `VideoBudget` (pure rolling-24h rate/concurrency cap over append-only ledger, `now:`-injected) | Core | `swift test` — under-cap allows, rolling window not gamed across midnight, survives relaunch (ledger replay), failed launch refunds, over-budget degrades not drops | -| Effective-tier resolution for `generate_video` (cloud → `.dangerous`; local → master-gated) | Core | `swift test` — cloud resolves dangerous, ∩ user whitelist, local gated by `fullPotentialEnabled` + `mediaGenEnabled` | -| `VideoUploadDisclosure` (does-bytes-leave + redacted summary builder: provider + truncated prompt + seed-present flag) | Core | `swift test` — summary never contains full prompt or raw seed bytes; upload flag true for cloud, false for local | -| Audit emission (exactly one `AuditRecord` per attempt: done/declined/failed/over-budget) | Core | `swift test` — one record per resolution, redacted summary, `wasBackground` set when parked | -| Parking + `MediaProgress` plumbing (stream step previews → finished, escalate/needsYou when slow) | Core | `swift test` with `StubCloudVideoRuntime` / `StubLocalVideoRuntime` (scripted progress → finished) | -| `MediaError` boundary mapping for video (over-budget / provider-disabled / upload-declined; + any video-only enum) | Core | `swift test` — vendor/OS surrogates map to taxonomy; routes through `AIError.message(for:)`; no raw text in headline | -| `CloudVideoRuntime` (real `URLSession` upload/poll against hosted API; maps `NSURLError` at boundary) | Native-linked (`GemmaRuntime`/sibling framework) | `xcodebuild` COMPILE-VERIFY only for an agent; real upload/poll/spend/latency verifiable ONLY in the user's stable-signed build | -| `LocalLTXVRuntime` (ComfyUI/MPS process bridge; maps `Process`/ComfyUI errors at boundary) | Native-linked | `xcodebuild` COMPILE-VERIFY only; real 35 GB residency, minutes-per-clip, chat eviction, heat verifiable ONLY in the user's stable-signed build | -| Canvas video player / reveal surface (reuses `DockPreviewOverlay` non-activating + synchronous `orderOut`; owned upstream) | Native-linked (consumed) | `xcodebuild` compile-verify; live player behavior in the user's stable-signed build | - -An agent NEVER builds/signs/installs the `.app` (ad-hoc signing breaks TCC; the `*.bundle` metallib copy in `build-app.sh` must not regress). Real video — a real clip, real $ spend, real eviction, real thermals — is the user's stable-signed build's job. diff --git a/openspec/changes/ai-video-animation-generation/proposal.md b/openspec/changes/ai-video-animation-generation/proposal.md deleted file mode 100644 index 6e33181..0000000 --- a/openspec/changes/ai-video-animation-generation/proposal.md +++ /dev/null @@ -1,28 +0,0 @@ -## Why - -The generative-media seam (`MediaRuntime`, addendum §B1) lets the agent paint **images** locally on the GPU — fast, private, free, fitting alongside chat in the 48 GB budget. **Video is a different animal.** A few seconds of LTXV-class diffusion is **minutes of work and tens of gigabytes of weights** (LTXV ComfyUI graphs run 35 GB+, not an in-process MLX model), it **evicts chat and pins the GPU**, and it makes the fans scream. To pretend a laptop generates video the way it generates an image would be dishonest — and it would silently degrade the calm default the rest of V2 ships with. - -So this slice is honest about the cost. The **default video backend is a CLOUD escalation** (LTX Studio's hosted API / equivalent), gated **exactly like the Claude handoff** (addendum §B3, blueprint §3.8): off until the user opts in, **confirm-by-default** because real money and a remote upload are on the line, **per-day budget + rate capped** so an autonomous loop physically cannot rack up spend, **fully audited**, and **fire-and-forget with step progress** so the slow remote job parks and the notch glows on completion. The **local LTXV backend is a FRONTIER option** behind the master full-potential toggle (addendum §D1) — for the user who *wants* to spend 35 GB and minutes-per-clip and thermal headroom, disclosed in the same breath it is offered. - -Crucially, this is **one seam, two interchangeable backends**: both the cloud-escalation runtime and the (frontier) local LTXV runtime are concrete `MediaRuntime` conformers (`video` capability), selected by the model fleet's `videoProvider` descriptor — built so a future local LTXV drops into the SAME seam without touching feature code, mirroring exactly how `LLMRuntime` lets Gemma be swapped. img2video flows from the **seed image** already carried on `MediaRequest.seed` (the screen-region / clipboard capture becomes the first frame). - -## What Changes - -- **A `video`-capable `MediaRuntime`, defaulting to cloud escalation.** This slice contributes the concrete video backend(s) behind the `MediaRuntime` seam (owned by `ai-media-runtime`, addendum §B1) — it does NOT redefine the seam, the `generate_video` `ToolDescriptor`, the `MediaGenSink`, or the Files-band/canvas output (all owned by `ai-media-runtime`). The default conformer is a **hosted-API runtime** (`CloudVideoRuntime`); the registry's `videoProvider` selects it. -- **Cloud video is `.dangerous` + budget-capped, mirroring the Claude handoff.** The `generate_video` tool's effective gate is **dangerous** when `videoProvider` is cloud (not the image tool's `.confirm`): confirm-by-default per call, a **per-rolling-24h budget/rate cap** over an append-only ledger that survives relaunch, **needs-you escalation when parked**, and **exactly one audit record per attempt** (done / declined / failed / over-budget) — reusing `WritePolicyTier` + `AuditLog` (§3.7) and the handoff budget pattern (§3.8), NOT a new gating mechanism. A persisted `mediaVideoBudgetPerDay` key (addendum §1) caps it; a persisted `videoProvider` key selects the backend. -- **Honest upload + cost disclosure.** Because cloud video sends the prompt (and any seed frame) to a remote service, the confirm step and the audit summary state that the bytes leave the device; the budget cap states the per-clip $ order. The **redacted** audit summary carries the provider, a truncated prompt, and whether a seed image was sent — never the full prompt, never the raw seed bytes. -- **Local LTXV is a frontier backend behind the master toggle.** A `LocalLTXVRuntime` (a second `MediaRuntime` video conformer over a ComfyUI/MPS graph, native-linked, NOT in-process MLX) is selectable only when `fullPotentialEnabled` AND `mediaGenEnabled` are on; selecting it discloses the 35 GB+ residency, the minutes-per-clip latency, the **chat eviction** (the fleet, §C1, decides residency — video evicts chat; this slice consumes that decision and surfaces "the assistant is busy painting"), and the thermal cost. Local video is **off the cloud budget/audit-spend path** (no money leaves) but still emits an audit record and still parks. -- **img2video from the seed; fire-and-forget with progress; parks because it is slow.** Both backends consume the optional `MediaRequest.seed` (PNG) as the first frame for img2video, stream `MediaProgress.step(index:total:preview:)` while the remote/local job runs, end in a `MediaProgress.finished(MediaAsset)` whose `kind == .video` / `durationMs != nil`, and **park** via `ParkScheduler` (owned elsewhere) so the slow job runs in the background and the notch glows on completion / `needsYou`. - -## Capabilities - -### Modified Capabilities - -- `ai-generative-media`: ADDS the video-backend requirements behind the existing `MediaRuntime` seam — the cloud-escalation default video runtime gated dangerous + budget-capped + audited (the Claude-handoff template applied to video), the honest upload/cost disclosure, the frontier local-LTXV backend behind the master toggle with its eviction/thermal disclosure, img2video from the seed, and the swap-in contract that a later local backend joins the SAME seam. (The seam, value types, `generate_video` descriptor, sink, and Files/canvas output are ADDED by `ai-media-runtime`; this slice's delta is purely the video backend behavior and its dangerous-tier gating.) - -## Impact - -- **Code (MLX-free Core, `AI/Media/Video/`):** pure value types this slice OWNS — a `VideoProvider` selector value (`cloud` / `localLTXV`) backing the `videoProvider` key; a `VideoBudget` (pure rolling-24h rate/budget gate over an append-only ledger, `now:`-injected — structurally the `HandoffBudget` pattern, reused not reinvented); a `VideoUploadDisclosure` value (does-bytes-leave + redacted summary builder); the effective-tier resolution for `generate_video` (cloud → `.dangerous`, local → still gated by the master toggle). A `StubCloudVideoRuntime` + `StubLocalVideoRuntime` (scripted progress → finished) make the gating, budget, disclosure, parking, and progress-plumbing all `swift test`-verified without weights or network. New errors map into the shared `MediaError` taxonomy (owned by `ai-media-runtime`) at the boundary; a video-only `enum` is added ONLY if `MediaError`/`RuntimeError` cannot carry an over-budget / provider-disabled / upload-declined case. -- **Native-linked (`xcodebuild` COMPILE-VERIFY ONLY for an agent):** `CloudVideoRuntime` (the real `URLSession` upload/poll against the hosted API — maps `NSURLError` at the boundary into `MediaError`); `LocalLTXVRuntime` (the ComfyUI/MPS process bridge — maps `Process`/ComfyUI errors at the boundary). Real correctness — a real clip, real latency, real eviction, real heat, real spend — is verifiable ONLY in the **user's stable-signed build** (an agent never builds/signs the `.app`; ad-hoc signing breaks TCC, and the `*.bundle` metallib copy in `build-app.sh` must not regress). -- **Reuse, not rebuild:** the `MediaRuntime` seam + `MediaRequest`/`MediaProgress`/`MediaAsset`/`MediaParameters`/`MediaKind` value types + `generate_video` `ToolDescriptor` + `MediaGenSink` + Files-band/canvas output (all CONSUMED, owned by `ai-media-runtime`); `WritePolicyTier`/`AuditRecord`/`AuditLog`/`WritePolicyResolving` (CONSUMED, owned by `ai-background-autonomy`); the budget/rate-cap pattern + needs-you-when-parked escalation (the `ai-claude-handoff` template, mirrored); `ParkScheduler.escalate`/`runnableSessions` (CONSUMED, owned by `ai-parked-sessions`); `ModelRegistry`/`ModelDescriptor`/`ModelRole.video`/`ModelProvider`/`ComputeLane` (CONSUMED, owned by `ai-model-fleet` / `ai-compute-tiers`); `fullPotentialEnabled`/`mediaGenEnabled`/`fleetCloudEscalationEnabled` (CONSUMED, owned by `ai-full-potential-toggle`); `AIError.message(for:)` extended to translate any video error (the one translator); the canonical two-finger compass (swipe-DOWN extracts the finished clip — owned by `ai-conversational-canvas`). -- **Out of scope:** the `MediaRuntime` seam / tool / sink / output surfaces themselves (owned by `ai-media-runtime`); the image backend (`ai-local-image-generation`); the fleet residency/eviction MATH (consumed from `ai-model-fleet`); the master Hub toggle page (owned by `ai-full-potential-toggle`); choosing the specific hosted provider's API shape beyond "upload prompt+optional seed, poll progress, fetch a file" and any non-LTXV/non-LTX-Studio backend; a structured round-trip beyond fire-and-forget-with-progress; video EDITING (trim/splice) of the result — it lands as a Files-band asset and is handed off like any file. diff --git a/openspec/changes/ai-video-animation-generation/specs/ai-generative-media/spec.md b/openspec/changes/ai-video-animation-generation/specs/ai-generative-media/spec.md deleted file mode 100644 index d1f3c81..0000000 --- a/openspec/changes/ai-video-animation-generation/specs/ai-generative-media/spec.md +++ /dev/null @@ -1,159 +0,0 @@ -## ADDED Requirements - -> These requirements ADD the **video backend** behavior behind the `MediaRuntime` seam. The seam, the -> `MediaKind`/`MediaParameters`/`MediaRequest`/`MediaProgress`/`MediaAsset` value types, the -> `generate_video` tool descriptor, the `MediaGenSink`, and the Files-band / canvas output are ADDED by -> `ai-media-runtime` and are CONSUMED (not redefined) here. Every requirement below is video-backend -> behavior layered onto that seam. - -### Requirement: Video generation is a MediaRuntime backend with a cloud-escalation default - -The system SHALL provide video/animation generation as a concrete `MediaRuntime` backend advertising the `video` capability, selected by a persisted **video provider** setting. The default provider SHALL be **cloud escalation** (a hosted video API) and SHALL NOT download or run a local video model by default. The backend SHALL NOT introduce a second runtime seam, a second media tool, or a second output path: it SHALL plug into the existing `MediaRuntime` seam and the existing `generate_video` tool, and a later video backend (a different hosted provider, or a local one) SHALL join the SAME seam selected by the same provider setting, with no change to the tool, the executor/sink, the Files-band output, the canvas output, the gating, or the budget code. - -#### Scenario: Default provider is cloud, nothing downloaded - -- **WHEN** media generation is enabled and the user has not chosen a video provider -- **THEN** the active video backend is the cloud-escalation provider, and no local video weights are downloaded or made resident - -#### Scenario: A second backend joins the same seam - -- **WHEN** a later video backend is added and selected via the video provider setting -- **THEN** it conforms to the same `MediaRuntime` seam and advertises the `video` capability, and the `generate_video` tool, the sink, the Files-band and canvas output, the gating, and the budget code are unchanged - -### Requirement: Cloud video is a dangerous-tier, budget-capped action mirroring the Claude handoff - -When the video provider is the cloud escalation, the effective gate for `generate_video` SHALL be the **dangerous** write-policy tier — confirm-by-default per call — because real money is spent and bytes leave the device. The effective gate SHALL be the descriptor's dangerous tier intersected with the user's whitelist; the system SHALL enforce a **per-rolling-24-hour** call cap and a **maximum concurrent in-flight** cap over an append-only spend ledger that survives a relaunch within the window, keyed off an injected current time (NOT a calendar-day reset). A per-clip cap MAY tighten the global cap; a per-skill cap of zero SHALL fall back to the global default. A cloud video whose launch fails after a spend was recorded SHALL refund that spend so the cap stays honest. An over-budget call SHALL NEVER be silently dropped. - -#### Scenario: Cloud video defaults to confirm at the dangerous tier - -- **WHEN** `generate_video` is selected with the cloud provider and no explicit auto opt-in -- **THEN** the call requires a foreground per-call approval before any upload or spend occurs, at the dangerous tier - -#### Scenario: Under the cap a cloud video is allowed - -- **WHEN** the number of cloud videos in the last 24 hours is below the cap and concurrency is below its limit -- **THEN** the generation is allowed to proceed (subject to its confirm/auto gate) - -#### Scenario: Rolling window cannot be gamed across midnight - -- **WHEN** cloud videos are spread across a calendar-day boundary but fall within the same rolling 24-hour window -- **THEN** they are counted together against the cap, with no midnight reset - -#### Scenario: Cap survives a relaunch - -- **WHEN** the process restarts within the rolling window -- **THEN** prior cloud videos inside the window still count against the cap and the budget is not reset - -#### Scenario: A failed cloud launch refunds its spend - -- **WHEN** a cloud video records a spend but its upload/launch then fails -- **THEN** the spend is refunded and the in-flight count is decremented, leaving the cap unchanged - -### Requirement: Over-budget cloud video degrades, never runs unprompted - -When a cloud video is over the budget cap, it SHALL degrade to a foreground per-call confirmation (in an active session) rather than running unprompted, and SHALL escalate to the needs-you badge (in a parked session) — it SHALL NOT auto-run over budget and SHALL NOT be silently dropped. The user, never the loop, SHALL be the only authority that can spend over the cap, and the approval surface SHALL indicate that the budget cap has been reached. - -#### Scenario: Over budget in an active session degrades to confirm - -- **WHEN** a cloud video is requested but the daily cap has been reached and the session is active -- **THEN** the request degrades to a foreground confirmation that states the budget cap was reached, and it does not auto-run - -#### Scenario: Over budget in a parked session escalates - -- **WHEN** a cloud video is over budget and the session is parked -- **THEN** the request escalates to a needs-you badge for the user to decide, and no upload or spend occurs until the user returns and approves - -### Requirement: Cloud video discloses that bytes leave the device and the cost - -Because cloud video sends the prompt and any seed frame to a remote service, the confirmation surface SHALL state that the bytes leave the device and the per-clip cost order, before any upload occurs. The disclosure SHALL be derived from a pure value (does-bytes-leave plus a redacted summary) so it is testable without network. A local video backend SHALL set the does-bytes-leave flag false (nothing is uploaded). - -#### Scenario: Cloud confirm states the upload and cost - -- **WHEN** a cloud video awaits confirmation -- **THEN** the confirmation surface states that the prompt and any seed image will be sent to the remote service and indicates the per-clip cost order - -#### Scenario: Local video does not claim an upload - -- **WHEN** the active video provider is local -- **THEN** the does-bytes-leave flag is false and no upload disclosure is presented - -### Requirement: Every video attempt is audited with a redacted summary - -Every video attempt — confirmed, auto, declined, failed, or over-budget, for the cloud OR the local provider — SHALL emit exactly one append-only audit record naming the tool, the policy tier, a redacted/short argument summary (the provider, a truncated prompt, and a flag for whether a seed image was sent — NEVER the full prompt verbatim, NEVER the raw seed bytes), the outcome, whether it ran in the background, and a timestamp. The audit SHALL route into the single shared audit log rather than a separate video-only log. Raw prompt text and raw seed bytes SHALL appear only in logs or behind an opt-in details disclosure, never in the audit summary or any headline. - -#### Scenario: One audit record per video attempt - -- **WHEN** any `generate_video` attempt resolves (done, declined, failed, or over-budget) -- **THEN** exactly one audit record is appended naming the tool, the tier, the redacted argument summary, the outcome, the background flag, and a timestamp - -#### Scenario: The audit summary never carries the full prompt or raw seed - -- **WHEN** an audit record for a video attempt is inspected -- **THEN** its argument summary contains the provider, at most a truncated prompt, and a seed-present flag, and neither the full prompt text nor the raw seed bytes are present - -### Requirement: Local LTXV video is a frontier backend behind the master full-potential toggle - -The local LTXV video backend SHALL be selectable ONLY when the master full-potential toggle AND media generation are both enabled. It SHALL NOT be downloaded, made resident, or selectable by default. Selecting it SHALL disclose, in the same breath it is offered, its residency cost (tens of gigabytes), its minutes-per-clip latency, that a video generation EVICTS the chat model (the assistant goes quiet while it paints, per the fleet residency decision), and its thermal cost. Local video SHALL NOT consume the cloud spend ledger or the cloud budget cap (no money is spent), but it SHALL still be audited and SHALL still park because it is slow. - -#### Scenario: Local LTXV requires the master toggle - -- **WHEN** the master full-potential toggle is off -- **THEN** the local LTXV provider is not selectable and is not downloaded, and the cloud provider remains the only video backend - -#### Scenario: Selecting local LTXV discloses its cost - -- **WHEN** the user selects the local LTXV provider with the master toggle on -- **THEN** the selection discloses the tens-of-gigabytes residency, the minutes-per-clip latency, that chat is evicted while it generates, and the thermal cost - -#### Scenario: Local video is audited and parks but spends no budget - -- **WHEN** a local LTXV video is generated -- **THEN** it emits one audit record and parks while it runs, and it consumes no cloud spend ledger entry and no cloud budget cap - -### Requirement: img2video from the seed, fire-and-forget with step progress - -A video generation SHALL consume the optional seed image carried on the request (a PNG — the screen-region or clipboard capture) as the first frame for image-to-video, and SHALL also support a prompt-only generation when no seed is provided. Because the job is minutes-long, it SHALL be fire-and-forget with progress: it SHALL stream diffusion/render step progress (optionally with a preview frame) and SHALL end in a finished media asset whose kind is video and whose duration is set. The slow job SHALL park via the park scheduler so it runs in the background, and the completion (or a needs-you state) SHALL surface on the notch. The finished clip SHALL land as a Files-band asset and a canvas player, and a swipe-down SHALL extract it per the canonical two-finger compass. - -#### Scenario: A seed image becomes the first frame - -- **WHEN** `generate_video` is run with a seed PNG -- **THEN** the seed is used as the first frame for image-to-video and the upload disclosure flags that a seed image was sent - -#### Scenario: Progress streams then finishes as a video asset - -- **WHEN** a video generation runs -- **THEN** it streams step progress (optionally with a preview) and ends in a finished media asset whose kind is video and whose duration is set, which lands as a Files-band asset and a canvas player - -#### Scenario: A slow video parks and glows on completion - -- **WHEN** a video generation is dispatched -- **THEN** it parks so it runs in the background, and on completion (or when it needs the user) the notch surfaces it - -#### Scenario: Swipe-down extracts the finished clip - -- **WHEN** a finished video is shown in the canvas player and the canvas is at the top -- **THEN** a two-finger swipe-down extracts the clip (save / paste / set-as), per the canonical compass - -### Requirement: Video failures surface bounded and non-blocking through the one translator - -Video backend failures SHALL map at the layer boundary into the shared media error taxonomy (vendor/OS errors — the hosted API's network errors, the local ComfyUI/process errors, over-budget, provider-disabled, upload-declined — converted where they cross into app code) and SHALL surface through the single media error→message translator, bounded and non-blocking. A failure SHALL NEVER be presented via an app-modal alert and SHALL NEVER place raw OS or vendor error text in a headline (raw text is allowed only in logs or behind an opt-in details disclosure). A video that did not produce a written clip SHALL become an observable failed state carrying a clean headline — never a false "Done" — and, for the cloud provider, its spend SHALL be refunded. A user discard during approval SHALL end the loop quietly with no upload, no spend, and SHALL NOT be reported as a failure. - -#### Scenario: A failed upload surfaces a clean failed state - -- **WHEN** a cloud video upload or poll fails -- **THEN** the generation is reported failed with a clean headline, the spend is refunded, and no app-modal alert appears - -#### Scenario: A failed local render surfaces a clean failed state - -- **WHEN** the local LTXV process fails to produce a clip -- **THEN** the generation is reported failed with a clean headline (not a false Done), and no app-modal alert appears - -#### Scenario: No raw error text in a headline - -- **WHEN** any video failure is presented -- **THEN** the headline is a clean, human-readable message and any raw OS/vendor error text appears only in logs or behind an opt-in details disclosure - -#### Scenario: Discard during approval is quiet, not a failure - -- **WHEN** the user discards the canvas while a video generation awaits approval -- **THEN** the loop ends quietly, nothing is uploaded, no spend occurs, and it is not reported as a failure diff --git a/openspec/changes/ai-video-animation-generation/tasks.md b/openspec/changes/ai-video-animation-generation/tasks.md deleted file mode 100644 index dfa5349..0000000 --- a/openspec/changes/ai-video-animation-generation/tasks.md +++ /dev/null @@ -1,70 +0,0 @@ -# Tasks — ai-video-animation-generation - -> Backend-only slice behind the `MediaRuntime` seam (owned by `ai-media-runtime`). Depend on the -> ADDENDUM's pinned contracts, not on sibling change files existing yet. OpenSpec markdown is authored -> in this run; the Swift below is the IMPLEMENTATION plan a later coding run follows. No `.app` -> build/sign by an agent. - -## 1. Value types (Core, owned here) - -- [x] 1.1 Add `VideoProvider` (`.cloud` default / `.localLTXV`) backing the `videoProvider` key; `.localLTXV` is only valid when `fullPotentialEnabled` + `mediaGenEnabled`. — Verify: `swift test` Codable round-trip + default `.cloud` + invalid-without-master guard. -- [x] 1.2 Add `VideoUploadDisclosure` (does-bytes-leave flag + redacted-summary builder: provider, truncated prompt, seed-present flag). — Verify: `swift test` summary never contains the full prompt or raw seed bytes; upload flag true for cloud, false for local. -- [x] 1.3 Add the persisted keys `videoProvider` + `mediaVideoBudgetPerDay` (camelCase, agent-scoped, per addendum §1) to AppSettings; defaults = cloud / a conservative per-day cap. — Verify: `swift test` defaults + persistence round-trip. - -## 2. Budget / rate cap (Core, the §3.8 pattern reused) - -- [x] 2.1 Add `VideoBudget`: pure rolling-24h call cap + concurrency cap over an append-only ledger, `now:`-injected. — Verify: `swift test` under-cap allows. -- [x] 2.2 Rolling window keyed off injected time, NOT a calendar reset (cannot be gamed across midnight). — Verify: `swift test` cross-midnight-same-window counts together. -- [x] 2.3 Ledger survives relaunch within the window (replay). — Verify: `swift test` reconstruct ledger → prior calls still count. -- [x] 2.4 A failed launch refunds its spend + decrements in-flight; an over-budget call degrades (never silently drops). — Verify: `swift test` refund leaves cap unchanged; over-budget returns a degrade signal, not a drop. - -## 3. Effective-tier resolution (Core) - -- [x] 3.1 Resolve `generate_video`'s effective tier: cloud → `.dangerous` ∩ user whitelist; local → master-toggle-gated (not on the spend axis). — Verify: `swift test` cloud=dangerous, local gated by `fullPotentialEnabled` + `mediaGenEnabled`. -- [x] 3.2 Over-budget cloud video degrades to foreground confirm (active) / needs-you (parked), stating the cap was reached. — Verify: `swift test` active→confirm-with-cap-message, parked→needsYou; never auto-run over budget. - -## 4. Audit (Core, reuses §3.7 AuditLog) - -- [x] 4.1 Emit exactly one `AuditRecord` per attempt (done / declined / failed / over-budget) into the shared `AuditLog`, with the redacted summary + `wasBackground` set when parked. — Verify: `swift test` one record per resolution; summary carries provider + truncated prompt + seed-present flag, never the full prompt. - -## 5. Backend conformers (Core stubs + native-linked real) - -- [x] 5.1 `StubCloudVideoRuntime` + `StubLocalVideoRuntime`: `MediaRuntime` conformers, `capabilities = [.video]`, scripted `MediaProgress.step` → `.finished(MediaAsset kind: .video, durationMs:)`. — Verify: `swift test` progress plumbing + finished asset shape. -- [x] 5.2 img2video: both stubs consume the optional `MediaRequest.seed` (PNG) as the first frame; a seed-present run sets the disclosure flag. — Verify: `swift test` seed threads through to the asset + disclosure. -- [x] 5.3 `CloudVideoRuntime` (native-linked): real `URLSession` upload prompt+optional seed, poll progress, fetch the file; map `NSURLError` at the boundary into `MediaError`. — Verify: `xcodebuild` compile-verify only; real upload/poll in the USER's stable-signed build. -- [x] 5.4 `LocalLTXVRuntime` (native-linked): ComfyUI/MPS process bridge, 35 GB+ graph; map `Process`/ComfyUI errors at the boundary into `MediaError`. — Verify: `xcodebuild` compile-verify only; real residency/latency/eviction/heat in the USER's stable-signed build. - -> NOTE (5.3 / 5.4 / 6.2 native player): the native-linked runtimes + the canvas video PLAYER are written -> COMPLETE in `Sources/GemmaRuntime/` and marked `// FLAGGED: user xcodebuild + stable-signed build`. The -> agent verified Core under `swift build` + `swift test` (and `swift build` compiled the GemmaRuntime -> target here). REAL correctness — a real clip, a real upload + $ spend, real 35 GB residency / minutes-per- -> clip / chat eviction / thermals, and the live player surface — is the USER's stable-signed run-verify -> (an agent never builds/signs/installs the `.app`; ad-hoc signing breaks TCC). - -## 6. Parking + output (Core wiring; native player consumed) - -- [x] 6.1 Video generation parks via `ParkScheduler` (slow); the notch glows on completion / `needsYou`. — Verify: `swift test` with a fake scheduler — slow job parks, completion raises the glow signal. -- [x] 6.2 The finished clip lands as a Files-band asset + a canvas player (CONSUMED, owned by `ai-media-runtime`); swipe-DOWN extracts it (canonical compass). — Verify: `swift test` the sink hands the `MediaAsset` to the Files/canvas seam; native player compile-verifies under `xcodebuild`. - -## 7. Disclosure UX (Core values; surface consumed) - -- [x] 7.1 Cloud confirm step states bytes leave the device + per-clip $ order (from `VideoUploadDisclosure` + `mediaVideoBudgetPerDay`). — Verify: `swift test` disclosure values; the live confirm surface (canvas) compile-verifies under `xcodebuild`. -- [x] 7.2 Local-LTXV selection discloses 35 GB+ residency, minutes-per-clip latency, chat eviction (from the fleet §C1), thermal cost. — Verify: `swift test` the disclosure value carries these; eviction decision consumed from `ai-model-fleet`. - -## 8. Errors (Core, one taxonomy / one translator) - -- [x] 8.1 Map video failures (over-budget, provider-disabled, upload-declined, `NSURLError`, `Process`/ComfyUI) into `MediaError` at the boundary; add a video-only enum ONLY if `MediaError`/`RuntimeError` cannot carry the case. — Verify: `swift test` vendor/OS surrogates map to the taxonomy. -- [x] 8.2 Surface every video error through `AIError.message(for:)` → bounded + non-blocking (never `NSAlert.runModal`, never raw text in a headline; raw text only in logs / behind Show-details). A side effect that did not land is `.failed`, never a false Done. — Verify: `swift test` the translator path; no raw text in headline; failed-not-done on a dropped clip. - -## 9. Swap-in contract (Core) - -- [x] 9.1 Specify + test that a second video backend joins the SAME `MediaRuntime` seam selected by `videoProvider`, with no change to the sink / canvas / Files output / gating / budget code. — Verify: `swift test` swap a stub for another stub via the selector → feature path unchanged. - -## 10. Spec + validation - -- [x] 10.1 Write the `ai-generative-media` delta (MODIFIED Requirements — video-backend behavior + dangerous gating + disclosure + frontier-local + img2video + swap contract). — Verify: reads as a true delta against the seam owner's ADDED requirements. -- [x] 10.2 `openspec validate ai-video-animation-generation --strict` passes. — Verify: command exits clean. - -## 11. User run-verify (stable-signed build — agent never builds/signs the .app) - -- [ ] 11.1 **User run-verify:** in a stable-signed build, a cloud video actually generates (real upload + `$`-spend within `mediaVideoBudgetPerDay`, audited; over-budget degrades, never silently drops); a local-LTXV selection evicts chat ("busy painting") and produces a clip at the documented latency/thermal cost; the finished clip plays in the canvas player and swipe-DOWN extracts it as a Files-band asset. — Verify: USER stable-signed build (native upload/poll, 35 GB residency, live player cannot be agent-verified; `xcodebuild` compile already GREEN). diff --git a/openspec/changes/canvas-bidi-and-selection-fixes/design.md b/openspec/changes/canvas-bidi-and-selection-fixes/design.md deleted file mode 100644 index e21519c..0000000 --- a/openspec/changes/canvas-bidi-and-selection-fixes/design.md +++ /dev/null @@ -1,37 +0,0 @@ -# Design — canvas bidi + selection fixes - -## Fix 1 — first-strong, per line, stable - -``` -per line: base direction = direction of the FIRST strong char (first word/char), then FIXED - "AICommand שומר" → first strong 'A' → LTR (stays LTR as Hebrew follows) - "שומר AICommand" → first strong 'ש' → RTL (stays RTL as Latin follows) -``` - -### D1 — Why first-strong (not dominant) -The user's explicit rule: the line's side is decided by how it *starts*, and must not change based on characters that arrive later. Dominant/majority (the prior attempt) re-decides from the whole line, so a streaming line flips alignment mid-arrival — jarring, and against the rule. First-strong reads the first strong char and stops; appending content never changes that char, so the side is stable. The one legitimate transition is a line that begins with neutrals (a number, a bullet, whitespace): it stays LTR-default until its first strong char streams in, then locks. That is "deciding," not "re-deciding," and preserves the existing streamed-content requirement. - -### D2 — All RTL scripts -`isStrongRTL` is one contiguous range `U+0590–U+08FF` (Hebrew, Arabic, Syriac, Thaana, N'Ko, Samaritan, Mandaic, Arabic Extended-A/B) plus presentation forms `U+FB1D–U+FB4F`, `U+FB50–U+FDFF`, `U+FE70–U+FEFF`. Strong-LTR is "alphabetic and not in an RTL block." Digits/punctuation/whitespace are neutral (skipped). `dominantDirection` is deleted; `firstStrongDirection` + `naturalTextDirection` are the only base-direction paths. - -## Fix 2 — read the selection before the panel takes key - -### D3 — The bug is an ordering race -``` -handleFire (AI): onFire → executor.fire() // schedules run() as a Task — read happens LATER - setCanvasInteractive(true) // makeKeyAndOrderFront — SYNCHRONOUS, runs FIRST -main actor free: run() → readSelectedText() // panel is already key ⇒ front app not key ⇒ read empty -``` -A `.nonactivatingPanel` becoming key doesn't activate our app, but it *does* take the system key window from the captured front app. With that app no longer key, `AXFocusedUIElement`→`AXSelectedText` is empty and a ⌘C posted to its PID doesn't land — so `readSelectedText()` returns nil and the ambient cascade falls through to the clipboard. This is the "doesn't recognize the selection" report. - -### D4 — Defer key-focus until after the read (a callback, not a delay) -The executor knows exactly when the read is done (it just did it). So it calls `onReadyForInteraction()` right after acquisition; the controller makes the panel key **then**. No fixed delay, no polling. The panel is pass-through in the meantime — fine, because there is nothing to scroll/tap yet (the model hasn't produced output), and the four-finger swipe-to-resolve rides the multitouch device, not window events, so commit/discard keep working. For the `.unavailable` state (no read happens) the hook fires immediately so Enable/Download stay clickable. `makeCanvasInteractive()` is guarded on `model.canvasActive` so a hook that arrives just after a fast discard is a no-op. - -### D5 — Retain the acquired input for a language re-run -Once the user opens the language pill, the panel *is* key (they clicked it) — so a `setLanguage` re-fire must NOT re-read the selection (it would fail exactly as in D3). The executor retains the first fire's `Acquisition` and `setLanguage` passes `reuseInput: true`, so a re-translate reuses the same source. Bonus correctness: it re-translates the *original* text even if the selection changed since. Mirrors how `presuppliedCapture` is already retained for a vision re-translate. A fresh fire (`reuseInput: false`) always clears the retained value and re-reads. - -### D6 — Why not just drop `makeKeyAndOrderFront`? -Relying on click-to-key (`canBecomeKey = true`) alone risks the SwiftUI language `Picker`/unavailable buttons needing two clicks (first focuses the window, second acts) or not opening from a non-key window. Deferring the *proactive* key-grab past the read keeps the exact interaction behavior the canvas had, and only moves *when* it happens — the low-risk change. - -## Verification note -The heuristics and the acquire/retain/hook logic are pure/observable and unit-tested. The actual AX selection read and RTL `NSTextView` rendering ride system APIs that can't be exercised headless — final confirmation is on the user's signed build (fire a command with text selected → it uses the selection and a down-swipe replaces it; fire with nothing selected → clipboard + paste; fire a Hebrew translation → each line anchors to its first word). diff --git a/openspec/changes/canvas-bidi-and-selection-fixes/proposal.md b/openspec/changes/canvas-bidi-and-selection-fixes/proposal.md deleted file mode 100644 index eee2a97..0000000 --- a/openspec/changes/canvas-bidi-and-selection-fixes/proposal.md +++ /dev/null @@ -1,36 +0,0 @@ -## Why - -Two defects surfaced while using the AI preview canvas with real bilingual content (Hebrew/English screenshot): - -1. **Line direction was unstable.** An earlier attempt keyed each paragraph's base direction off its *dominant* script, so a line could flip alignment (LTR↔RTL) as more characters streamed in. The user's rule is the opposite and simpler: **a line's direction is decided by its first word/char and then held fixed** — it must not re-decide from characters that arrive later in the sentence. (First-strong, per line, stable.) Separately, RTL detection only covered Hebrew/Arabic, not "all RTL languages." - -2. **The app doesn't recognize selected text — it falls back to the clipboard.** When an AI command fires, the canvas panel calls `makeKeyAndOrderFront` to become key-interactive (for the language pill / Enable+Download controls). That runs *synchronously* at fire, while the executor's selection read runs *asynchronously* a moment later — so the non-activating panel grabs key focus **before** the read. With the captured front app no longer key, its AX focused element is empty and the ⌘C fallback doesn't land, so `readSelectedText()` returns nothing and the input silently cascades to the clipboard. The context-resolution work fixed the *logic*; this fixes the *timing* that was starving it. - -## What Changes - -**Fix 1 — per-line first-strong base direction, stable, all RTL scripts** (`BidiText.swift`): -- The per-paragraph base direction + alignment is chosen by the paragraph's **first strong directional character** (the first word/char decides the side). Leading neutrals (whitespace, a number, a bullet) are skipped to that first strong char; once it is present the side is **stable** — later characters never re-decide it. (The dominant/majority heuristic is removed.) -- RTL detection is broadened to **all RTL scripts** — the contiguous `U+0590–U+08FF` block (Hebrew, Arabic, Syriac, Thaana, N'Ko, Samaritan, Mandaic, Arabic Extended) plus the Hebrew/Arabic presentation-form blocks — so any RTL/LTR language is honored, not just Hebrew/Arabic. `naturalTextDirection` (short review-field values) uses the same rule. Mixed runs within a paragraph still resolve via the Unicode Bidi algorithm. - -**Fix 2 — read the selection before the canvas panel takes key focus** (`AICommandExecutor`, `LauncherOverlayController`, `AppCoordinator`): -- The canvas panel stays **pass-through (non-key)** at fire; it becomes key-interactive only **after** the executor has read the fire's input, via a new `onReadyForInteraction` callback the executor invokes once acquisition completes (and immediately for the `.unavailable` state, whose Enable/Download controls need the mouse). So the selection is read while the captured front app still holds key focus. -- The executor **retains the acquired input**; a language re-run (`setLanguage`) **reuses** it (`reuseInput`) instead of re-reading the selection — which would now fail, since by then the user has interacted with the language pill and the panel holds key. This also makes a re-translate use the exact same source, not a since-changed selection. - -**Fix 2b — the ⌘C fallback actually lands** (`SelectionService`; found via on-device `[Selection]` logging): -- With the ordering fixed, on-device logs showed the remaining failure: for an AX-opaque app (Terminal), the ⌘C fallback was posted to the app's PID **without activating it**, so it never processed the copy (`changeCount advanced=false`) and the input fell through to the clipboard. The copy path now **re-asserts the captured app as frontmost** (`activate` + settle) before synthesizing ⌘C — exactly what the working paste path already does for ⌘V. -- **Secure Keyboard Entry** (Terminal's, or any app's) blocks synthesized keystrokes system-wide; the fallback now **detects and logs** this (`IsSecureEventInputEnabled()`) so it is diagnosable rather than a silent clipboard fall-through. -- A `Selection`-category `os.Logger` across the read path (kept for now) makes the whole chain observable in Console: which app, whether AX exposed a selection, whether the ⌘C landed, and which channel won. - -## Capabilities - -### Modified Capabilities - -- `launcher-overlay`: the "Bidirectional (RTL/LTR) text rendering" requirement is restated as **first-strong per line, stable, covering all RTL scripts** (replacing the short-lived dominant-direction wording); a new requirement states the **canvas panel takes key focus only after the fired command's input is read**, and a language re-run reuses the acquired input. -- `selection-io`: the "Clipboard fallback with restore" requirement gains **activate-the-captured-app-before-⌘C** (a ⌘C posted to a non-active app doesn't land) and **Secure-Keyboard-Entry detection** (a system-wide synthetic-key block is surfaced/logged, not a silent success). - -## Impact - -- **Code (MLX-free Core):** `Overlay/BidiText.swift` (first-strong base direction, broadened `isStrongRTL`; `dominantDirection` removed); `AI/AICommandExecutor.swift` (`onReadyForInteraction` hook, `Acquisition` refactor of the input read, `retainedAcquisition` + `reuseInput`); `Overlay/LauncherOverlayController.swift` (defer `setCanvasInteractive` → new guarded `makeCanvasInteractive()`); `App/AppCoordinator.swift` (wire the hook). -- **Tests:** `BidiTextDirectionTests` (first-strong decides + is stable as content appends, leading-neutral skip, all-RTL-scripts, neutral→LTR, class guards); `AICommandExecutorTests` (the interaction hook fires once after acquisition; a language re-run reuses the retained input rather than re-reading a changed selection). -- **No gesture/permission/signing/model impact.** The two-finger resolve grammar is unchanged; the four-finger swipe-to-resolve already rides the multitouch device, so it keeps working while the panel is pass-through. Verified under `swift build`/`swift test`; the live selection read + RTL rendering ride AX/`NSTextView` and are confirmed on the user's signed build. -- **Out of scope:** markdown rendering of the model's output (`**`, `$\rightarrow$` are a model artifact); the notch conversation surfaces (same `BidiText` benefits from Fix 1 automatically; its own key-focus timing is not in scope here); any change to how `SelectionService` reads (AX-then-⌘C) — only *when* it runs relative to the panel taking key. diff --git a/openspec/changes/canvas-bidi-and-selection-fixes/specs/launcher-overlay/spec.md b/openspec/changes/canvas-bidi-and-selection-fixes/specs/launcher-overlay/spec.md deleted file mode 100644 index 7a5ea96..0000000 --- a/openspec/changes/canvas-bidi-and-selection-fixes/specs/launcher-overlay/spec.md +++ /dev/null @@ -1,45 +0,0 @@ -## MODIFIED Requirements - -### Requirement: Bidirectional (RTL/LTR) text rendering in the preview canvas -The preview canvas SHALL render text **bidirectionally**: each paragraph's **base direction is decided by its FIRST strong directional character** (the first word/char of the line) — leading neutrals (whitespace, digits, punctuation, a bullet, a URL) are skipped to that first strong character. Once the first strong character is present the paragraph's side SHALL be **stable**: characters that arrive later in the line SHALL NOT re-decide its direction (a line that starts in a left-to-right script stays left-aligned even as right-to-left content follows, and vice versa). A paragraph that has streamed only neutrals so far SHALL default to left-to-right until its first strong character arrives, then adopt that character's side. RTL detection SHALL cover **all RTL scripts** (the `U+0590–U+08FF` block — Hebrew, Arabic, Syriac, Thaana, N'Ko, Samaritan, Mandaic, Arabic Extended — plus the Hebrew/Arabic presentation-form blocks), not only Hebrew and Arabic. **Mixed** left-to-right and right-to-left runs within a paragraph SHALL resolve via the Unicode Bidi algorithm so combined text reads cleanly (e.g. a Latin word or URL inside a Hebrew sentence). This SHALL apply to the **streamed output**, the **input echo**, and the **task-review fields**. - -#### Scenario: A right-to-left result starts from the correct side -- **WHEN** the streamed result is Hebrew text -- **THEN** it renders right-aligned with a right-to-left base direction and correct punctuation placement - -#### Scenario: The first word decides the line and the side is stable -- **WHEN** a line begins with a strong character of one direction and later contains runs of the other direction -- **THEN** the line's base direction follows the first strong character and does NOT change as the later runs stream in (the embedded runs are still placed correctly by the Bidi algorithm) - -#### Scenario: Leading neutrals are skipped to the first strong character -- **WHEN** a paragraph begins with whitespace, a number, or a bullet marker before its first letter -- **THEN** the base direction is decided by the first strong letter, not the leading neutrals - -#### Scenario: A neutral-only line adopts its side once the first strong char arrives -- **WHEN** a paragraph has streamed only neutral characters so far -- **THEN** it renders left-to-right until its first strong directional character streams in, then locks to that character's side - -#### Scenario: Non-Hebrew/Arabic RTL scripts resolve right-to-left -- **WHEN** a line's first strong character is in another RTL script (e.g. Syriac, Thaana, or N'Ko) -- **THEN** it renders right-aligned with a right-to-left base direction - -#### Scenario: Left-to-right text is unaffected -- **WHEN** the streamed result is English text -- **THEN** it renders left-aligned with a left-to-right base direction - -## ADDED Requirements - -### Requirement: The canvas reads the fired command's input before taking key focus -When an AI command is fired, the preview canvas panel SHALL remain **pass-through (non-key)** until the executor has **read the command's input**, and SHALL become key-interactive only afterward. This ordering exists because the panel is a non-activating panel: taking key focus removes the system key window from the app that was frontmost when the launcher opened, which empties that app's Accessibility focused element and prevents the clipboard-copy fallback from landing — so a selection read taken while the panel is key returns nothing and the input wrongly falls through to the clipboard. The executor SHALL signal readiness after acquisition (and immediately for the availability/`unavailable` state, whose enable/download controls need the mouse); the controller SHALL make the canvas interactive on that signal, guarded on the canvas still being open. While the panel is pass-through, the two-finger resolve gestures (which ride the multitouch device, not window events) SHALL keep working. - -#### Scenario: A selection is read before the panel takes key focus -- **WHEN** an AI command is fired with text selected in the front app -- **THEN** the selection is read while the front app still holds key focus (the canvas panel has not yet taken key), so the selected text — not the clipboard — is used as the input - -#### Scenario: The canvas becomes interactive after the input is read -- **WHEN** the executor finishes acquiring the fired command's input -- **THEN** the canvas panel becomes key-interactive so its controls (language pill, scroll, and the unavailable enable/download controls) respond - -#### Scenario: A language re-run reuses the acquired input -- **WHEN** the user changes the runtime language of an in-flight command after the canvas has become interactive -- **THEN** the re-run reuses the input acquired by the first fire rather than re-reading the selection (which would fail now that the panel holds key focus), and re-runs against the same source diff --git a/openspec/changes/canvas-bidi-and-selection-fixes/specs/selection-io/spec.md b/openspec/changes/canvas-bidi-and-selection-fixes/specs/selection-io/spec.md deleted file mode 100644 index 1f4dbb0..0000000 --- a/openspec/changes/canvas-bidi-and-selection-fixes/specs/selection-io/spec.md +++ /dev/null @@ -1,16 +0,0 @@ -## MODIFIED Requirements - -### Requirement: Clipboard fallback with restore -When Accessibility does not expose the selection, the system SHALL fall back to synthesizing ⌘C against the captured front app, reading the resulting pasteboard text, and then **restoring the previous pasteboard contents** so the user's clipboard is left as it was. Before synthesizing the ⌘C, the system SHALL **re-assert the captured app as frontmost** (activate it and let activation settle), because a ⌘C posted to an app that is not the active app is not processed (e.g. Terminal) — mirroring the write-back paste path, which already activates before synthesizing ⌘V. When **Secure Keyboard Entry** is enabled (Terminal's Secure Keyboard Entry, or any app that enabled it), synthesized keystrokes are blocked system-wide; the fallback SHALL then yield no text (the caller surfaces "no input" / the existing clipboard fall-through) rather than appearing to succeed, and this condition SHALL be recorded in the diagnostic log so it is distinguishable from a genuinely empty selection. - -#### Scenario: Fallback reads via copy and restores the clipboard -- **WHEN** the AX selection read fails but text is selected, and the fallback is used -- **THEN** the captured app is re-asserted as frontmost, a synthesized ⌘C captures the selection, and the prior clipboard contents are restored afterward - -#### Scenario: Fallback does not clobber a password on the clipboard -- **WHEN** the clipboard held sensitive content before the fallback ran -- **THEN** that content is restored unchanged after the fallback read - -#### Scenario: Secure Keyboard Entry blocks the synthesized copy -- **WHEN** Secure Keyboard Entry is enabled and an AX-opaque app's selection is read via the ⌘C fallback -- **THEN** the copy does not land, the system yields no text (surfaced as no input rather than a false success), and the blocked-by-secure-input condition is logged diff --git a/openspec/changes/canvas-bidi-and-selection-fixes/tasks.md b/openspec/changes/canvas-bidi-and-selection-fixes/tasks.md deleted file mode 100644 index 7df4cf5..0000000 --- a/openspec/changes/canvas-bidi-and-selection-fixes/tasks.md +++ /dev/null @@ -1,31 +0,0 @@ -# Tasks — canvas bidi + selection fixes - -## 1. Fix 1 — first-strong, stable, all RTL scripts (`Overlay/BidiText.swift`) -- [x] 1.1 `isStrongRTL` covers the full `U+0590–U+08FF` RTL block + presentation forms (all RTL scripts) -- [x] 1.2 `BidiText.apply` picks each paragraph's base from `firstStrongDirection` (first word/char decides, stable) -- [x] 1.3 `naturalTextDirection(for:)` uses `firstStrongDirection` -- [x] 1.4 Remove `dominantDirection`; update doc comments (first-strong, stable) - -## 2. Fix 2 — read selection before the panel takes key -- [x] 2.1 `AICommandExecutor`: `onReadyForInteraction` hook, called after acquisition + in the `.unavailable` branch -- [x] 2.2 `AICommandExecutor`: refactor the input read into `acquire(_:) -> Acquisition`; retain it; `fire(reuseInput:)` + `run(reuseInput:)` -- [x] 2.3 `AICommandExecutor`: `setLanguage` re-fires with `reuseInput: true` (reuse the acquired source, don't re-read) -- [x] 2.4 `LauncherOverlayController`: drop `setCanvasInteractive(true)` from the fire / `showCanvas` paths; add guarded `makeCanvasInteractive()` -- [x] 2.5 `AppCoordinator`: wire `aiCommandExecutor.onReadyForInteraction = { launcherOverlay.makeCanvasInteractive() }` - -## 2b. Fix 2b — the ⌘C fallback lands (on-device diagnosis via `[Selection]` logging) -- [x] 2b.1 `SelectionService`: diagnostic `os.Logger` (category `Selection`) across `readSelectedText` / `copyWithRestore` + the executor's channel resolution — surfaced the real failure (`⌘C changeCount advanced=false` for Terminal) -- [x] 2b.2 `copyWithRestore` re-asserts the captured app as frontmost (`activate` + 40ms settle) before synthesizing ⌘C — mirrors the working paste path; a ⌘C posted to a non-active app didn't land -- [x] 2b.3 Detect + log **Secure Keyboard Entry** (`IsSecureEventInputEnabled()`) so a system-wide synthetic-key block is diagnosable, not a silent clipboard fall-through -- [x] 2b.4 `selection-io` spec delta: activate-before-copy + Secure-Keyboard-Entry behavior - -## 3. Tests -- [x] 3.1 `BidiTextDirectionTests`: first-strong decides + is stable as content appends; leading-neutral skip; all RTL scripts; neutral→LTR; strong-class guards -- [x] 3.2 `AICommandExecutorTests`: interaction hook fires once after acquisition; a language re-run reuses the retained input rather than re-reading a changed selection -- [x] 3.3 `SelectionServiceTests` still green after the `copyWithRestore(pid:)` → `copyWithRestore(app:)` signature change - -## 4. Verify -- [x] 4.1 `swift build` — clean -- [x] 4.2 `swift test` — 1491 tests, 0 failures -- [ ] 4.3 User confirms on a signed build: (a) a command with text selected uses the selection (down-swipe replaces); with nothing selected uses the clipboard (pastes); (b) a Hebrew translation anchors each line by its first word and stays stable while streaming -- [ ] 4.4 Apply the `launcher-overlay` delta to the main spec at archive (`/opsx:archive` / `/opsx:sync`) diff --git a/openspec/changes/notch-timeline-and-tuning/.openspec.yaml b/openspec/changes/notch-timeline-and-tuning/.openspec.yaml deleted file mode 100644 index eb139cc..0000000 --- a/openspec/changes/notch-timeline-and-tuning/.openspec.yaml +++ /dev/null @@ -1,2 +0,0 @@ -schema: spec-driven -created: 2026-07-19 diff --git a/openspec/changes/notch-timeline-and-tuning/design.md b/openspec/changes/notch-timeline-and-tuning/design.md deleted file mode 100644 index 61922b3..0000000 --- a/openspec/changes/notch-timeline-and-tuning/design.md +++ /dev/null @@ -1,55 +0,0 @@ -# Design — notch-timeline-and-tuning - -## Context - -The runtime seam already streams two tagged channels (`Token.channel = .thinking | .response` in `AI/LLMRuntime.swift`), and `NotchSessionEngine` already consumes them — but flattens them into two disconnected strings: `@Published thinking` (one flat accumulator, rendered as a single collapsible pinned *after* the whole message list) and `state = .conversing(partial:)` (the answer bubble). Per-message `AgentMessage.thinking` is persisted but never rendered, so history loses its reasoning. In the **routed** path (`runRoutedTurn`, the production path once tools are wired), `AgentLoop` exposes both `onThinking` and `onResponseToken` sinks but the engine wires only `onThinking` — the answer never streams; it lands whole at settle. - -For tuning: `aiReasoningEnabled` is snapshot per session at `startNew()`/`bind()` (the exact "applies to new conversations" semantic), but it is a **global** Hub toggle shared with the launcher band; the context budget is **not wired at all** for the notch engine (`makeNotchSessionEngine` passes no `budgetProvider` → hardcoded `DefaultContextBudget()` = 8192), even though `AgentContextPreset`/`AgentContextBudgetProvider` exist and the Hub surfaces them. - -Constraints inherited from the codebase: the panel is non-activating (key only while the composer is focused); teardown stays synchronous on the ghost-on-Space-switch paths; no repeating animation that isn't visibility-gated (idle-CPU-spin postmortem); persistence growth must be decode-safe optionals (no schema bump); **`text` is the only re-fed content — thinking is display-only and structurally excluded from assembly** (`AgentConversation.swift` invariant, `ChatTemplate` echo). - -## Goals / Non-Goals - -**Goals:** -- The expanded transcript is a **timeline**: every assistant turn (live and historical) renders its thinking and answer as interleaved segments in token-arrival order; the live turn streams both channels token-by-token — including the routed path's answer. -- A notch-native **settings zone** (gear above "+ New chat" → the same panel morphs into settings mode) with **one slider** over ordered thinking+context stops, persisted, applied to each **new** conversation until changed. -- Tuning is **born-with**: a conversation carries the (reasoning, context-tokens) it was created under for its whole life. - -**Non-Goals:** -- Interleaving **tool steps** into the persisted timeline (the current-turn tool-step list stays as-is; a follow-up can fold them in). -- Touching the runtime seam, the channel classifier, or the launcher AI band / voice surfaces. -- Any Hub redesign — the Hub's global reasoning/context controls remain for the other surfaces; the notch slider is the notch's own dial. -- A per-conversation (retroactive) tuning editor — the slider shapes *future* conversations only. - -## Decisions - -**D1 — Segments are a display-only, decode-safe extension of `AgentMessage`.** `TurnSegment { kind: .thinking | .answer; text }` (Codable/Equatable/Sendable, Core); `AgentMessage.segments: [TurnSegment]?` optional. Assembly (`assembleRequest`/`ChatTemplate`) continues to read `text` only — segments join `thinking` in the display-only tier. `text` and flat `thinking` are still written on settle, so older builds and existing readers see exactly what they saw before. A message with `segments == nil` (pre-change history) renders the legacy fallback: flat `thinking` as one leading thinking segment, `text` as one answer segment. *Alternative — a parallel per-turn sidecar store: rejected; the message already persists `thinking`, and one optional field keeps one owner (the durable store) and zero migration.* - -**D2 — One ordered accumulator in the engine, fed by both turn paths.** `NotchSessionEngine` grows `@Published liveSegments: [TurnSegment]` with a single append-or-coalesce rule: a token whose channel matches the last segment's kind appends to it; otherwise a new segment starts. The plain path feeds it directly from the `runtime.chat` token loop. The routed path generalizes its existing ordered `AsyncStream` thinking consumer to `AsyncStream<(TokenChannel, String)>` and feeds **both** `AgentLoop.onThinking` and the newly wired `onResponseToken` through it, so cross-channel order is serialized exactly as emitted. The flat `@Published thinking` and `state = .conversing(partial:)` keep accumulating in parallel (existing seams/tests; the partial still drives pinned-scroll and the collapsed-while-streaming badge classification). On settle, `appendAssistantTurn` persists `segments = liveSegments` (coalesced) alongside `text`/`thinking` and clears the accumulator. *Alternative — deriving order in the view from two flat strings: impossible; interleaving information is lost at accumulation time.* - -**D3 — The routed answer streams by wiring the sink that already exists.** `runRoutedTurn` passes `onResponseToken` into `AgentLoop` (today omitted → answer invisible until settle). No `AgentLoop` change: `answer(...)` already emits per-token callbacks for both channels. - -**D4 — Transcript renders per-message segments; the live turn is a synthetic timeline entry.** `NotchConversationView` drops the bottom `thinkingSection` and the `.conversing(partial)` state bubble; instead the thread renders each assistant message's segments in order (answer segments as today's `BidiText` bubbles; thinking segments as muted, per-block collapsibles), and while a turn is in flight it renders the same segment view over `engine.liveSegments`. A **live** thinking segment streams expanded (you watch it think); a **settled** thinking segment collapses to a compact "Thinking" row, expandable per block. Auto-scroll pins on `liveSegments` mutations (superset of the old partial/thinking triggers). No new repeating animation (idle-CPU-spin rule) — streaming text itself is the motion. - -**D5 — Settings zone is a third panel mode, not a second panel.** `NotchHomeZoneViewModel.Mode` grows `.settings`; the gear is a small icon button stacked **above the "+ New chat" card** in the rail's leading column. Clicking it mode-switches the same panel with the same border-stretch resize used by `expandSession` (a `NotchHomeZoneLayout` settings solve, compact fixed size); a back affordance (and clicking the gear again) returns to rail mode. The panel **never becomes key in settings mode** (slider + buttons are mouse-only, honoring the non-activating contract; only the composer ever takes key). **Grace-dismiss applies in settings mode as in rail mode** — it is a transient tweak surface, and only an expanded *conversation* pins the panel open; a grace-dismissed settings zone reopens as the rail. Feature-off/Space-switch teardown stays synchronous. *Alternative — hosting settings inside the expanded conversation chrome: rejected; the ask is a rail-level surface shaping future conversations, not a per-session control.* - -**D6 — One slider, four ordered stops, one persisted level.** A Core enum `NotchTuning: String, CaseIterable — quick, balanced, deep, max` with derived semantics: `reasoning` (false for `quick`, true otherwise) and `contextTokens(modelMax:)` (quick/balanced → 8_192, deep → 32_768, max → modelMax — reusing `AgentContextPreset.tokens(modelMax:custom:)` resolution values). Persisted as one `AppSettings` key (`notchTuning`, default `.balanced`, matching today's effective defaults: reasoning on, 8192). The slider is a 0…3 discrete `Slider` mapped over `CaseIterable`, captioned with the stop's title plus "thinking on/off · N tokens". *Alternative — writing the global `aiReasoningEnabled`/`agentContextPreset`: rejected; the user asked for tuning “specifically this AI interface,” and the globals also steer the launcher band and every other agent surface.* - -**D7 — Born-with tuning rides the conversation.** `AgentConversation` grows decode-safe optionals `reasoningOverride: Bool?` and `contextTokens: Int?`. `startNew()` snapshots the injected `tuningDefault: () -> (reasoning: Bool, contextTokens: Int)` (a closure reading `settings.notchTuning` + the model max at snapshot time, mirroring the existing `reasoningDefault` idiom) and stamps both onto the newborn conversation; `bind()` prefers the stored values and falls back to the legacy behavior (`reasoningDefault()`, injected budget) when absent — so pre-change sessions behave exactly as before. The compactor's budget becomes per-session: stored `contextTokens` → `DefaultContextBudget(maxContextTokens:)`, else the injected provider. `makeNotchSessionEngine` finally wires real numbers (clamped to the model max via the `AgentContextBudgetProvider` idiom). *Alternative — snapshot-only in the engine (the current reasoning behavior, re-read on every re-bind): rejected; “applied to following new conversations” means an existing session must keep the tuning it was born under, which requires persisting it.* - -## Risks / Trade-offs - -- **[Store growth: segments duplicate `text`/`thinking` per assistant message]** → Bounded by the conversation itself (already compacted against the budget); segments are plain text, display-only, and per-message — no unbounded stream retained. -- **[Per-token `@Published` churn on a SwiftUI list]** → The coalesce rule mutates only the **last** segment's string per token (the array's identity list changes only at channel flips), the same publish cadence the flat `thinking`/`partial` already sustain today. -- **[A live thinking block could tempt a shimmer/pulse]** → None added; the idle-CPU-spin rule stands (any future liveness cue must be occlusion-gated). -- **[Settings mode on a non-activating panel]** → Slider/buttons are mouse-driven (no key status needed); the key-flip stays composer-only, so focus behavior is unchanged. -- **[Legacy sessions with no stored tuning]** → Explicit fallback tier in `bind()` keeps their exact pre-change behavior; only newborn conversations carry tuning. -- **[Old builds reading new rows]** → Synthesized Codable decoding ignores unknown keys; `text`/`thinking` are still written, so a rollback renders what it always rendered. - -## Migration Plan - -No migration. All persisted growth is optional fields on already-stored types (the `dimPercent` precedent); existing rows decode unchanged, and the new fields decode as `nil` → legacy rendering/behavior. Rollback is safe (new keys ignored, legacy fields still authoritative for old code). - -## Open Questions - -None blocking. Follow-up candidate: interleave tool steps as a third segment kind so the timeline shows think → tool → answer end-to-end. diff --git a/openspec/changes/notch-timeline-and-tuning/proposal.md b/openspec/changes/notch-timeline-and-tuning/proposal.md deleted file mode 100644 index bbffa75..0000000 --- a/openspec/changes/notch-timeline-and-tuning/proposal.md +++ /dev/null @@ -1,33 +0,0 @@ -# Proposal — Notch chat: timeline-ordered thinking/answer streaming + in-notch tuning zone - -## Why - -The notch conversation renders its two token channels as two disconnected flat strings: one "Thinking" collapsible pinned after the whole message list, the streaming answer in a separate state bubble — and in the **routed** turn path (the production path with tools) the answer does not stream at all; it lands whole at settle. Past turns lose their thinking entirely (per-message `thinking` is never rendered). The result reads nothing like the model actually worked: you can't see thinking and answer unfold **in the order they happened**. Separately, the two knobs that shape a conversation — reasoning on/off and context size — are buried in the Hub, are global, and the notch engine ignores the context setting entirely (hardcoded 8192-token default), so there is no way to tune the notch chat from the notch. - -## What Changes - -- **Timeline transcript (Claude-Code-style).** An assistant turn becomes an ordered sequence of display-only **segments** (thinking / answer), appended live in token-arrival order and persisted with the message, so both the live turn and every historical turn render as an interleaved timeline: thinking block(s) where thinking happened, answer text where the answer happened. Thinking segments stay visually distinct (muted, collapsible per block) and are still **never re-fed** to the model — assembly continues to read `text` only. -- **The routed turn streams its answer live.** `NotchSessionEngine.runRoutedTurn` wires the already-existing `AgentLoop.onResponseToken` sink (today omitted) so the final answer streams token-by-token exactly like the plain-chat path, instead of appearing whole at settle. -- **In-notch settings zone.** The rail grows a small **settings (gear) affordance above the "+ New chat" card**. Clicking it morphs the same notch panel (border-stretch, no second panel) into a **settings mode** hosting one slider — **Thinking + context** — over ordered stops (Quick: thinking off · 8k → Balanced: thinking on · 8k → Deep: thinking on · 32k → Max: thinking on · model max). The value persists in settings and applies to **each following new conversation** until changed again. -- **Tuning is born-with, per conversation.** A new conversation snapshots the slider's (reasoning, context-tokens) at birth and carries them for life (decode-safe optional fields on the stored conversation); changing the slider later never retunes an existing session. The notch engine finally honors a real context budget (per-conversation tokens clamped to the model max) instead of the hardcoded default. - -## Capabilities - -### New Capabilities - -(none) - -### Modified Capabilities - -- `ai-parked-sessions`: the expanded-conversation requirement changes from "reasoning behind a collapsible section" to a **timeline of interleaved thinking/answer segments, streamed live in arrival order (including the routed path's answer) and persisted per turn**; a new requirement adds the **in-notch settings zone** (gear above "+ New chat", panel morphs to settings mode, one thinking+context slider, persisted, applied to new conversations only, born-with per conversation). - -## Impact - -- **Code (all MLX-free Core, `swift build`/`swift test`):** - - `AI/Agent/AgentConversation.swift` — display-only ordered segments on `AgentMessage` (decode-safe optional; `text`-only assembly invariant preserved); optional born-with tuning fields on `AgentConversation`. - - `AI/Parked/NotchSessionEngine.swift` — live segment accumulation for both turn paths; wire `onResponseToken`; snapshot tuning at `startNew()`, restore it at `bind()`; per-conversation context budget feeding the compactor. - - `App/AppCoordinator.swift` (`makeNotchSessionEngine`) — inject settings-backed tuning defaults (today the context budget is not wired at all). - - `Overlay/NotchHomeZoneOverlay.swift` / `Overlay/NotchHomeZoneController.swift` — transcript rebuilt around segments; `Mode.settings` + gear affordance + settings-zone view + layout solve. - - `Settings/AppSettings.swift` — one persisted notch tuning level (default Balanced). -- **Specs:** `ai-parked-sessions` delta (one MODIFIED, one ADDED requirement). -- **No new permission, no gesture relocation, no schema bump** (all persisted growth is decode-safe optionals). Existing stored conversations/messages decode unchanged (old messages render their legacy flat `thinking` as a single leading thinking segment). Runtime seam (`LLMRuntime`, channels) is untouched — this consumes the existing `.thinking`/`.response` split. diff --git a/openspec/changes/notch-timeline-and-tuning/specs/ai-parked-sessions/spec.md b/openspec/changes/notch-timeline-and-tuning/specs/ai-parked-sessions/spec.md deleted file mode 100644 index 1940d4c..0000000 --- a/openspec/changes/notch-timeline-and-tuning/specs/ai-parked-sessions/spec.md +++ /dev/null @@ -1,118 +0,0 @@ -# ai-parked-sessions — delta for notch-timeline-and-tuning - -## ADDED Requirements - -### Requirement: A settings affordance morphs the panel into an in-notch tuning zone for new conversations -The rail SHALL host a **settings (gear) affordance stacked above the persistent "+ New chat" card**. Activating it SHALL morph the **same** notch panel in place into a **settings zone** — mode-switched like the expanded conversation, never a second panel, the rail↔settings size change stretching the border. The zone SHALL host a single **thinking + context slider** over **ordered discrete stops** — from a no-thinking/base-context stop, through balanced and deep stops, to a maximum stop (thinking on, model-maximum context) — each stop labeled with its thinking state and effective context-token count. The selected stop SHALL persist across relaunches and SHALL apply to **each conversation created after the change, until changed again**: a newborn conversation SHALL snapshot the stop's (reasoning, context-token) values **at birth** and carry them for its whole life (persisted with the conversation, decode-safe), its turns running with that reasoning flag and its compaction running against that context budget clamped to the model maximum. Moving the slider SHALL NOT retune any existing conversation (including a parked one re-expanded later). A conversation stored **before** this capability (no born-with tuning) SHALL keep the pre-change behavior (the global reasoning default and the injected budget). The settings zone SHALL remain **mouse-only** on the non-activating panel (it SHALL never take key status), a back affordance SHALL return the panel to rail mode, the grace-period dismiss SHALL apply in settings mode as in rail mode (a later reveal reopens the rail), and feature-off/Space-switch teardown SHALL remain synchronous. - -#### Scenario: The gear morphs the panel into the settings zone and back -- **WHEN** the user clicks the gear above the "+ New chat" card and later clicks the back affordance -- **THEN** the same panel stretches in place into the settings zone and back to the rail — no second panel, no key status taken at any point - -#### Scenario: The slider applies to following new conversations only -- **WHEN** an existing session is parked, the user moves the slider to a different stop, and then creates a new chat -- **THEN** the new conversation is born with the new stop's reasoning and context-token values (persisted with it), and the existing session keeps the tuning it was born under - -#### Scenario: The selection persists across relaunch -- **WHEN** the user selects a stop and the app relaunches -- **THEN** the settings zone reopens showing that stop, and a conversation created after the relaunch is born with its values - -#### Scenario: A no-thinking stop suppresses reasoning for conversations born under it -- **WHEN** the slider sits on the no-thinking stop and a new chat runs a turn -- **THEN** the turn requests no reasoning and no thinking segments appear, while a conversation born under a thinking stop still streams thinking - -#### Scenario: A born-with context budget drives compaction -- **WHEN** a conversation born at a deep or maximum stop grows past its born-with token budget -- **THEN** compaction runs against that conversation's own budget (clamped to the model maximum), not the default or another conversation's budget - -#### Scenario: Settings mode grace-dismisses like the rail -- **WHEN** the settings zone is open and the cursor leaves the live area for longer than the grace period -- **THEN** the panel dismisses (ordered out synchronously at the end of the shrink, as in rail mode) and the next reveal opens the rail - -## MODIFIED Requirements - -### Requirement: A card expands in place into a notch-anchored conversation panel -Clicking a session card (or creating a new chat) SHALL expand the notch panel **in place** into a conversation view for that session — the same merged-notch panel and chrome, mode-switched from the rail, never a second panel. The thread SHALL open scrolled to its **latest turn** — an existing session with history lands at the **end** of its conversation, not at the top — and SHALL stay pinned to the newest turn as it streams. The expanded view SHALL render the session's thread as a **timeline**: each assistant turn SHALL be an ordered sequence of **thinking and answer segments in token-arrival order** (a segment boundary wherever the channel flips), with answer segments rendered as normal turn text and thinking segments rendered visually distinct (muted) and collapsible **per block**. A live turn SHALL stream **both channels token-by-token into the timeline in arrival order** — including a **routed (tool-running) turn's final answer, which SHALL stream live rather than appearing whole at settle** — the in-flight thinking block streaming expanded and collapsing to a compact expandable row once the turn settles. The segment timeline SHALL be **persisted display-only per assistant turn**, so a historical turn re-renders its interleaved timeline after collapse/expand or relaunch; a turn stored **before** this capability (no segment timeline) SHALL render its flat stored reasoning as a single leading collapsible thinking block followed by its answer text. Thinking SHALL remain **display-only and never re-fed to the model** (assembly reads answer text only). The expanded view SHALL also render the ordered tool steps the routing loop has run, and a typed **composer**: Enter sends the turn; the panel SHALL become the **key window only while the composer field is focused** so keystrokes reach it, SHALL never become the main window, SHALL never activate the app, and SHALL drop key status when the conversation collapses so the previously frontmost app keeps focus. When the session's routing loop pauses awaiting a decision, the expanded view SHALL present the review as a card with explicit **Approve** and **Skip** buttons (the notch is a cursor-and-keyboard surface; the launcher's two-finger compass grammar is not imported). Each assistant answer SHALL offer a **Copy** affordance (the notch surface writes nothing into other apps). Exactly **one** session SHALL be expanded (foreground) at a time; expanding another card SHALL first collapse the current one. **Collapse** — via an explicit collapse affordance, Escape from the composer, or clicking the notch resting zone — SHALL persist the conversation back to the store, return the panel to rail mode, and hand the session back to background scheduling **without cancelling an in-flight turn** (the turn completes in the background and updates the card's badge). **An in-flight turn INCLUDES one paused at an approval:** collapsing while the routing loop awaits an Approve/Skip decision SHALL keep the suspended step alive (the session surfaces as needs-you), and re-expanding SHALL re-present the same approval card whose Approve/Skip resumes the original paused step — the turn is never restarted or silently dropped by a dock/expand round-trip. While a conversation is expanded, cursor departure SHALL NOT dismiss the panel (the grace-dismiss applies to rail and settings modes only); feature-off and Space-switch teardown SHALL remain synchronous. - -#### Scenario: Expanding a card shows its thread and composer in place -- **WHEN** the user clicks a parked session's card on the rail -- **THEN** the same notch panel expands in place into that session's conversation view — timeline thread, tool steps, and composer — with no second panel and no surface change - -#### Scenario: The panel is key only while the composer is focused -- **WHEN** the user focuses the composer, types a turn, and later collapses the conversation -- **THEN** keystrokes reach the composer while it is focused (the panel is key), the panel never becomes main and never activates the app, and on collapse key status is dropped so the previously frontmost app keeps focus - -#### Scenario: Enter sends and thinking and answer stream interleaved in arrival order -- **WHEN** the user types a message in the expanded composer and presses Enter on a conversation born with reasoning on -- **THEN** the turn is appended to the session's conversation and the assistant's thinking and answer stream live into the thread as interleaved timeline segments in exactly the order the channels produced them — the live thinking block expanded while it streams, collapsing to a compact row at settle - -#### Scenario: A routed turn's answer streams live -- **WHEN** a turn runs through the routing loop (tool steps) and reaches its final answer -- **THEN** the answer streams token-by-token into the timeline after the thinking that preceded it — it does not appear whole at settle - -#### Scenario: A historical turn re-renders its interleaved timeline -- **WHEN** a session with settled turns is re-expanded, or rebuilt from the durable store after a relaunch -- **THEN** each assistant turn renders its persisted thinking/answer segments in original arrival order, thinking blocks collapsed to compact expandable rows - -#### Scenario: A pre-change turn renders its flat reasoning as one leading thinking block -- **WHEN** a stored assistant message has no segment timeline (persisted before this capability) -- **THEN** it renders its flat stored reasoning as a single leading collapsible thinking block followed by its answer text, and nothing is dropped - -#### Scenario: A paused tool step is resolved with buttons -- **WHEN** the expanded session's routing loop pauses awaiting a confirm/dangerous decision -- **THEN** the review is presented as a card with Approve and Skip buttons, and choosing one resumes the loop accordingly - -#### Scenario: Collapse returns the session to background scheduling mid-turn -- **WHEN** the user collapses the conversation while an assistant turn is still streaming -- **THEN** the panel returns to rail mode, the in-flight turn is not cancelled, and its completion updates the session's card badge through the scheduler — the session is NOT dismissed when the turn completes - -#### Scenario: Docking a paused approval survives the round-trip -- **WHEN** the user collapses the conversation while the routing loop is paused awaiting an approval, then later re-expands the session and chooses Approve -- **THEN** the suspended step resumes exactly where it paused (the turn is not restarted), and the loop continues to completion - -#### Scenario: An expanded conversation does not grace-dismiss on cursor leave -- **WHEN** a conversation is expanded and the cursor moves away from the notch area -- **THEN** the panel stays open (the grace-dismiss applies only to rail and settings modes), and it closes only via an explicit collapse, feature-off, or synchronous teardown paths - -### Requirement: Crossing behind the notch reveals an expanding rail of parked sessions -Crossing the cursor **up behind the notch** SHALL reveal a **rail** of parked-session cards, **always including the persistent "+ New chat" card**. The **"+ New chat" card SHALL lead** the rail — with the **settings (gear) affordance stacked above it** — and the session cards SHALL follow it ordered **most-recently-used first** (by last activity), so the last-used session sits immediately after the "+ New chat" card and older sessions trail toward the far end. The reveal trigger SHALL be the physical notch cutout itself — the rail reveals ONLY when the cursor crosses UP into the notch band (its usable "behind the notch" space, reachable because within the notch's horizontal span the cursor travels up to the physical top), **never** when it merely grazes the resting strip below the notch. On a **notchless/external** display, which has no notch to cross, the trigger SHALL be a **thin band hugging the physical top edge at top-center** (slamming the cursor to the very top edge), mimicking the same deliberate gesture. On a notched display the rail SHALL **emerge from the notch as a downward extension** — its top spanning up behind the notch (reaching the physical top), the cards spreading **downward** below the notch; on a notchless/external display the rail SHALL **hang below** the top-center tab. The rail SHALL be **non-scrollable**: the panel SHALL be sized to **hug every rendered card** (the persistent "+ New chat" card plus one per session) and SHALL **expand** as sessions are added rather than scrolling, up to a screen-fraction safety ceiling. On a notched display the panel SHALL be sized so the **centered** card row has **symmetric** vertical padding of the **notch height plus a small clearance** on top and bottom — so the cards clear the notch (never clipped at the head) and sit **balanced**, not shoved to the bottom. The reveal SHALL reuse the edge-gated cursor-reveal pattern (a passive global cursor monitor needing **no new permission**; geometry read only when the cursor is near the trigger/live region while hidden; a unified live area — the resting zone, the rail, **and the notch band** — with a grace-period dismiss, so once shown, moving the cursor **up into the notch OR back down onto the rail docks** rather than dismisses; a coarse re-feed while shown). The resting strip below the notch SHALL remain a keep-open bridge inside the live area (it just no longer triggers the reveal). The reveal SHALL be gated by a small, **user-configurable dwell**: the cursor SHALL remain crossed behind the notch (inside the trigger) **continuously** for the dwell before the rail reveals, so a quick pass THROUGH the notch (reaching for the menu bar or travelling to another corner) does not pop the dock; leaving the trigger before the dwell elapses SHALL cancel it (a re-entry restarts it), a dwell of **zero** SHALL reveal immediately, and the reveal timing SHALL not depend on continued cursor movement (a perfectly still cursor still reveals once the dwell elapses). The dwell SHALL apply to the hidden→shown transition ONLY (keep-open is instant). The grace-period dismiss SHALL apply **only while the panel is in rail or settings mode** (an expanded conversation never grace-dismisses). The panel SHALL open and close by **growing from a point behind the notch to the full dock and shrinking back** — a fluid, spring-driven "droplet" spread anchored at its top edge, so the panel's **border itself stretches out of the point** (unfurling downward and out to both sides) and shrinks back into it. There SHALL be **no opacity fade** — the animation is geometric (the shape/border stretching), not a cross-fade. A **rail↔expanded** size change (opening a card / new chat into the conversation panel, or collapsing back) — and likewise a **rail↔settings** one — SHALL stretch the border between the two sizes rather than snapping or fading. Teardown for the grace-dismiss MAY defer the order-out until the shrink completes, but restore and feature-off teardown SHALL remain **synchronous** (the ghost-on-Space-switch path). - -#### Scenario: Crossing behind the notch reveals the rail -- **WHEN** the cursor crosses up behind the notch (into the notch band on a notched display), or slams to the physical top edge at top-center on a notchless/external display -- **THEN** the rail of parked-session cards — always including the "+ New chat" card with the gear affordance above it — is revealed, emerging downward from the notch on a notched display, or hanging below the tab on a notchless/external display - -#### Scenario: Grazing the strip below the notch does not reveal -- **WHEN** the cursor moves across the resting strip just below the notch (without crossing up into the notch band) while the rail is hidden -- **THEN** the rail is NOT revealed (only crossing behind the notch triggers it) - -#### Scenario: A dwell gates the reveal -- **WHEN** a reveal dwell is configured and the cursor crosses behind the notch but leaves the trigger before the dwell elapses -- **THEN** the rail does not reveal; it reveals only once the cursor stays crossed behind the notch continuously for the dwell (a dwell of zero reveals immediately, and a still cursor still reveals) - -#### Scenario: The panel grows from a point and shrinks back (no fade) -- **WHEN** the rail is revealed and later grace-dismissed -- **THEN** it grows from a point behind the notch to the full dock (its border stretching out of the point, no opacity fade) and shrinks back into the point on a fluid spring anchored at the top edge, and a reveal arriving mid-shrink cancels the teardown - -#### Scenario: Opening or closing a conversation stretches the border between sizes -- **WHEN** a card is opened (or a new chat created) into the expanded conversation panel, the settings zone is opened or closed, or the conversation is collapsed back to the rail -- **THEN** the panel's border fluidly stretches between the two sizes (rather than snapping or fading), the frame animation running to completion without a per-tick reposition snapping it - -#### Scenario: The dock expands with each session rather than scrolling -- **WHEN** a new session is added to the dock -- **THEN** the panel widens to hug all cards (the "+ New chat" card plus one per session) and the rail does not scroll (within the parked-session cap) - -#### Scenario: Cards sit balanced and clear of the notch -- **WHEN** the rail is revealed on a notched display -- **THEN** the panel is sized so the centered card row has symmetric padding of the notch height plus a small clearance on top and bottom — the cards clear the notch (none clipped behind it) and sit balanced, not shoved to the bottom - -#### Scenario: Moving up into the notch docks rather than dismisses -- **WHEN** the rail is shown and the cursor moves up into the notch band above the resting zone -- **THEN** the rail stays shown (the notch band is inside the contiguous live area), and it does not grace-dismiss - -#### Scenario: Leaving the zone and rail dismisses after a grace period in rail mode -- **WHEN** the panel is in rail mode and the cursor leaves the zone, the rail, and the notch band for longer than the grace period -- **THEN** the rail dismisses, ordered out synchronously - -#### Scenario: The reveal needs no new permission -- **WHEN** the cursor-reveal monitor is installed -- **THEN** it observes cursor moves passively and requires no Input Monitoring or other new permission diff --git a/openspec/changes/notch-timeline-and-tuning/tasks.md b/openspec/changes/notch-timeline-and-tuning/tasks.md deleted file mode 100644 index caf7f86..0000000 --- a/openspec/changes/notch-timeline-and-tuning/tasks.md +++ /dev/null @@ -1,37 +0,0 @@ -# Tasks — notch-timeline-and-tuning - -## 1. Segment model (Core, display-only) - -- [x] 1.1 Add `TurnSegment` (`kind: .thinking | .answer`, `text`; Codable/Equatable/Sendable) and `AgentMessage.segments: [TurnSegment]?` in `AI/Agent/AgentConversation.swift`; keep `text`-only assembly untouched and extend the file-header invariant note -- [x] 1.2 Add born-with tuning optionals `reasoningOverride: Bool?` / `contextTokens: Int?` to `AgentConversation` (decode-safe, like `skillID`/`autoApprove`) -- [x] 1.3 `AgentConversationTests`: Codable round-trip with/without segments + tuning fields; legacy JSON (no new keys) decodes; assembly still excludes thinking/segments - -## 2. Engine streaming (NotchSessionEngine) - -- [x] 2.1 Add `@Published liveSegments: [TurnSegment]` with the append-or-coalesce rule; feed it from the plain `runTurn` token loop (both channels), keeping `thinking` and `.conversing(partial:)` accumulating as today -- [x] 2.2 Generalize `runRoutedTurn`'s ordered thinking stream to `(TokenChannel, String)` events; wire `AgentLoop.onResponseToken` so the routed answer streams into `liveSegments` and `.conversing(partial:)` -- [x] 2.3 Persist the timeline: `appendAssistantTurn` stamps `segments` (plus existing `text`/`thinking`) and clears `liveSegments`; clear on turn failure/cancel paths too -- [x] 2.4 Tuning snapshot: inject `tuningDefault: () -> (reasoning: Bool, contextTokens: Int)`; `startNew()` stamps both onto the newborn conversation; `bind()` prefers stored values with the legacy fallback (reasoningDefault + injected budget); compactor uses the per-session budget -- [x] 2.5 `ConversationSessionTests`: interleave order (T,R,T,R script → segments in arrival order) on both paths; routed answer streams before settle; persisted segments on the settled message; born-with tuning stamped, retained across rebind, legacy conversation falls back - -## 3. Notch tuning setting - -- [x] 3.1 Add `NotchTuning` (quick/balanced/deep/max; `reasoning`, `contextTokens(modelMax:)`, titles) in Core; persist `AppSettings.notchTuning` (default `.balanced`, Keys/load/reset) -- [x] 3.2 Wire `makeNotchSessionEngine` (`AppCoordinator`) to pass a settings-backed `tuningDefault` (clamped to the model max via the `AgentContextBudgetProvider` idiom) -- [x] 3.3 `AppSettingsTests` round-trip + default; `NotchTuning` mapping unit test - -## 4. Notch UI — timeline transcript - -- [x] 4.1 Rebuild `NotchConversationView.thread`: per-message segment rendering (answer = `BidiText` bubble, thinking = muted per-block collapsible), legacy fallback (flat `thinking` → one leading block), live turn rendered from `engine.liveSegments`; drop the bottom `thinkingSection` and the `.conversing` partial bubble; auto-scroll pins on `liveSegments` -- [x] 4.2 Live thinking block streams expanded, collapses to a compact expandable row at settle; no new repeating animation (idle-CPU-spin rule) - -## 5. Notch UI — settings zone - -- [x] 5.1 Add `Mode.settings` to `NotchHomeZoneViewModel`; gear icon button stacked above `NotchNewChatCard`; `onOpenSettings` callback threaded like `onNewSession` -- [x] 5.2 `NotchHomeZoneLayout` settings solve + controller transition (same border-stretch resize as `expandSession`; back affordance returns to rail; panel never takes key in settings mode; grace-dismiss active as in rail mode; synchronous teardown paths untouched) -- [x] 5.3 Settings-zone view: the thinking+context slider (4 discrete stops over `NotchTuning`), stop title + "thinking on/off · N tokens" caption, bound to `AppSettings.notchTuning` - -## 6. Verify & ship - -- [x] 6.1 `swift build` + `swift test` green (MLX-free Core); `xcodebuild` compile-verify only if app-target files changed -- [x] 6.2 Update the `ai-parked-sessions` notes in `CLAUDE.md`-adjacent docs only if behavior notes exist there (none expected); keep the delta spec in this change dir diff --git a/openspec/changes/remove-local-ai/design.md b/openspec/changes/remove-local-ai/design.md new file mode 100644 index 0000000..120b2ee --- /dev/null +++ b/openspec/changes/remove-local-ai/design.md @@ -0,0 +1,29 @@ +# Design — remove-local-ai + +## D1 — Snapshot before the cut + +The full-featured app is preserved twice before anything is deleted: the **`v1` branch** (exact source) and the **`v1.0.0` tag** (triggers the release pipeline → a notarized DMG on GitHub Releases). Nothing in this change is unrecoverable. + +## D2 — Favorites migration: lossy decode, not a throwing one + +`FavoritesStore` loads the record with `try? decode` and **reseeds on failure**. Removing the `.aiCommand` case from `LaunchItemKind` would make every stored record containing one fail to decode — and the reseed would silently wipe the user's real bands on first launch. So: + +- `ContextBand` gains a custom `init(from:)` that decodes items through a `FailableItem` wrapper (`try? LaunchItem(from:)`): an undecodable **item** is dropped; the **band** (and record) survive. This also future-proofs any later kind retirement. +- `Favorites.currentSchemaVersion` bumps to **3**. The v3 step in `FavoritesStore.migrate` removes the seeded "AI" band — recognized by the retired `AIBand.bandID` sentinel UUID, hardcoded in the migration — but only when it is empty after the item drop (a renamed/repurposed band with surviving items belongs to the user and is kept). `homeBandID` is re-pointed if it referenced the removed band. +- `.fileEntry` was never persisted (synthetic band), so it needs no migration; `.action(.speakLastResponse)` items fall out via the same lossy decode. + +## D3 — GestureBindings shrink is decode-safe by construction + +The stored bindings blob is JSON; `JSONDecoder` ignores unknown keys. Removing the `canvas` and `filesDrill` properties means older blobs (which still carry those keys) decode into just the `switcher` binding — no version bump, no custom decoder. A corrupted blob already fell back to defaults. + +## D4 — What deliberately survives + +- **`ScrollEventTap` + the odometer + edge auto-repeat**: untouched; only the canvas/Files special cases inside them were removed. +- **Dock previews, ⌘-Tab, window groups, minimize-all, keyboard language, device link**: untouched keepers. (Device link is the *remote-sync* feature — a candidate for a later cleanup, explicitly out of scope here.) +- **The error-handling convention** (one taxonomy per domain, mapped at the boundary, bounded + non-blocking surfaces) originated in the AI work and remains the house style — restated in CLAUDE.md without the AI examples. +- **`xcodebuild` in `build-app.sh`**: kept. MLX was the original reason, but the script's packaging/signing flow is proven; `swift build` remains the agent verify loop. The generic `*.bundle` copy loop stays (harmless no-op today). +- **The `openspec/changes/archive/`**: all `ai-*`/`files-*` history stays — it documents decisions this repo may want to revisit. + +## D5 — Danger zone loses the AI-models card + +`DangerZoneSelection.aiModels` is gone; `.appData` now removes the whole Application Support root (there is no `models/` survivor split to preserve). Already-downloaded weights on a user's disk are not auto-deleted by this change — deleting user data as a side effect of an update would be wrong; the release notes point at the folder instead. diff --git a/openspec/changes/remove-local-ai/proposal.md b/openspec/changes/remove-local-ai/proposal.md new file mode 100644 index 0000000..23f3f54 --- /dev/null +++ b/openspec/changes/remove-local-ai/proposal.md @@ -0,0 +1,31 @@ +# Proposal — The great cleanup: remove the local AI stack and the Files band + +## Why + +The app drifted far from its intent. The features that earn daily use are the **window switcher**, the **four-finger launcher**, and **clipboard history**. The on-device AI platform (the command band, the conversational canvas, background agents, the notch timeline, voice/computer-use, the model fleet) and the Files band were built, shipped — and went unused. They carry enormous surface area: a second build path (MLX/Metal via `xcodebuild`), a multi-gigabyte model download, five extra TCC usage strings, ~150 source files, and roughly half the active spec estate. Cutting them refocuses the product and collapses the maintenance burden. + +The pre-cleanup app is preserved in full: the **`v1` branch** and the **`v1.0.0` release tag** (a Developer-ID-signed, notarized DMG built by CI) snapshot everything before this change. + +## What Changes + +**Removed outright (code, tests, specs):** + +- **On-device AI, all of it:** the `GemmaRuntime` target (MLX/Gemma 4, image/video runtimes), `Sources/ThreeFingerSwitcher/AI/` (executor, agent loop, tool routing, memory, skills, media, fleet, voice, computer-use, background autonomy/audit, parked sessions), the AI command canvas, the notch home zone + timeline, the region picker, the Hub AI page + fleet roster + autonomy views, `ModelManagementView`, the `.aiCommand` launch-item kind, the `speakLastResponse` system action, and the `mlx-swift`/`gemma-4-swift-mlx`/`flux-2-swift-mlx` package dependencies. +- **The Files band, all of it:** `Sources/ThreeFingerSwitcher/Files/`, `FilesBandView`/`FilesDwellArming`/`BubbleMorph`, `HubFilesPage`, the recognizer's Files-drill sub-state, the `.fileEntry` kind, and the `files*` settings. +- **Settings:** ~31 AI keys and ~14 Files keys (plus their enums) leave `AppSettings`; the canvas and Files-drill gesture bindings leave `GestureBindings` (older stored blobs still decode — unknown JSON keys are ignored). +- **Specs:** the 11 AI capability folders (`ai-*`, `on-device-ai-runtime`, `computer-use-tools`, `voice-conversation`, `screen-region-picker`, `selection-io`) and the 3 `files-*` folders are deleted; 16 active `ai-*`/notch change folders are deleted (the archive is untouched — it stays as design history). + +**Modified (surgical):** + +- `AppCoordinator` (~1,100 lines of AI/Files wiring out), `GestureRecognizer` (canvas-resolve, notch-flick, and Files-drill sub-states out), the launcher stack (`LauncherModel`/`View`/`OverlayController` lose canvas + Files threading), the Hub (AI/Files pages and seams out), onboarding (AI card + AI tour band out), `main.swift` (runtime injection out), `Package.swift`, `build-app.sh` (metallib copy rationale retired), `release.yml` (the Xcode 26.5 pin existed only for MLX), `Info.plist` (Calendar/Reminders/Contacts/Microphone/Speech usage strings out), README/CLAUDE.md. +- Mixed specs trimmed in place: `launcher-overlay`, `configuration-hub`, `tunable-settings`, `favorites-editor`, `first-run-onboarding`, `permissions-onboarding`, `gesture-recognition`, `launch-actions`, and the active `add-gesture-previews-and-bindings` delta (bindings are switcher-only now). + +**Migration (the one real design point):** + +- `Favorites` bumps to **schema v3** with a **lossy per-item band decode** (`ContextBand.FailableItem`): a stored record still carrying a retired kind (`.aiCommand`, `.action(.speakLastResponse)`) decodes with that item **dropped** instead of the whole record failing and being reseeded (which would wipe the user's bands). The v3 migration then removes the seeded "AI" band (recognized by its sentinel id) once it holds nothing else. + +## Impact + +- Affected code: −~200 files, −~40k lines; `swift build`/`swift test` now cover the whole app (no more MLX/`xcodebuild`-only target). 805 tests green (was 1,733; the deleted suites were AI/Files-only). +- No behavior change to the keepers: switcher, launcher, clipboard history, Dock previews, ⌘-Tab, window groups, minimize-all, keyboard language, device link, onboarding. +- Stale UserDefaults keys from removed settings are left in place (harmless); downloaded model weights are NOT auto-deleted — the user can remove `~/Library/Application Support/ThreeFingerSwitcher` content or the HF cache by hand if they want the disk back. diff --git a/openspec/changes/remove-local-ai/tasks.md b/openspec/changes/remove-local-ai/tasks.md new file mode 100644 index 0000000..ec53c1f --- /dev/null +++ b/openspec/changes/remove-local-ai/tasks.md @@ -0,0 +1,33 @@ +# Tasks — remove-local-ai + +## 1. Snapshot +- [x] 1.1 Branch `v1` + tag `v1.0.0` pushed; release CI builds the notarized DMG. +- [x] 1.2 Delete the merged side branches and their worktrees (all were fully merged into main). + +## 2. Delete +- [x] 2.1 `Sources/GemmaRuntime/`, `Sources/ThreeFingerSwitcher/AI/`, `Files/`, `RegionPicker/`. +- [x] 2.2 Overlay: AI canvas, notch overlays, region picker, media player, BidiText, FilesBandView, FilesDwellArming, BubbleMorph. +- [x] 2.3 App/Settings/Hub/Gesture: ParkController, ModelManagementView, HubBackgroundAutonomy, HubFleetRosterView, HubFilesPage, FlickExcursionClassifier. +- [x] 2.4 Tests: 46 AI + 11 Files suites. +- [x] 2.5 Docs: the four ai-agent-v2 docs + notch-geometry-reference. +- [x] 2.6 openspec: 15 AI/Files capability specs, 16 active AI change folders, `explore/ai-command-band`. + +## 3. Surgical edits +- [x] 3.1 `Package.swift` + `Package.resolved` + `main.swift` (MLX deps and injection out). +- [x] 3.2 `AppSettings` (45 keys + enums out), `GestureBindings` (switcher-only), `GesturePose` (canvasResolve out). +- [x] 3.3 `GestureRecognizer` (canvas-resolve / notch-flick / Files-drill sub-states out). +- [x] 3.4 Launcher stack: `LaunchItem` (kinds + speakLastResponse out, lossy decode in), `FavoritesStore` (v3 migration, no fold-in/seeded AI band), `LaunchService`, `LauncherModel`/`View`/`OverlayController`/`GridLayout`, `OverlayController` (SwitcherPanel simplification). +- [x] 3.5 Hub: `HubView` (destinations/seams), `HubFeaturePages` (AIPage out, demos trimmed), `BandsCanvas` (AI source/form/pickers out), `HubBindingPicker`, `HubOverviewPage`, `HubPreviewModels`. +- [x] 3.6 Onboarding: `WizardTourBands` (AI band out), `WizardContext`/`FirstTouchWizardModel`/`WizardActs`. +- [x] 3.7 `AppCoordinator` (~1,100 lines of AI/Files wiring out), `StatusItemController`, `AppDataReset`, `KeyboardSwitcherTap`. +- [x] 3.8 `build-app.sh`, `release.yml` (Xcode pin relaxed), `Info.plist` (5 usage strings out). +- [x] 3.9 README + CLAUDE.md rewritten for the refocused app. + +## 4. Specs (in place) +- [x] 4.1 Trim `launcher-overlay`, `configuration-hub`, `tunable-settings`, `favorites-editor`, `first-run-onboarding`, `permissions-onboarding`, `gesture-recognition`, `launch-actions`. +- [x] 4.2 Rewrite the active `add-gesture-previews-and-bindings` gesture-bindings delta (switcher-only). + +## 5. Verify +- [x] 5.1 `swift build` clean. +- [x] 5.2 `swift test` — 805 tests, 0 failures. +- [ ] 5.3 User: `INSTALL=1 ./scripts/build-app.sh` and confirm switcher / launcher / clipboard / Dock previews on the real signed build. diff --git a/openspec/explore/ai-command-band/design.md b/openspec/explore/ai-command-band/design.md deleted file mode 100644 index 5eec406..0000000 --- a/openspec/explore/ai-command-band/design.md +++ /dev/null @@ -1,233 +0,0 @@ -# AI Command Band — design exploration (thinking, not a committed change) - -> Status: **explore seed.** No proposal/specs/tasks yet. Promote to `openspec/changes/` when ready. - -## Context - -Clipboard history left us three reusable substrates: **the Band** (non-activating scrub/dwell/lift surface with a master-detail preview pane), **the Hand-off** (capture the front app, act into it via paste/keystroke/AX — `LaunchService.pasteEntry` + `frontAppProvider` + `capturedFrontApp`), and **the Stream/Store** (privacy-aware recorder + on-disk index/blobs). The clipboard band is a *streamed* band. Favorites are *static* bands. The unexplored third kind is **verbs that act on what you're looking at right now** — and the most valuable verb is an LLM. - -The AI Command Band is a launcher band whose items are **named AI commands** (curated prompt presets). You scrub to a command, lift, and it takes your current **input** (the text you have selected in the front app, or the clipboard, or an OCR of a screen region), runs it through an on-device model, and routes the **output** somewhere — back into the app in place, or into a background **task** that *does something* (calendar event, save to a project, open a tool with a generated payload, send-to). - -The keyboardless identity is the design language, not a limit: you can't type a freeform prompt at use time, so commands are **presets** configured in Settings. The grid of canned verbs *is* the interface. - -## Goals / Non-Goals - -**Goals:** -- A band of user-configurable AI commands, fired by the existing dwell/lift, non-activating, working in **any** app. -- Two execution modes from one mechanism: - - **In-place edit** — transform the selection and paste the result back where the user was (fix grammar, make concise, translate, explain, rewrite). - - **Background task** — interpret the input and *perform an action* without pasting: "add to calendar" (parse a meeting agreement → `EventKit` event), "save to project N" (append/file the highlighted research under a project), "open Claude Code with this idea" (launch a tool with a generated prompt/payload), "send to ". -- **On-device by default** — target the latest Apple Silicon Macs and run a local model (Gemma) for zero cost, full privacy, no network. (Runtime/model choice is its own exploration — see Open Questions and the companion Gemma notes.) -- The **preview pane** (already built for clipboard) becomes the **streaming output canvas**: watch the model's result stream, then resolve it with a fresh four-finger swipe — a **down swipe commits** (paste / run the task), a **horizontal swipe discards** (an up swipe is ignored; a stray re-lift is a no-op). -- Each command is fully **customizable**: name, prompt template, input source, output target, and (for tasks) the action it triggers. - -**Non-Goals:** -- No freeform prompt typing at use time (keyboardless). Custom prompts are authored in Settings. -- Not a chat UI. One-shot command → result. (A "send to Claude Code / a chat app" task hands conversation off to a real chat tool.) -- Not cloud-first. A cloud fallback is an explicit, consent-gated option, never the default. -- No autonomous multi-step agents firing irreversible actions without confirmation (see Risks). - -## Decisions (proposed) - -### The command model -A command is a value type roughly: -``` -AICommand { - name, icon, tint - input: .selection | .clipboard | .screenRegionOCR | .none - prompt: String // template with tokens: {input}, {date}, {app}, {url?} - output: .replaceSelection | .pasteAtCursor | .previewOnly - | .runTask(TaskKind) | .sendTo(Destination) - model: .onDevice | .cloud(provider) // v1 = onDevice (Gemma 4) only - confirmBeforeRun: Bool // defaults on for irreversible tasks; user-overridable -} -``` -Commands live in a new band kind (`.aiCommand`) or as a configurable band in the favorites editor, persisted like favorites. - -### Input via Accessibility selected-text (the key enabler) -Read the front app's selection with AX `AXSelectedText` / `AXSelectedTextRange` (no clipboard clobber; we already hold Accessibility). Fallback: synthesize ⌘C, read pasteboard, then **restore** the prior clipboard. For `.replaceSelection`, set `AXSelectedText` if settable, else paste. This single capability — "what is the user looking at / has highlighted" — underpins this whole band (and the Transforms band). **Note (codebase grounding below): none of the AX selected-text read/replace exists yet — it's the one genuinely new primitive we must build.** - -### The preview pane is the latency-and-trust surface -On lift, the band stays open showing the command's preview pane; the model **streams** tokens into it. The user sees the result form before committing. A fresh four-finger **down swipe commits**: pastes for in-place, or runs the task. A fresh four-finger **horizontal swipe discards** (an up swipe is ignored; a stray re-lift while the canvas is open is a no-op). This turns model latency from a wait into a *review*, and makes "it rewrote my text, undo undo" impossible. - -### Background tasks reuse existing plumbing + structured output -The "task" modes are the agentic part. The model emits **structured output** (JSON for an action), which we parse and dispatch through machinery we already have or can add thinly: -- **Add to calendar** → `EventKit` (new permission, consent-gated) from a parsed `{title, start, end, attendees, notes}`. -- **Save to project N** → append the text (+ source URL/app) to a per-project note/file (reuses the clipboard store pattern; projects tie into the Workspaces idea, openspec/explore/project-workspaces/design.md). -- **Open Claude Code / a tool with this idea** → `LaunchService` opens the app / runs a script/shortcut with the generated prompt as the payload (e.g. write a prompt file + `open -a`, or a Shortcut). -- **Send to ** → a destination adapter (shell-out / Shortcut / URL scheme) fed the (optionally AI-refined) content. -"Send-to" is therefore *not* a separate feature — it's an AI command whose output target is a destination, and the model can refine the text and pick/open the necessary app as part of the task. - -### Model strategy — one model now, swappable later (decided 2026-06-08) -- **v1 ships exactly one model: Gemma 4 31B** (the largest, most capable Gemma 4 — text + vision, max reasoning quality), run **in-process via MLX-Swift**. We target the **strongest Apple Silicon only** (M4/M5-class, ample unified memory) and use the best the hardware allows — *no* accommodation for low-end Macs. -- **Build the seam for "many models, swap per task" now, but only wire Gemma 4 for v1.** Every call goes through an `LLMRuntime` protocol so a per-command/per-task model choice (and later Apple Foundation Models / cloud / a future Gemma) drops in without touching feature code. We *test only Gemma 4 models* first; alternates and fallbacks come later. -- **Audio is the one capability the flagship lacks** (see family table): 31B and 26B-A4B are text/vision/video only. When audio verbs land (a later sub-feature), that command routes — via the same seam — to the largest audio-capable Gemma 4 (the 12B). This is exactly what the swap-per-task seam is for. - -## Risks / Trade-offs - -- **Irreversible actions from a probabilistic model.** Calendar events, sends, file writes must show a **confirmation preview** of the parsed action before firing (the preview pane already exists for this). `confirmBeforeRun` defaults on for any side-effecting task. -- **No freeform prompt at use time.** Mitigated by rich Settings authoring + token templates; the few canned verbs cover the 90%. -- **Model RAM / first-run download (GBs).** Targeted at strong Macs; explicit opt-in + download UX; quantized (QAT 4-bit) weights; keep the model resident between calls or lazy-load with a spinner in the preview. -- **AX selected-text not exposed by every app.** Fallback to ⌘C-with-restore; if even that fails, fall back to clipboard input and `.previewOnly`/paste output. -- **Latency on big models.** The streaming preview makes it tolerable; on the target hardware even the 31B streams faster than reading speed (see numbers below). -- **Privacy of selection → model.** On-device keeps it local (on-brand). A cloud command is a per-command consent gate, clearly labeled. -- **Scope creep into "an agent."** Hold the line at *one-shot command → reviewed result/action*. Multi-step autonomy is out. - -## Open Questions - -- ~~**Which Gemma + which runtime**~~ → **RESOLVED:** Gemma 4 31B, in-process via **MLX-Swift**; `LLMRuntime` seam for later per-task swap. See model section below. -- ~~**Structured output / tool-calling** strict/validated?~~ → **RESOLVED (path chosen):** **schema-targeted structured output** — target the task schema, validate, and repair/retry on mismatch, with the model free to **decline** ("not applicable"). Grammar-guided decoding (XGrammar via `mlx-swift-structured`) is one optional technique, not a hard cage. Not regex-after-the-fact. See model section. -- ~~**Older-Mac / no-model fallback**~~ → **RESOLVED:** target the newest M4/M5-class Macs only, ship the single best model, no low-end fallback for v1. -- **Confirmation UX** for side-effecting tasks in a non-activating, keyboardless overlay (dwell-to-confirm? a distinct "armed-danger" state?). — still open. -- **Where do commands live** in the editor, and how are prompt templates authored without a heavy text UI? — *informed* by codebase grounding (AppSettings array + a new editor section, or the `FavoritesEditorView` inspector pattern; build a synthetic band on open like clipboard). Authoring-UI shape still open. -- **Multimodal**: Gemma 4 sees images, so `.screenRegionOCR` becomes "screen region → vision model" for "explain this chart / extract this table". yes, this will be a sub featuer to analys screens and scence lateer on. we could wuery "what dose this this products"/how to use that thing.." and so on. (Screen Recording permission is **already held** for thumbnails — see grounding — so screen capture is free.) - ---- - -## Gemma & the on-device model layer — research notes (captured 2026-06-08) - -> Snapshot of the on-device model landscape **at the moment this feature was scoped**. Gemma moves *fast* (a full generation jump inside ~12 months), so **re-verify against the model card before building**. These are the "companion Gemma notes" referenced above. Sources at the end. -> -> **Design stance (per owner, 2026-06-08):** We power the app with **Gemma 4** — the whole family is fair game, but **v1 uses one model and tests only Gemma 4**. Everything sits behind a swappable `LLMRuntime` seam so any capable model (a future Gemma, Apple Foundation Models, cloud) can drop in *later*, and so a command can pick a different Gemma 4 size per task (e.g. audio). We aim for the **best result on strong hardware**, not broad-Mac compatibility. - -### TL;DR recommendation - -- **Engine:** the **Gemma 4** family (Apache-2.0), run **in-process via MLX-Swift** (no sidecar, no daemon — fits an unsandboxed menubar app cleanly). -- **v1 model: Gemma 4 31B (dense)** — the flagship, max reasoning quality (85.2 % MMLU-Pro). Text + vision + video. ~17–20 GB at 4-bit. The right pick when "best result" beats "lightest footprint" and the Mac is strong. -- **Speed alternative (only if interactive latency disappoints): Gemma 4 26B-A4B (MoE, ~3.8 B active)** — near-31B quality (82.6 %) at noticeably higher tok/s. Same modalities (no audio). Keep it in our pocket; the seam makes switching trivial. -- **Audio (later, via the seam): Gemma 4 12B** — the *largest* audio-capable variant. 31B/26B do **not** take audio. -- **Structured output: schema-targeted + validate/repair** (model may decline; XGrammar via `mlx-swift-structured` optional) for task JSON. -- **Deferred (not v1):** Apple Foundation Models (zero-download macOS-26 alternate) and any cloud provider — behind the same seam. - -### The Gemma 4 family - -Encoder-free multimodal decoder-only transformers. Architectural highlights: -- **No separate vision/audio encoders.** Vision enters via a lightweight embedding (single matmul + positional + norm); raw 16 kHz audio is projected straight into the text token space. Smaller weights, simpler pipeline, one model for all its modalities. -- **Multi-Token Prediction (MTP) drafters** built in → self-speculative decoding → lower latency (helps our streaming preview pane). -- **140 languages**, Apache-2.0, **QAT** (quantization-aware-trained) 4-bit checkpoints published alongside the base weights. -- Google also shipped an **official Skills/agentic repo** for Gemma 4. -- **Release timeline (verify):** core family (E2B/E4B/26B-A4B/31B) launched ~**2026-04-02**; the **12B** dense, "first mid-sized model with native audio, runs in 16 GB", followed ~**2026-06-03**. - -| Variant | Active / Total | Modalities | Context | 4-bit size | MMLU-Pro | Role for us | -|---|---|---|---|---|---|---| -| E2B | 2.3 B | text · vision · **audio** · video | 128K | ~3.6 GB | 60.0 % | (too small) | -| E4B | 4.5 B | text · vision · **audio** · video | 128K | ~5 GB | 69.4 % | small audio option | -| 12B (dense) | 12 B | text · image · **audio** | 256K | ~8 GB* | n/a* | **audio verbs (later)** | -| 26B-A4B (MoE) | 3.8 B / 26 B | text · vision · video | 256K | ~14 GB | 82.6 % | **speed alternative** | -| **31B (dense)** | 30.7 B | text · vision · video | 256K | ~17–20 GB | **85.2 %** | **★ v1 default** | - -\* 12B 4-bit size / MMLU-Pro not firmly pinned in sources — verify. **⚠ Audio caveat:** audio input is **E2B / E4B / 12B only**. The two highest-quality variants (26B-A4B, 31B) are **text + image + video, no audio**. Audio clips ≤ 30 s; video sampled (≤ 60 s). - -**Rough speed (31B dense, Q4, tok/s):** M4 Max ~40–50 · M4 Pro 36 GB ~20–35. The MoE 26B-A4B is materially faster (only ~3.8 B active). Human reading ≈ 4–5 tok/s, so **even the 31B streams several× faster than the user reads** → the "latency-as-review" preview-pane bet holds comfortably on our target hardware. - -### Why Gemma 4 fits *this* feature (capability → leverage) - -``` - Gemma 4 capability → AI Command Band use - ───────────────────────────────────────────────────────────────────── - Encoder-free native vision → .screenRegionOCR becomes a real - vision query: "what is this - product / how do I use this thing" - — one model, screenshot + prompt. - (Screen Recording perm already held.) - Schema-targeted JSON → the BACKGROUND-TASK modes: model - (validate/repair, declinable) returns add_to_calendar{…} / - save_to_project{…} / send_to{…} — - or declines; we validate + dispatch - (EventKit, LaunchService, adapter) - + review (default-on, overridable). - MTP drafters + MLX in-process → fast streaming → preview = review. - 256K context (26B / 31B) → whole-document inputs, multi-file - "save to project" summarization. - Native audio (12B / E4B) → later voice verbs, reached via the - seam (31B itself has no audio). - Apache-2.0 → ship / bundle weights, commercial-OK. -``` - -### Running it on Apple Silicon — runtime options - -| Option | Shape | Pros | Cons / gotchas | -|---|---|---|---| -| **MLX-Swift, in-process** ★ | Swift package linked into the app | No daemon, native, fastest on Apple Silicon, unified-memory streaming, fits unsandboxed menubar app | Must build w/ `xcodebuild` (Metal shaders) | -| **LiteRT-LM sidecar** | `litert-lm serve` → OpenAI-compatible localhost | Google's official agentic stack + Skills; OpenAI API | Extra process to ship/manage | -| **Ollama sidecar** | local daemon, OpenAI API | Trivial model mgmt; **MLX-powered on Apple Silicon since 03/2026** | ~10–20 % slower than raw MLX; another process | -| **llama.cpp / GGUF** | sidecar/lib | Ubiquitous; GBNF grammar-constrained JSON | Slower than MLX on M-series | - -**Chosen: MLX-Swift in-process.** Building blocks that already exist: -- `ml-explore/mlx-swift-lm` — official LLM+VLM Swift lib. -- `VincentGourbin/gemma-4-swift-mlx` — **native Gemma 4 multimodal (text+vision+audio+video) for Apple Silicon**. API surface: `Gemma4Pipeline`, `ChatSession`, `chatStream(prompt:)` → `for try await token in stream`, plus LoRA + speculative decoding + a `Gemma4TokenFilter` for thinking-mode. Requires macOS 14+, Swift 6, Xcode 16, **`xcodebuild` (not `swift build`)** for Metal. -- `petrukha-ivan/mlx-swift-structured` — **grammar-guided decoding for MLX-Swift** (XGrammar): grammar-based generation, JSON-Schema → `Decodable`, a `@Generable`-style typed path, **streaming `PartiallyGenerated`**, and tool-call handling. We treat this as **one optional technique**; the default path is target-the-schema + validate/repair so the model keeps latitude to decline. - -### Structured output / tool-calling — the agentic backbone - -**Decision: schema-targeted output, validate-and-repair, let the model decline — don't cage the decoder.** Each `TaskKind` (`add_to_calendar`, `save_to_project`, `open_tool_with_payload`, `send_to`) is one JSON schema. We ask the model for output matching it, then **validate and repair/retry on mismatch** before decoding into a Swift `Decodable`, and we let the model **decline ("not applicable")** rather than fabricate values to satisfy the schema. Hard grammar-caging is a **latent footgun** — it degrades content and forces invented fields — so we treat constrained decoding (`mlx-swift-structured`, XGrammar) as **one optional technique** for the cases that benefit, not a mandatory cage on every call. This still sidesteps brittle regex parsing of Gemma's native tool-call tokens (incomplete in mlx-swift-lm #259) by targeting a schema and validating. - -We **validate and (by default) show an action-review preview** before executing any side-effecting task; `confirmBeforeRun` **defaults on** for side effects but is **user-overridable**. The model's job ends at "produce the structured result or decline"; *we* own validation, dispatch, and the review gate. - -### Deferred alternates (behind the same seam — not v1) - -- **Apple Foundation Models** — built into macOS 26+ (no download, free, always present, ~3 B). `@Generable` macro → type-safe guided generation, built-in tool calling, stateful sessions, a few lines of Swift. Far less capable / less multimodal than big Gemma 4. Future role: the zero-download fallback and simplest in-place edits — *not* the v1 engine. -- **Cloud provider** — consent-gated per command, clearly labeled. Later. -- **Tiny on-device router** — a sub-1B model could pre-classify which command/task an input wants and cheaply extract the call before waking the big model. Model-agnostic, deferred; the seam supports it. - -### The swappable model layer (the "any capable model" requirement) - -Everything hides behind one seam. Sketch (design, not code): - -``` -protocol LLMRuntime { - var capabilities: Set // .text .vision .audio - func stream(_ req: LLMRequest) -> AsyncThrowingStream - func structured(_ req: LLMRequest, schema: Schema) async throws -> T -} - -GemmaMLXRuntime : LLMRuntime // v1 — gemma-4-swift-mlx + mlx-swift-structured -AppleFMRuntime : LLMRuntime // later — Foundation Models, @Generable -CloudRuntime : LLMRuntime // later — consent-gated -``` - -`AICommand.model` selects the runtime; for v1 it always resolves to `GemmaMLXRuntime`. A `ModelManager` owns download / residency / eviction and (later) **per-task model selection** — e.g. an audio command asks for `.audio` capability and gets the 12B instead of the 31B. **No feature code references Gemma directly** — only `LLMRuntime`. Adding a model later = one conformer. - -### Open risks specific to the model layer - -- **Model residency / load time** — even on a strong Mac, a ~17–20 GB model has a cold-load cost; lazy-load with a "loading model" preview state, keep resident between calls, evict on memory pressure. -- **First-run download UX** — multi-GB (QAT 4-bit); explicit opt-in, resumable, verified (sha). Apple FM (later) could cover the gap while it downloads. -- **Audio capability gap** — the flagship 31B has no audio; audio verbs must route to 12B/E4B via the seam. Bake the capability check into `ModelManager` from the start so it's not a retrofit. -- **Version churn** — Gemma cadence is fast; the `LLMRuntime` seam + a model registry (id → size/sha/url/caps) keep upgrades off the feature code. -- **`swift build` won't compile MLX** (needs Metal/`xcodebuild`) — clashes with CLAUDE.md's "verify with `swift build`" rule; the model layer needs a stubbed `LLMRuntime` for unit tests and a separate `xcodebuild` verification path. - ---- - -## Codebase grounding — what we reuse vs. what we build (mapped 2026-06-08) - -Read against `Sources/ThreeFingerSwitcher/`. The three substrates are real and reusable; only the AX selected-text primitive and the model layer are genuinely new. - -### The Band (UI surface) — reuse, extend with one new kind -- **Non-activating overlay:** `SwitcherPanel(NSPanel)`, `canBecomeKey/Main = false`, `[.borderless, .nonactivatingPanel]` — `Overlay/OverlayController.swift:112`. Launcher recreates its panel per `show()` bound to the current Space — `Overlay/LauncherOverlayController.swift:206`. -- **Dwell / lift:** `LauncherModel` holds `arming`/`armed` + `armingToken`; dwell is a `DispatchWorkItem` re-armed on every move (`LauncherOverlayController.swift:175`, `:191`); **lift fires only if `armed`** and hides the panel *before* firing (`:76`). Haptic on arm (`:201`). -- **Preview pane = the streaming canvas:** `Overlay/ClipboardBandView.swift` is the master-detail (left 340 px key list + right value pane). Async content already updates via `.task(id:)` — **that is exactly the hook for streaming model tokens** into the right pane. -- **Band kinds are a flat enum, not a protocol:** `enum LaunchItemKind: Codable` (8 cases) — `Launcher/LaunchItem.swift:165`; `ContextBand` is a value type (`:215`). The clipboard band is **synthetic**, built fresh on open by `Clipboard/ClipboardBandBuilder.swift` and detected by a sentinel UUID. -- **Extension points for our band:** (1) add `case .aiCommand(AICommand)` to `LaunchItemKind`; (2) add a fire branch in `Launcher/LaunchService.swift:47`; (3) render icon in `Overlay/LauncherView.swift:108`; (4) for the preview-canvas, add a conditional band view in `LauncherView.body` (mirror the `currentBandIsClipboard` branch) and build a synthetic AI band on open like `ClipboardBandBuilder`. Wire-in point: `AppCoordinator.swift:372`. - -### The Hand-off (input/output into the front app) — mostly reuse, **one missing primitive** -- **Paste back:** `LaunchService.pasteEntry()` = NSPasteboard write → `app.activate()` → `postKey(⌘V, toPid:)` with a 40 ms settle (`Launcher/LaunchService.swift:74`). Deliberately paste, not AX-set — paste is universal. -- **Front-app capture:** `AppCoordinator.capturedFrontApp` + injected `frontAppProvider` (`AppCoordinator.swift:41`,`:48`); `postKey()` CGEvent synth posts per-pid (`LaunchService.swift:354`). -- **AX is already used** for windows/menus/geometry via `axCopy/axString` (`AXPrivate.swift:15`) and `AXUIElementCreateApplication`. -- **⚠ MISSING (build this):** reading **`kAXSelectedTextAttribute`** off `AXFocusedUIElement`, the **`AXSelectedText` set / replace**, and a **clipboard save-restore** helper for the ⌘C fallback. None exist today. This is the core new primitive (call it `SelectionService`). -- **Permissions:** Accessibility is **already held** (so AX selected-text read is "free"); **Screen Recording is already held** for thumbnails (so `.screenRegionOCR` vision is "free"); Input Monitoring held — `PermissionsService.swift:19`. New permission needed only for **EventKit** (calendar task). - -### The Stream/Store + authoring — reuse the patterns -- **On-disk store pattern:** `Clipboard/ClipboardStore.swift` = Codable JSON index + externalized blobs (>16 KB) under `~/Library/Application Support/…`, content-hash names. Reuse this shape for "save to project" output. -- **Structured user config:** `Favorites` = one Codable JSON blob in `UserDefaults` via `FavoritesStore.mutate()` (`Launcher/FavoritesStore.swift:31`). `AppSettings` = `@MainActor ObservableObject`, scalar-per-key UserDefaults with `didSet` persist (`Settings/AppSettings.swift`). -- **Command authoring:** persist `[AICommand]` as a `@Published` array on `AppSettings` (encode in `didSet`), OR a dedicated store if it grows. Editor: a new section in `Settings/SettingsView.swift`, or reuse the kind-specific **`ItemInspector`** pattern in `Settings/FavoritesEditorView.swift:587` (it already does per-kind config UIs — the model for editing prompt template / input source / output target). - -### Build/verify caveat -MLX needs Metal → must build via `xcodebuild`, not `swift build`. Keep the model behind `LLMRuntime` with a stub conformer so the rest of the feature still verifies under the project's `swift build`/`swift test` rule (CLAUDE.md). - -### Sources (verify before building — captured 2026-06-08) - -- Gemma 4 launch — blog.google "Introducing Gemma 4 12B" / "Gemma 4: byte-for-byte most capable open models"; deepmind.google/models/gemma/gemma-4 -- Family/specs — ai.google.dev/gemma/docs/core + model_card_4; HF google/gemma-4-{31B,26B-A4B,12B,E4B}; audio-modality: dev.to "Gemma 4's Audio and Video Inputs" + HF model cards -- Function calling — ai.google.dev/gemma/docs/capabilities/text/function-calling-gemma4 -- Apple Silicon / Swift — gemma4.dev MLX guide; sudoall.com Apple-Silicon benchmarks; github VincentGourbin/gemma-4-swift-mlx, ml-explore/mlx-swift-lm (incl. issue #259), **petrukha-ivan/mlx-swift-structured** (XGrammar constrained decoding) -- Apple FM — developer.apple.com/documentation/FoundationModels diff --git a/openspec/specs/ai-background-autonomy/spec.md b/openspec/specs/ai-background-autonomy/spec.md deleted file mode 100644 index c1f3c59..0000000 --- a/openspec/specs/ai-background-autonomy/spec.md +++ /dev/null @@ -1,119 +0,0 @@ -# ai-background-autonomy Specification - -## Purpose - -Define the trust boundary for agent tool steps that run while a session is parked (in the background): a blast-radius classification (contained / whitelisted / dangerous) over the tool-routing write-policy tiers, a user-visible, user-editable whitelist, the effective-tier resolver wired into the routing seam, an append-only audit log of every background action, and the per-step auto-vs-escalate decision (auto runs in the background; confirm waits parked; dangerous escalates to the foreground needs-you state). -## Requirements -### Requirement: Blast-radius tiers classify every tool's background autonomy -The system SHALL classify every agent tool/sink into one of three **blast-radius tiers**, collapsing onto the existing write-policy tiers (`auto`/`confirm`/`dangerous`, owned by the tool-routing capability): - -- **CONTAINED** — the app's own stores (agent memory, project notes) and read-only retrieval. These tools SHALL ship `auto` and SHALL run in the background **even when the session is parked** (still audited). -- **WHITELISTED** — a `confirm` tool whose argument **target** (a path under a trusted prefix, or a command matching a trusted pattern) matches the user whitelist. These SHALL resolve to `auto` **only on a match** (still audited). -- **DANGEROUS** — delete, overwrite an existing file, arbitrary shell, anything off-list, and the Claude-handoff cost. These SHALL ship `dangerous` and SHALL be **foreground-only**: a parked session encountering one SHALL escalate to the needs-you state rather than act in the background. - -The whitelist SHALL be able to **lower `confirm` to `auto`** on a match but SHALL **never** lower `dangerous` — a destructive operation is dangerous regardless of where it lands. CONTAINED-ness SHALL be intrinsic to the tool (its shipped `auto` tier), not a whitelist entry, so the app's own stores are auto with an empty whitelist. - -#### Scenario: A contained store write runs in the background while parked -- **WHEN** a parked agent runs an agent-memory write or a save-to-project append (CONTAINED) -- **THEN** it executes without confirmation in the background and is recorded in the audit log - -#### Scenario: A whitelisted write runs auto; an off-list one does not -- **WHEN** a parked agent runs a `confirm` write whose target matches a whitelist entry -- **THEN** the effective tier is lowered to `auto` and it runs in the background (audited) -- **AND WHEN** the target matches no whitelist entry, the effective tier stays `confirm` and the step does not act in the background - -#### Scenario: A dangerous operation is never lowered by the whitelist -- **WHEN** a delete / overwrite-existing / arbitrary-shell tool would write inside a whitelisted folder -- **THEN** its tier remains `dangerous` and it is treated as foreground-only, not auto - -### Requirement: A user-visible, user-editable whitelist is the security boundary -The system SHALL provide a **whitelist** the user can view and edit: a list of **trusted folder path prefixes** and a list of **trusted command patterns**. The whitelist SHALL **default empty** for arbitrary entries — a fresh install SHALL trust nothing on the wider filesystem — while the app's **memory store and project-note store SHALL be pre-trusted as CONTAINED** (auto without any whitelist row). Whitelist edits SHALL persist with the same keys/defaults/reset semantics as the other AI opt-ins. - -A **path target** SHALL match the whitelist iff its standardized absolute path (with `..`/symlinks resolved before matching) has a trusted prefix at a **path-component boundary**. A **command target** SHALL match iff it matches a trusted pattern as an anchored glob. A target that is both a command and a path SHALL require **both** a command-pattern match and a path-prefix match (the stricter rule wins). - -#### Scenario: Default-empty trusts nothing arbitrary -- **WHEN** the whitelist is at its default -- **THEN** no arbitrary path or command is trusted, and only the CONTAINED memory/project stores run auto - -#### Scenario: Path matching respects component boundaries and resolves escapes -- **WHEN** a write targets a path that, once standardized, lies under a trusted prefix at a component boundary -- **THEN** it matches the whitelist; a path that only shares a string prefix (e.g. `/Notes2` vs trusted `/Notes`) or that escapes via `..`/symlinks does NOT match - -#### Scenario: A command and a path must both be trusted -- **WHEN** a whitelisted command is aimed at a path outside every trusted prefix -- **THEN** the target does not match the whitelist and the step stays `confirm` - -#### Scenario: Whitelist edits persist -- **WHEN** the user adds or removes a trusted folder or command pattern -- **THEN** the change persists across launches and is preserved by a reset-to-defaults like the other AI opt-ins - -### Requirement: Effective-tier resolution wired into the tool-routing seam -The system SHALL provide the concrete write-policy resolver that the tool-routing loop already injects (the `WritePolicyResolving` seam), replacing the stand-alone descriptor-identity default. The resolver SHALL compute a step's **effective tier** as: the descriptor's own tier, lowered to `auto` **only** when the tool is CONTAINED or its target matches the whitelist, and **never** lowered when the descriptor is `dangerous`. The resolver SHALL satisfy the existing descriptor-only seam method AND offer a target-aware resolution for the routed call's argument target, **without changing the routing protocol**. - -#### Scenario: Resolution lowers a matched confirm to auto -- **WHEN** the resolver evaluates a `confirm` external tool whose routed target matches the whitelist -- **THEN** the effective tier is `auto` - -#### Scenario: Resolution leaves an unmatched confirm and never lowers dangerous -- **WHEN** the resolver evaluates a `confirm` tool with no matching target, or any `dangerous` tool -- **THEN** the effective tier is `confirm` (unmatched) or `dangerous` (unconditional), respectively - -### Requirement: Append-only audit log of background actions -The system SHALL keep an **append-only audit log** in which **every** agent tool step writes one record — auto, confirmed, declined, escalated, or failed. Each record SHALL carry the session id, the tool name, the **effective tier** the step ran at, a **redacted, short arguments summary** (never raw secrets or full bodies), the outcome (reusing the tool-step status, a failure carrying a clean headline only), a `wasBackground` flag, and a timestamp. The log SHALL be **viewable** both in the notch rail drop-down and on the Hub AI page as a reverse-chronological "what did my agents do while I was away" ledger. Writing a record SHALL be **non-blocking** and SHALL NOT throw into the agent loop; a persistence failure SHALL be surfaced bounded and non-blocking, never as an app-modal alert, and SHALL NOT lose the in-memory record or stop the running step. - -The ledger SHALL support exactly **one removal operation**: an explicit, **user-initiated per-session purge** (the purge-delete gesture on that session's expanded conversation — see `ai-parked-sessions`), which removes every record attributed to that session id from the in-memory ring **and** the durable file. Nothing else SHALL remove or edit records (the bounded cap's oldest-first trimming aside): no agent, tool, or background process can invoke the purge, so the model can never erase its own tracks — only the user's deliberate gesture can. A purge SHALL be non-blocking like a write; a persistence failure during the file rewrite SHALL surface on the same bounded, non-blocking channel as a write failure while the in-memory ring remains purged. - -#### Scenario: Every background action is recorded -- **WHEN** a parked agent runs, declines, escalates, or fails a tool step -- **THEN** one record is appended with the tool, effective tier, redacted args, outcome, `wasBackground=true`, and a timestamp - -#### Scenario: Arguments are redacted, failures carry a clean headline -- **WHEN** a step's arguments contain a long body or an embedded secret, or the step fails -- **THEN** the record's arguments summary is short and redacted (no raw secret), and a failure stores only the clean headline, not raw OS error text - -#### Scenario: The ledger is viewable in the notch and the Hub -- **WHEN** the user opens the notch rail drop-down or the Hub AI page after the agent worked in the background -- **THEN** the recent records appear in reverse-chronological order, distinguishing background actions from foreground ones - -#### Scenario: An audit persistence failure does not break the agent -- **WHEN** persisting the audit log to disk fails -- **THEN** the running step is unaffected, the record stays in the in-memory log, and the failure is shown as a bounded non-blocking banner (never an app-modal alert) - -#### Scenario: A user purge removes one session's records everywhere -- **WHEN** the user purge-deletes a session -- **THEN** every audit record attributed to that session id disappears from the in-memory ring and the durable file, and records of other sessions are untouched - -#### Scenario: Only the user can purge -- **WHEN** any agent tool step, background runner, or automated process executes -- **THEN** it has no path to remove or edit audit records — the purge is reachable only from the user's explicit gesture - -### Requirement: A parked agent decides auto vs escalate per step -The system SHALL decide, per tool step, whether a **parked** session acts automatically or escalates, from the step's **effective tier** and the session's park state: - -- an `auto` effective tier SHALL **run in the background** and be audited; -- a `confirm` effective tier on a parked session SHALL **wait parked** (resolved later when the session is brought back, via the routing approval affordance) without escalating; -- a `dangerous` effective tier on a parked session SHALL **escalate to the foreground** by raising the needs-you state, with a clean one-line reason; -- when the session is **active** (foreground), this decision SHALL defer to the existing routing approval gate (no escalation). - -**A parked wait or escalation SHALL genuinely pause the agent loop:** the loop SHALL end the turn with a distinct **paused-awaiting-user outcome** that appends **no fabricated final answer** and is **never** classified as a completed task — the step is neither run nor skipped, and the session's recorded tool steps show it pending. A **wait-parked** session SHALL become dormant (parked with no scheduled next-run time — not runnable) until the user brings it back. An **escalation SHALL route through the parked-sessions controller seam** (not a raw scheduler mutation) so the needs-you state is **persisted durably and repainted on the rail** — a needs-you raised in the background survives relaunch and is never a silent in-memory flag. An escalation SHALL be resolved by the user pulling the session back and approving, at which point the step proceeds as a foreground step; a skip SHALL decline it. A session already in needs-you SHALL NOT be double-escalated. A completed side effect SHALL NOT be rolled back — the audit log records it; it is not reversed. - -#### Scenario: Auto step runs in the background -- **WHEN** a parked session reaches a step whose effective tier is `auto` -- **THEN** the step runs in the background and is audited, without user attention - -#### Scenario: Confirm step waits parked without escalating -- **WHEN** a parked session reaches a step whose effective tier is `confirm` -- **THEN** the loop pauses with the paused-awaiting-user outcome (no fabricated answer, not a completed task), the session becomes dormant, and the step is resolved when the user brings the session back — without raising the needs-you state - -#### Scenario: Dangerous step escalates to needs-you -- **WHEN** a parked session reaches a step whose effective tier is `dangerous` -- **THEN** the loop pauses with the paused-awaiting-user outcome, the escalation routes through the parked-sessions controller seam so the needs-you badge is persisted and repainted with a clean reason, and the step waits until the user pulls the session back and approves it - -#### Scenario: A paused step never fabricates completion -- **WHEN** a parked session's loop pauses on a confirm or dangerous step -- **THEN** no final answer is synthesized for the turn, the outcome is not a completed task, and the session is not dismissed — the pending step is observable state - -#### Scenario: Already-escalated session is not double-escalated -- **WHEN** a session is already in needs-you and another dangerous step arrives -- **THEN** the session is not escalated again (the badge count already reflects the pending attention) and the step is recorded as awaiting approval - diff --git a/openspec/specs/ai-command-band/spec.md b/openspec/specs/ai-command-band/spec.md deleted file mode 100644 index 2d3df6d..0000000 --- a/openspec/specs/ai-command-band/spec.md +++ /dev/null @@ -1,168 +0,0 @@ -# ai-command-band Specification - -## Purpose - -Define the AI command feature surfaced in the launcher: a Codable AI-command value model persisted as a first-class band item inside Favorites (folded in from the former separate store), inline authoring on the Hub's Bands page, prompt-template token resolution, command input acquisition with sensible fallback, in-place output routing into the captured front app, a one-time migration of legacy commands into a normal "AI" band, and a single opt-in that gates the on-device model's download/residency (but never the visibility of AI items). -## Requirements -### Requirement: AI command value model and persistence -The system SHALL define an AI command as a Codable value type carrying: a stable identifier, a display name, an icon and tint, an **input source** (`selection` | `clipboard` | `clipboardImage` | `screenRegion` | `none`), a **prompt template** string, an **output target** (`replaceSelection` | `pasteAtCursor` | `previewOnly` | `task(TaskKind)` | `sendTo(Destination)`), a **model selector** (v1: on-device Gemma 4), and a **confirmBeforeRun** flag. The **image** input sources (`clipboardImage`, `screenRegion`) SHALL require a vision-capable model (their `requiredCapabilities` include `vision`); the text sources (`selection`, `clipboard`, `none`) SHALL require only the text capability — this mapping SHALL be derived statically from the input source and SHALL NOT depend on runtime clipboard contents. An AI command SHALL be a first-class, persisted **band item**: it is stored **inside the Favorites record** as the item of a context band, persists across launches, applies immediately when changed, and SHALL be movable between bands like any other item. Its stable identifier SHALL be preserved across edits and the migration into the band model (the identifier keys the executor and the UI). - -#### Scenario: Commands persist across launches -- **WHEN** the user creates AI commands and relaunches the app -- **THEN** the same commands, in the same order within their bands, are present - -#### Scenario: Commands are stored as Favorites band items -- **WHEN** AI commands exist and the Favorites record is inspected -- **THEN** the commands are present as items of context bands (they are no longer kept in a separate store) - -#### Scenario: confirmBeforeRun defaults on for side-effecting output but is honored -- **WHEN** a command with a side-effecting task or send-to destination is created without an explicit choice -- **THEN** its `confirmBeforeRun` defaults to true; if the user later sets it false, that stored value is honored at run time (not overridden) - -#### Scenario: A clipboard-image command requires a vision-capable model -- **WHEN** a command whose input source is `clipboardImage` is inspected for its required capabilities -- **THEN** the required capabilities include `vision` (statically, the same as a `screenRegion` command), so model selection routes it to a vision-capable model - -### Requirement: Migration of existing AI commands into the band model -On upgrade from a version that stored AI commands separately, the system SHALL perform a one-time, idempotent migration that imports the previously stored commands into a normal, editable context band (named "AI", carrying the prior AI-band color) appended to the Favorites record, **preserving each command's identifier and order**. The migration SHALL run at most once (guarded by the Favorites schema version), SHALL retire the old separate storage only after the new record is written successfully, and SHALL NOT duplicate commands on subsequent launches. A fresh install (no prior commands) SHALL instead seed the default "AI" band as part of normal seeding; an upgrading user who never opted in SHALL likewise get the default "AI" band seeded. - -#### Scenario: Existing commands move into an editable AI band -- **WHEN** a user who had configured AI commands upgrades to the band model -- **THEN** their commands appear as items of a normal "AI" band in the Favorites record, with identifiers and order preserved - -#### Scenario: Migration is idempotent -- **WHEN** the migrated app is relaunched -- **THEN** the commands are not imported again and no duplicate "AI" band is created - -#### Scenario: Fresh install seeds the default AI band -- **WHEN** the app is first installed with no prior AI commands -- **THEN** a default "AI" band with the seeded starter commands is created as part of normal seeding - -### Requirement: Prompt template token resolution -A command's prompt template SHALL support tokens that are resolved at fire time from the captured context: `{input}` (the acquired input text), `{date}` (the current date/time), `{app}` (the captured front app's name), `{url}` (the front document/page URL when available), and `{lang}` (the command's **active runtime language** — the canvas selection, falling back to the persisted last choice, falling back to the command's language-parameter default). Unknown tokens SHALL be left untouched, and a missing `{url}`/`{app}` SHALL resolve to an empty string rather than failing the command. A `{lang}` token on a command that declares no language parameter SHALL resolve to an empty string rather than failing. - -#### Scenario: Input token is substituted -- **WHEN** a command with template `"Fix the grammar:\n{input}"` is fired on selected text -- **THEN** the model receives the template with `{input}` replaced by the selected text - -#### Scenario: Missing context token degrades to empty -- **WHEN** a template references `{url}` but the front app exposes no URL -- **THEN** `{url}` resolves to an empty string and the command still runs - -#### Scenario: Language token resolves to the active language -- **WHEN** a translate command with template `"Translate to {lang}:\n{input}"` and active language "Hebrew" is fired -- **THEN** the model receives the template with `{lang}` replaced by "Hebrew" - -#### Scenario: Language token without a language parameter degrades to empty -- **WHEN** a command that declares no language parameter references `{lang}` -- **THEN** `{lang}` resolves to an empty string and the command still runs - -### Requirement: Command input acquisition -When an AI command is fired, the system SHALL acquire its input according to the command's configured input source: `selection` reads the front app's selected text, `clipboard` reads the current clipboard text, `clipboardImage` reads the current clipboard **image** (the live pasteboard image, normalized to PNG) as the request's image input, `screenRegion` captures a screen region for the vision model, and `none` supplies no input. If a **text** source yields nothing the system SHALL fall back where sensible (selection → clipboard). An **image** source (`clipboardImage`, `screenRegion`) SHALL NOT fall back to text. If no input can be obtained for a command that requires input — including a `clipboardImage` command fired with no image on the clipboard — the system SHALL surface a clear "no input" state rather than running the model on empty. - -#### Scenario: Selection source reads highlighted text -- **WHEN** a `selection` command is fired with text highlighted in the front app -- **THEN** the highlighted text is used as `{input}` - -#### Scenario: Empty selection falls back to clipboard -- **WHEN** a `selection` command is fired with no current selection but text on the clipboard -- **THEN** the clipboard text is used as `{input}` - -#### Scenario: Clipboard-image source reads the pasteboard image -- **WHEN** a `clipboardImage` command is fired with an image on the clipboard -- **THEN** the clipboard image (as PNG bytes) is supplied to the runtime as the request's image input and the vision result streams into the canvas - -#### Scenario: Clipboard with no image surfaces no input -- **WHEN** a `clipboardImage` command is fired and the clipboard holds no image -- **THEN** the preview shows a clear "no input" state and the model is not invoked (no fallback to text) - -#### Scenario: No input available is surfaced -- **WHEN** an input-requiring command is fired and neither selection nor clipboard yields text -- **THEN** the preview shows a clear "no input" state and the model is not invoked - -### Requirement: In-place output routing -For non-task output targets, after the model result is committed the system SHALL route it: `replaceSelection` replaces the front app's selected text (via selection replace when settable, else paste), `pasteAtCursor` pastes the result at the insertion point, and `previewOnly` shows the result without writing into the app. Output SHALL be delivered into the app that was frontmost when the launcher opened. - -#### Scenario: Replace selection commits in place -- **WHEN** a `replaceSelection` command's streamed result is committed -- **THEN** the front app's selection is replaced by the result, in the app that was frontmost at open - -#### Scenario: Preview-only never writes -- **WHEN** a `previewOnly` command's result is committed -- **THEN** the result is shown but nothing is written into the front app - -### Requirement: Opt-in gates the model, not the visibility of AI items -The AI commands opt-in SHALL gate the on-device model's **download and residency** only; it SHALL default to OFF, and with it off no model is downloaded or loaded. The opt-in SHALL NOT hide AI-command items from the launcher: AI-command items SHALL always appear and be fireable regardless of the opt-in. When AI is off (or the selected model is not yet available), firing an AI-command item SHALL open its preview canvas in an availability state offering to enable/download rather than silently doing nothing (see launcher-overlay). Turning the opt-in off after use SHALL evict the resident model from memory. - -#### Scenario: Off by default downloads no model -- **WHEN** the app runs for the first time -- **THEN** the AI commands opt-in is off and no model is downloaded or loaded - -#### Scenario: AI items appear even when the opt-in is off -- **WHEN** the opt-in is off and the launcher is opened on a band containing AI commands -- **THEN** the AI-command items still appear and can be fired - -#### Scenario: Turning the opt-in off frees the model -- **WHEN** the user turns the opt-in off after using the feature -- **THEN** the resident model is evicted from memory (the AI items remain visible but inert until re-enabled) - -### Requirement: Runtime-adjustable command parameter -A command MAY declare a **runtime parameter** that is chosen at fire time rather than baked into the template. In v1 the only runtime parameter SHALL be a **target language** (`language(default:)`). When a command declares a language parameter, its **active language** SHALL be resolved into the `{lang}` token at fire time, and the system SHALL expose the parameter for in-canvas adjustment (see launcher-overlay). Re-resolving the parameter SHALL re-run the command (see launcher-overlay) using the existing cancellable generation. The runtime parameter SHALL be optional and default to absent, so commands without one behave exactly as before. - -#### Scenario: A command without a runtime parameter is unchanged -- **WHEN** a command that declares no runtime parameter is fired -- **THEN** it resolves and runs exactly as before, with no parameter UI - -#### Scenario: A language-parameter command resolves its active language -- **WHEN** a command declaring `language(default: "English")` is fired with no prior choice -- **THEN** its active language is "English" and `{lang}` resolves to "English" - -### Requirement: Per-command runtime-parameter persistence -The system SHALL persist the **last chosen value** of a command's runtime parameter **per command** (keyed by the command's identifier) and use it as the **default active value on the next run** of that command. Persistence SHALL NOT mutate the stored command (so seeds, catalog presets, and band edits are unaffected); it SHALL be an out-of-band preference that survives relaunch. The command's declared default SHALL be the cold-start fallback when no value has been chosen yet. The store SHALL be **best-effort**: an entry whose command no longer exists (deleted, or a never-re-added copy) MAY be pruned and SHALL otherwise be ignored, and a persisted entry SHALL never block deleting its command. - -#### Scenario: A chosen language is remembered next run -- **WHEN** the user picks "Spanish" while running a translate command, then fires the same command again later -- **THEN** the command's initial active language is "Spanish" - -#### Scenario: Two language commands remember independently -- **WHEN** the user sets one command to "Hebrew" and another to "French" -- **THEN** each command re-opens with its own remembered language - -#### Scenario: Persistence does not change the stored command -- **WHEN** a runtime-parameter value is chosen and persisted -- **THEN** the stored `AICommand` (and any catalog/seed source it came from) is unchanged; only the out-of-band preference records the choice - -#### Scenario: A deleted command's persisted value is harmless -- **WHEN** a command with a persisted runtime-parameter value is deleted -- **THEN** the deletion succeeds and the now-orphaned persisted entry is ignored (and MAY be pruned), never resurfacing on an unrelated command - -### Requirement: Screen-region commands acquire via the region picker before the canvas -When a `screenRegion` command is fired, the system SHALL acquire its image through the interactive region picker **before** opening the preview canvas: the launcher is dismissed to reveal the desktop, the user designates a region, and only the captured image is then supplied to the command. The executor SHALL accept this **pre-supplied** image and SHALL NOT perform its own additional screen capture for that command. If the pick is **cancelled** (a click without a drag), the command SHALL **abort** — no preview canvas opens and the model is not invoked. A cancelled pick is distinct from the "no input" state: it is a deliberate user abort, not a missing-input failure, and SHALL NOT surface a "no input" error. - -#### Scenario: The picker runs before the canvas opens -- **WHEN** a `screenRegion` command is fired -- **THEN** the launcher is dismissed and the region picker runs first; the preview canvas opens only after a region is captured - -#### Scenario: Executor uses the pre-supplied image -- **WHEN** a region is captured for a `screenRegion` command -- **THEN** the executor fires the request with that captured image and does not perform an additional screen capture - -#### Scenario: Cancelled pick aborts the command -- **WHEN** the region pick is cancelled (a click without a drag) -- **THEN** no preview canvas opens and the model is not invoked, and no "no input" failure is shown - -### Requirement: A fired AI command is one-shot and never becomes a conversation -A fired AI command SHALL execute as a **one-shot preset action**: it auto-fires exactly one generation from its configured input and template, streams the result into the preview canvas, and resolves by the canonical two-finger compass — **DOWN commits** (only when the canvas is scrolled to the top), **RIGHT/LEFT discards**, **UP scrolls**. The preview canvas SHALL NOT offer a typed composer, SHALL NOT accept follow-up turns, and SHALL NOT require the overlay panel to become the key window. A quick action SHALL NOT create a durable conversation or session of any kind: it SHALL never be handed to the notch dock's store or scheduler, SHALL never appear as a notch dock card, and SHALL leave nothing behind after commit or discard (the notch dock is populated exclusively by sessions born at the notch — see `ai-parked-sessions`). Re-running, adjusting the runtime parameter, and the armed confirmation for side-effecting tasks SHALL all operate within the same single-fire model. - -#### Scenario: A preset auto-fires once and resolves by the compass -- **WHEN** the user fires an AI command item on selected text -- **THEN** the command auto-fires one generation that streams into the preview canvas, a two-finger DOWN at the top commits the result to the command's output target, and a horizontal swipe discards it - -#### Scenario: The canvas offers no conversation affordance -- **WHEN** the preview canvas is open for a fired AI command -- **THEN** there is no typed composer, no follow-up-turn affordance, and no way to continue the result as a multi-turn thread from this surface - -#### Scenario: A quick action never reaches the notch dock -- **WHEN** an AI command is fired and then committed or discarded — including while it is still streaming -- **THEN** no durable session is created, nothing is written to the notch dock's store or scheduler, and no card for it ever appears on the notch dock's rail - diff --git a/openspec/specs/ai-command-catalog/spec.md b/openspec/specs/ai-command-catalog/spec.md deleted file mode 100644 index 4baee51..0000000 --- a/openspec/specs/ai-command-catalog/spec.md +++ /dev/null @@ -1,72 +0,0 @@ -# ai-command-catalog Specification - -## Purpose -TBD - created by archiving change expand-ai-command-catalog. Update Purpose after archive. -## Requirements -### Requirement: Categorized AI command catalog -The system SHALL provide a curated **catalog** of ready-made `AICommand` presets, each a complete, fireable command (name, icon, tint, input source, prompt template, output target, and — where applicable — a runtime parameter). Every preset SHALL belong to exactly one **category**, and the catalog SHALL cover at least these categories: **Writing**, **Tone**, **Understand**, **Translate**, **Developer**, **Reply**, **Capture** (side-effecting tasks), **Vision** (screen-region), and **Format**. The catalog SHALL be the single source of the presets used both by the Bands-editor browser and by the fresh-install seed. - -#### Scenario: Catalog spans the named categories -- **WHEN** the catalog is enumerated -- **THEN** it contains presets grouped under Writing, Tone, Understand, Translate, Developer, Reply, Capture, Vision, and Format, and every preset declares its category - -#### Scenario: Each preset is a complete, fireable command -- **WHEN** any catalog preset is inspected -- **THEN** it carries a name, icon, input source, prompt template, and output target sufficient to fire without further editing - -### Requirement: Catalog browser source in the Bands editor -The Bands editor's AI source SHALL be a **catalog browser** (mirroring the system-actions browser): it SHALL list presets grouped by category, and SHALL let the user **add a single preset** to the active band in one click. The browser SHALL also let the user **add a whole category as a band** (creating a new band named after the category, carrying the **category's color**, populated with that category's presets); adding a category SHALL **append** a new band even when a band of the same name already exists (it does not merge into or replace the existing one). A trailing **"Custom command"** entry SHALL remain that adds a blank, editable AI command (preserving the prior blank-then-edit flow). Adding a preset or category SHALL NOT require the AI opt-in. - -#### Scenario: Browse by category and add one preset -- **WHEN** the user opens the AI source, picks a category, and clicks a preset -- **THEN** that preset is added as an `.aiCommand` item to the active band and selected for editing - -#### Scenario: Add a whole category as a band -- **WHEN** the user chooses "add as a band" for a category -- **THEN** a new band named after the category, carrying the category's color, is created and populated with that category's presets - -#### Scenario: Adding a category appends rather than merging -- **WHEN** the user adds a category as a band and a band of the same name already exists -- **THEN** a new band is appended (the existing same-named band is not merged into or replaced) - -#### Scenario: Custom blank command still available -- **WHEN** the user chooses the "Custom command" entry -- **THEN** a blank, editable AI command is added (input `selection`, template `{input}`, output preview-only) as before - -#### Scenario: Browser works regardless of the AI opt-in -- **WHEN** the AI opt-in is off and the user adds a preset from the catalog -- **THEN** the command is added normally (the opt-in gates only the model, never authoring or visibility) - -### Requirement: Added presets are independent editable copies -Adding a catalog preset SHALL produce an **independent copy**: the added command SHALL receive a freshly minted identifier (the catalog template's identifier is a stencil, never the live item's id), so the same preset MAY be added multiple times without identifier collision, and editing an added command SHALL NOT mutate the catalog or any other added copy. - -#### Scenario: The same preset can be added twice -- **WHEN** the user adds the same catalog preset to a band twice -- **THEN** two items exist with distinct identifiers, each independently editable - -#### Scenario: Editing an added command does not change the catalog -- **WHEN** the user edits a command that was added from the catalog -- **THEN** the catalog preset and any other added copies are unchanged - -### Requirement: Fresh-install seed is drawn from the catalog -The fresh-install "AI" band SHALL be composed from a **curated subset of the catalog** rather than a separate hand-maintained list, so the seeded defaults and the browsable catalog stay consistent. The seed SHALL remain a single, curated band (not the entire catalog), and SHALL only apply on a fresh install (the existing migration/idempotency guard is unchanged — an upgrading user is not re-seeded). - -#### Scenario: Fresh install seeds a curated catalog subset -- **WHEN** the app is first installed with no prior AI commands -- **THEN** the seeded "AI" band's commands are drawn from the catalog and form one curated band, not the whole catalog - -#### Scenario: Upgrading users are not re-seeded -- **WHEN** an existing user who already has AI commands upgrades -- **THEN** their bands are left untouched and the grown seed is not applied - -### Requirement: Clipboard-image vision presets -The catalog SHALL include at least one **clipboard-image** vision preset whose input source is `clipboardImage` and whose output target is `previewOnly`, so a user can analyze an image already on the clipboard without capturing the screen. These presets SHALL belong to the **Vision** category alongside the screen-region presets, and each SHALL be a complete, fireable command (name, icon, the `clipboardImage` input, a prompt template, and `previewOnly` output) requiring no further editing. - -#### Scenario: Catalog offers a clipboard-image vision preset -- **WHEN** the catalog is enumerated -- **THEN** it contains at least one Vision-category preset whose input source is `clipboardImage` - -#### Scenario: Clipboard-image preset is complete and fireable -- **WHEN** a clipboard-image vision preset is inspected -- **THEN** it carries a name, icon, the `clipboardImage` input source, a prompt template, and `previewOnly` output sufficient to fire without further editing - diff --git a/openspec/specs/ai-command-tasks/spec.md b/openspec/specs/ai-command-tasks/spec.md deleted file mode 100644 index 18ae8ff..0000000 --- a/openspec/specs/ai-command-tasks/spec.md +++ /dev/null @@ -1,121 +0,0 @@ -# ai-command-tasks Specification - -## Purpose - -Define the side-effecting background tasks an AI command can target: schema-driven, validated/repairable/declinable structured output as the action contract, action-review before side effects (default on, user-overridable), and the concrete tasks — add-to-calendar (EventKit), save-to-project, open-tool-with-payload, and send-to-destination. -## Requirements -### Requirement: Tasks use schema-targeted structured output (validated, repairable, declinable) -Each background task kind SHALL be defined by a JSON Schema describing the action's parameters. The model SHALL produce output targeting that schema; the system SHALL **validate** it and **repair or retry** on mismatch within a bounded loop before treating it as an action, and SHALL allow the model to **decline** ("not applicable") rather than fabricate values when the input does not fit the task. The system SHALL NOT depend on hard token-level caging to obtain structure, and SHALL NOT dispatch an action that failed validation. - -#### Scenario: Calendar task produces a well-formed event -- **WHEN** an "add to calendar" command runs on text describing a meeting -- **THEN** the model returns a schema-valid object with the event fields (e.g. title, start, end, attendees, notes) - -#### Scenario: Non-conforming output is repaired, not dispatched raw -- **WHEN** the model output does not satisfy the task schema -- **THEN** the system repairs or retries it; if it still cannot produce a valid action, the dispatcher receives no action and the user is told it could not be produced - -#### Scenario: Model declines when the input does not fit the task -- **WHEN** an "add to calendar" command runs on text that describes no meeting -- **THEN** the model may return a "not applicable" result and no event action is dispatched (it does not invent an event) - -### Requirement: Action review before side effects (default on, user-overridable) -A side-effecting task SHALL present an **action-review preview** (the concrete fields that will be applied) before it executes **when the command's `confirmBeforeRun` is enabled**, and `confirmBeforeRun` SHALL **default to enabled** for side-effecting tasks. The user MAY disable it per command; when disabled, the task commits without the extra action-review step (the baseline deliberate commit still applies). Discarding SHALL always cancel the task with no side effect. - -#### Scenario: Action is shown before it fires (default) -- **WHEN** a calendar/save/open/send task whose command has `confirmBeforeRun` enabled has produced its parsed action -- **THEN** the user sees the action's fields and nothing is applied until they commit - -#### Scenario: Confirmation defaults on for side-effecting tasks -- **WHEN** a side-effecting command is created without an explicit choice -- **THEN** its `confirmBeforeRun` defaults to enabled - -#### Scenario: User may disable review for a trusted task -- **WHEN** the user disables `confirmBeforeRun` on a side-effecting command and commits it -- **THEN** the task executes its side effect without the extra action-review step, honoring the stored value - -#### Scenario: Discard cancels with no effect -- **WHEN** the user discards a task before committing -- **THEN** no event is created, no file is written, no tool is opened, and nothing is sent - -### Requirement: Add-to-calendar task -The system SHALL provide an "add to calendar" task that creates a calendar event from the parsed `{title, start, end, attendees, notes}` via EventKit. Creating the event SHALL require the Calendar permission, requested only at first use (see permissions-onboarding), and SHALL occur only after the confirmation preview is confirmed. - -#### Scenario: Confirmed event is created -- **WHEN** the user confirms a parsed calendar action and Calendar permission is granted -- **THEN** a matching event is created in the user's calendar - -#### Scenario: Permission denied is handled -- **WHEN** Calendar permission is denied -- **THEN** the task does not create an event and the user is told the permission is required - -### Requirement: Save-to-project task -The system SHALL provide a "save to project" task that appends the (optionally model-refined) content, with its source app/URL and a timestamp, to a per-project note on disk, reusing the on-disk store pattern used by clipboard history. The target project SHALL be part of the command/task configuration. - -#### Scenario: Content is appended to the project note -- **WHEN** the user confirms a "save to project N" action -- **THEN** the content plus its source and timestamp are appended to project N's note on disk - -### Requirement: Open-tool-with-payload task -The system SHALL provide an "open tool with this payload" task that generates a payload (e.g. a prompt) and opens a target tool with it — by writing a payload file and opening the tool via the existing launch mechanism, or by invoking a Shortcut — after confirmation. - -#### Scenario: Tool opens with the generated payload -- **WHEN** the user confirms an "open tool with this idea" action -- **THEN** the target tool is opened with the generated payload available to it - -### Requirement: Send-to-destination task -The system SHALL provide a "send to destination" task whose output target is a configured destination (e.g. a Shortcut, a URL scheme, or a shell-out adapter), fed the (optionally model-refined) content after confirmation. Send-to SHALL be modeled as a command output target, not a separate feature. - -#### Scenario: Confirmed content is routed to the destination -- **WHEN** the user confirms a "send to " action -- **THEN** the content is delivered to that destination via its adapter - -#### Scenario: Model may refine before sending -- **WHEN** a send-to command specifies refinement in its prompt -- **THEN** the content delivered to the destination is the model-refined version - -### Requirement: Tool and shortcut task targets are chosen from a list -The command editor SHALL let the user choose an **open-tool** target (an installed app or a Shortcut) and a **send-to Shortcut** destination from an **enumerated list** — the installed applications and the user's Shortcuts — rather than requiring the user to type the exact identifier (which the user cannot reliably know). Selecting an app SHALL store a launchable reference the dispatcher can act on (the app's path), and selecting a Shortcut SHALL store its name, so the authored value is always well-formed without guesswork. A free-text **custom** entry SHALL remain available for targets not in the list (e.g. a not-yet-created Shortcut), and the URL-scheme / shell destinations (which are inherently free-form) SHALL keep their text entry. - -#### Scenario: Open-tool target picked from installed apps and shortcuts -- **WHEN** the user configures an "open tool with payload" task -- **THEN** they pick an installed app or a Shortcut from a list, and the stored tool value is a launchable reference (an app path, or a Shortcut name) — no manual identifier typing is required - -#### Scenario: Send-to Shortcut destination picked from a list -- **WHEN** the user configures a "send to Shortcut" destination -- **THEN** they pick from the list of the user's Shortcuts and the chosen name is stored - -#### Scenario: Custom target remains available -- **WHEN** the desired target is not in the enumerated list (e.g. a Shortcut not yet created) -- **THEN** a custom free-text entry is still available so the command can still be authored - -### Requirement: Add-to-reminders task -The system SHALL provide an "add to reminders" task that creates a reminder from a schema-targeted, validated, **declinable** parsed action (`{title, due?, notes?, priority?}`) via EventKit reminders. The model SHALL be able to decline ("not applicable") rather than fabricate a reminder when the input describes no task. Creating the reminder SHALL require the **Reminders** permission, requested only at first use (see permissions-onboarding), SHALL occur only after the action-review confirmation when `confirmBeforeRun` is enabled (the default for side-effecting tasks), and a denied permission SHALL surface a clean, recoverable failure naming Reminders (per ai-error-handling) rather than silently completing. - -#### Scenario: Confirmed reminder is created -- **WHEN** the user confirms a parsed reminder action and Reminders permission is granted -- **THEN** a matching reminder is created in the user's Reminders - -#### Scenario: Model declines when the input is not a task -- **WHEN** an "add to reminders" command runs on text that describes no task -- **THEN** the model returns a "not applicable" result and no reminder is created - -#### Scenario: Reminders permission denied is handled -- **WHEN** Reminders permission is denied -- **THEN** no reminder is created and the user is told the Reminders permission is required, with a pointer to the relevant System Settings pane - -### Requirement: New-contact task -The system SHALL provide a "new contact" task that creates a contact card from a schema-targeted, validated, **declinable** parsed action (`{name, email?, phone?, organization?, notes?}`) via the Contacts framework. The model SHALL be able to decline ("not applicable") when the input contains no contact details. Creating the contact SHALL require the **Contacts** permission, requested only at first use (see permissions-onboarding), SHALL occur only after the action-review confirmation when `confirmBeforeRun` is enabled (the default for side-effecting tasks), and a failure to save SHALL surface a clean, recoverable failure (per ai-error-handling) rather than reporting success. - -#### Scenario: Confirmed contact is created -- **WHEN** the user confirms a parsed contact action and Contacts permission is granted -- **THEN** a matching contact card is created in the user's Contacts - -#### Scenario: Model declines when there is no contact -- **WHEN** a "new contact" command runs on text containing no contact details -- **THEN** the model returns a "not applicable" result and no contact is created - -#### Scenario: Contacts permission denied is handled -- **WHEN** Contacts permission is denied -- **THEN** no contact is created and the user is told the Contacts permission is required, with a pointer to the relevant System Settings pane - diff --git a/openspec/specs/ai-error-handling/spec.md b/openspec/specs/ai-error-handling/spec.md deleted file mode 100644 index 2670297..0000000 --- a/openspec/specs/ai-error-handling/spec.md +++ /dev/null @@ -1,107 +0,0 @@ -# ai-error-handling Specification - -## Purpose - -Define how the AI feature handles, translates, and surfaces errors so that failures are clean, non-blocking, observable, recoverable, and never silent. Every error the AI feature originates is classified into one shared taxonomy and translated to a concise headline (with opt-in details) by a single central translator; raw vendor/OS errors are mapped at the runtime boundary and never reach the UI; failures become observable states that never stall, are surfaced through non-blocking, bounded surfaces, always offer a way forward, and are reported rather than masked as success. - -## Requirements - -### Requirement: Single error taxonomy and translator -The AI feature SHALL classify every error it originates (provision/download, model load, generation, -structured/task output, selection I/O, and side-effecting tasks) into one shared, bounded taxonomy and -translate it to a user message via a single central translator. The translator SHALL return a concise -human-readable **headline** and an optional, separately-carried **details** payload. The headline SHALL be -derived in priority order: the app's own `LocalizedError` description when the error is an app error; -otherwise a classifier for vendor/OS errors; otherwise a generic fallback ("Something went wrong."). The -same error SHALL produce the same headline regardless of which surface displays it. - -#### Scenario: One error, identical message on every surface -- **WHEN** the same underlying error is shown in the Settings model-status row, in any alert, and in the overlay canvas -- **THEN** all three display the identical concise headline produced by the central translator - -#### Scenario: Unknown error falls back safely -- **WHEN** an error of a type the translator does not recognize is encountered -- **THEN** the headline is a generic safe sentence and the raw error is attached only as opt-in details, never as the headline - -### Requirement: No raw error text in user-facing strings -A user-facing string (any status text, alert body, or canvas caption) SHALL NOT contain raw error text — -i.e. the result of interpolating an `Error` (`"\(error)"`), `String(describing:)` of an error, or the -`.localizedDescription` of an operating-system error directly into the headline. Raw error text MAY appear -only in logs and in the opt-in **details** payload. The app's own error types that are user-facing SHALL -conform to `LocalizedError` with a clean `errorDescription` for every case. - -#### Scenario: Offline provision shows a clean message, not an NSError dump -- **WHEN** the model download is attempted with no internet connection -- **THEN** the Settings status row shows a short connectivity message and never the `Error Domain=…Code=…UserInfo={…}` dump - -#### Scenario: A runtime error is self-describing -- **WHEN** any code reads a runtime error's localized description -- **THEN** it yields the clean per-case message (the runtime error type conforms to `LocalizedError`), not a reflected enum dump or the generic default - -### Requirement: Errors are mapped at the runtime boundary -Raw vendor and operating-system errors SHALL be converted into the shared taxonomy at the seam where they -cross into application code, and SHALL NOT be propagated raw to a view model or the UI. This applies to the -model-download library's errors, `NSURLError`, EventKit, `FileManager`/`FileHandle`, and `Process`. -Diagnostic logging of the original error at the boundary is permitted and encouraged. - -#### Scenario: Download-library error is converted before it leaves the runtime -- **WHEN** the model runtime's prepare step fails with a vendor download-library error -- **THEN** it is converted to the corresponding taxonomy case (e.g. offline, server-unavailable, access-denied, model-load-failed) before it is thrown out of the runtime, and the original error is only logged - -### Requirement: Failure is an observable state that never stalls -A failed AI operation SHALL transition its owning state machine to a failed state carrying the clean headline, -rather than leaving the caller to surface a thrown error. No AI operation SHALL be able to leave its state -machine stuck in an in-progress state because of an unhandled (non-cancellation) error. Cancellation SHALL -continue to be treated as not-a-failure and SHALL NOT be surfaced as an error. - -#### Scenario: A non-cancellation download error resolves the state -- **WHEN** a download fails for any reason other than user cancellation -- **THEN** the model lifecycle state becomes failed with a clean message (it does not remain in "downloading") - -#### Scenario: Cancellation is silent -- **WHEN** the user cancels a download or discards a generation -- **THEN** no error is surfaced and the state returns to its prior resting state, not a failed state - -### Requirement: Error surfaces are non-blocking and bounded -AI error surfaces SHALL keep their host window interactive and SHALL bound the rendered message. A -Settings-scoped failure SHALL NOT be surfaced by an application-modal alert that suspends the window's event -loop; it SHALL use the in-window failed state (or a window-modal/non-modal presentation). User-facing error -text SHALL be length-capped (truncating) with full text available via an opt-in details disclosure and a copy -action, and the containing layout SHALL degrade to scrolling rather than overflowing a fixed frame. - -#### Scenario: Settings stays interactive on failure -- **WHEN** a model preparation fails while the Settings window is open -- **THEN** the window remains scrollable and clickable (no application-modal alert blocks it) and shows the failed state inline - -#### Scenario: A long error never breaks the layout -- **WHEN** the failure carries an unusually long message or details -- **THEN** the status row shows a capped, truncated headline and the content scrolls; it does not overflow or freeze the window - -#### Scenario: Details are available on demand -- **WHEN** the user wants the full technical error -- **THEN** a "Show details"/"Copy" affordance exposes the raw details without showing them inline by default - -### Requirement: A failed state offers recovery -A surfaced AI failure SHALL be paired with a way forward — a retry, a dismiss, or a pointer to the relevant -system setting — so a failure is never a dead end. - -#### Scenario: A failed download offers retry -- **WHEN** the model download is in a failed state in Settings -- **THEN** a Retry action is available that re-attempts the download - -#### Scenario: A permission failure points to the fix -- **WHEN** a task fails because a system permission (e.g. Calendar) is denied -- **THEN** the message names the missing permission and points the user to the relevant System Settings pane - -### Requirement: Failure is never silent -A side-effecting or selection operation that does not actually accomplish its effect SHALL report a failure, -not a success. Opening a tool, running a destination adapter, writing a project note, replacing the selection, -and pasting SHALL each surface a failed state when the effect did not land, rather than reporting "Done". - -#### Scenario: A write that did not land is reported -- **WHEN** replacing the selection (or pasting the result) does not actually apply to the focused app -- **THEN** the command surfaces a failed state with a clean message, not a committed/"Done" state - -#### Scenario: A task whose side effect failed is reported -- **WHEN** a task's side effect (file write, tool open, adapter run) throws or returns a non-success status -- **THEN** the task surfaces a clean task-failed message rather than silently completing diff --git a/openspec/specs/ai-parked-sessions/spec.md b/openspec/specs/ai-parked-sessions/spec.md deleted file mode 100644 index 5d675f5..0000000 --- a/openspec/specs/ai-parked-sessions/spec.md +++ /dev/null @@ -1,289 +0,0 @@ -# ai-parked-sessions Specification - -## Purpose -Define the notch-native conversational sessions: the notch home zone (merged black-on-black with a physical notch, degrading to a top-center tab), the cursor-reveal rail of session cards with a persistent "+ New chat" card, sessions that are born at the notch (durable from their first message — an empty new chat is never saved), expand in place into a notch-anchored conversation panel (typed composer with a focused-only key flip, Approve/Skip buttons, Copy on answers), collapse back to background scheduling without cancelling an in-flight turn, per-card state badges (including a persistent needs-you badge), the one-active-now/K-ready scheduler seam, the durable store, and the expiry (auto-dismiss countdown) / deletion (authoritative discard) lifecycle. Sessions live only at the notch; the launcher's quick actions are one-shot and never appear here (see `ai-command-band`). -## Requirements -### Requirement: The notch home zone attaches to the notch and degrades gracefully -The system SHALL provide a **notch home zone** — an interactive, **non-activating** overlay panel anchored **top-center** — as the resting place for parked sessions. The panel SHALL reuse the mouse-interactive, non-activating popup species (the Dock-preview overlay pattern): it SHALL accept hover and click, SHALL NOT become the app's key/main window, SHALL NOT steal focus from the foreground app, and SHALL tear down **synchronously** (ordered out, no deferred close that could ghost on a Space switch). The zone SHALL anchor with notch awareness, and on a notched display it SHALL read as a **downward extension of the notch itself**, not a panel floating below it. On a display that **reports a physical notch**, the zone and its revealed panel SHALL **attach to the notch**: the panel SHALL be a **plain rounded rectangle** whose top edge reaches the **physical top** of the display (drawing over the menu-bar strip beneath it), **horizontally centered on the cutout** and at least as wide as it. The notch SHALL NOT be carved out of the panel — because the panel fill is **opaque black** and the notch is black, the panel's black simply spans up **behind** the notch and the two read as **one continuous shape** (no cutout, no seam). The parked content SHALL be **centered** within the panel (both axes) so it sits clear of the notch that overlaps the top-center; it SHALL NEVER be rendered behind the notch. On a **notchless built-in display or an external display** it SHALL degrade to a **top-center menu-bar tab** hanging **below** the menu bar at a fixed margin, with all other parked-session behavior identical. The home zone SHALL **never** hard-depend on a physical notch: the notch box (position, width, height) SHALL be **detected at runtime** from the display's safe-area inset and the menu-bar areas flanking the camera housing, and its absence SHALL cleanly select the tab path. - -#### Scenario: Notched display attaches the zone as an extension of the notch -- **WHEN** the active display reports a physical notch (a non-zero top safe-area inset with resolvable flanking menu-bar areas) -- **THEN** the home zone attaches flush to the notch as a plain black rounded rectangle whose black spans up behind the (also black) notch so they read as one continuous shape (no carved cutout), and the parked content is centered in the panel, clear of the notch - -#### Scenario: Notchless or external display degrades to a top-center tab -- **WHEN** the active display reports no notch (a built-in notchless display or an external monitor) -- **THEN** the home zone degrades to a top-center menu-bar tab hanging below the menu bar and all parked-session behavior is otherwise identical - -#### Scenario: The panel never steals focus and tears down synchronously -- **WHEN** the home zone is shown and later dismissed -- **THEN** it never becomes the app's key/main window, the foreground app stays the focus target, and the panel is ordered out synchronously - -### Requirement: Per-card state badges reflect each parked session's state -Each parked session SHALL carry an observable state — **thinking**, **done**, **needs-you**, or **failed** — and the rail card for that session SHALL render a badge reflecting it. The **done** badge SHALL show the count of unseen results. The **failed** badge SHALL carry only a clean, user-facing headline (from the single error translator); raw error text SHALL appear only behind an opt-in disclosure in the **expanded conversation panel**, never in the badge. A side effect that did not land SHALL surface as **failed**, never as a false "done." The state SHALL be driven by the scheduler's advance feedback and the routing loop's observable state. A session **collapsed while its turn is in flight** SHALL surface honestly on the rail: a detached **streaming** turn shows the **thinking** badge; a detached turn **paused at an approval** shows the **needs-you** badge. - -#### Scenario: A generating session shows the thinking badge -- **WHEN** a parked session is being advanced in the background, or a session was collapsed while its turn was still streaming -- **THEN** its card shows the thinking badge - -#### Scenario: A completed session shows an unseen-result count -- **WHEN** a parked session produces a new result the user has not seen -- **THEN** its card shows a done badge with the unseen-result count - -#### Scenario: A session docked at an approval shows needs-you -- **WHEN** a session is collapsed while its routing loop is paused awaiting an approval decision -- **THEN** its card shows the needs-you badge and the session is protected from expiry and eviction - -#### Scenario: A failed step shows a clean headline, never raw text -- **WHEN** a parked session's step fails -- **THEN** its card shows a failed badge carrying a clean headline only, and any raw detail is hidden behind an opt-in disclosure in the expanded conversation panel - -### Requirement: The needs-you escalation surfaces a persistent, non-intrusive needs-you signal -When a parked session escalates to **needs-you** (a dangerous write or required approval surfaced from the background-autonomy capability), the system SHALL signal it **ambiently and non-intrusively** — a peripheral cue, **not** an intrusive interruption. It SHALL NOT be a modal alert, SHALL NOT play a sound, SHALL NOT bounce, and SHALL NOT steal focus (the panel stays non-activating and never becomes key while unexpanded). The needs-you state SHALL be tracked while at least one parked session is in needs-you and SHALL **persist** until the user **addresses** every needs-you session (expanding it and resolving its pending decision, or deleting it); merely revealing the rail SHALL NOT clear it. The per-session signal SHALL live on the card as a **needs-you badge** (with the per-session count), so an escalation is never silently missable yet never blocks work. - -> Note: the earlier always-on **animated ambient glow** on the notch zone was **removed** — hosted in a retained `NSHostingView`, its perpetual `TimelineView` breather pinned the main thread while idle (see `docs/postmortem-idle-cpu-spin.md`). The needs-you state is retained (`NotchHomeZoneController.hasNeedsYou` + the per-card badge); if a dedicated ambient cue returns later it MUST be gated on real window visibility. - -#### Scenario: A background escalation surfaces the needs-you signal -- **WHEN** a parked session escalates to needs-you -- **THEN** the session's card shows a persistent needs-you badge and no modal alert, sound, bounce, or focus-steal occurs - -#### Scenario: The needs-you signal persists until the escalation is addressed -- **WHEN** the user reveals the rail but does not yet expand-and-resolve or delete the needs-you session -- **THEN** the needs-you signal persists, and it clears only once every needs-you session has been addressed - -#### Scenario: The needs-you signal is absent when nothing needs the user -- **WHEN** no parked session is in needs-you -- **THEN** no needs-you signal is shown - -### Requirement: Crossing behind the notch reveals an expanding rail of parked sessions -Crossing the cursor **up behind the notch** SHALL reveal a **rail** of parked-session cards, **always including the persistent "+ New chat" card**. The **"+ New chat" card SHALL lead** the rail; the session cards SHALL follow it ordered **most-recently-used first** (by last activity), so the last-used session sits immediately after the "+ New chat" card and older sessions trail toward the far end. The reveal trigger SHALL be the physical notch cutout itself — the rail reveals ONLY when the cursor crosses UP into the notch band (its usable "behind the notch" space, reachable because within the notch's horizontal span the cursor travels up to the physical top), **never** when it merely grazes the resting strip below the notch. On a **notchless/external** display, which has no notch to cross, the trigger SHALL be a **thin band hugging the physical top edge at top-center** (slamming the cursor to the very top edge), mimicking the same deliberate gesture. On a notched display the rail SHALL **emerge from the notch as a downward extension** — its top spanning up behind the notch (reaching the physical top), the cards spreading **downward** below the notch; on a notchless/external display the rail SHALL **hang below** the top-center tab. The rail SHALL be **non-scrollable**: the panel SHALL be sized to **hug every rendered card** (the persistent "+ New chat" card plus one per session) and SHALL **expand** as sessions are added rather than scrolling, up to a screen-fraction safety ceiling. On a notched display the panel SHALL be sized so the **centered** card row has **symmetric** vertical padding of the **notch height plus a small clearance** on top and bottom — so the cards clear the notch (never clipped at the head) and sit **balanced**, not shoved to the bottom. The reveal SHALL reuse the edge-gated cursor-reveal pattern (a passive global cursor monitor needing **no new permission**; geometry read only when the cursor is near the trigger/live region while hidden; a unified live area — the resting zone, the rail, **and the notch band** — with a grace-period dismiss, so once shown, moving the cursor **up into the notch OR back down onto the rail docks** rather than dismisses; a coarse re-feed while shown). The resting strip below the notch SHALL remain a keep-open bridge inside the live area (it just no longer triggers the reveal). The reveal SHALL be gated by a small, **user-configurable dwell**: the cursor SHALL remain crossed behind the notch (inside the trigger) **continuously** for the dwell before the rail reveals, so a quick pass THROUGH the notch (reaching for the menu bar or travelling to another corner) does not pop the dock; leaving the trigger before the dwell elapses SHALL cancel it (a re-entry restarts it), a dwell of **zero** SHALL reveal immediately, and the reveal timing SHALL not depend on continued cursor movement (a perfectly still cursor still reveals once the dwell elapses). The dwell SHALL apply to the hidden→shown transition ONLY (keep-open is instant). The grace-period dismiss SHALL apply **only while the panel is in rail mode** (an expanded conversation never grace-dismisses). The panel SHALL open and close by **growing from a point behind the notch to the full dock and shrinking back** — a fluid, spring-driven "droplet" spread anchored at its top edge, so the panel's **border itself stretches out of the point** (unfurling downward and out to both sides) and shrinks back into it. There SHALL be **no opacity fade** — the animation is geometric (the shape/border stretching), not a cross-fade. A **rail↔expanded** size change (opening a card / new chat into the conversation panel, or collapsing back) SHALL likewise **stretch the border between the two sizes** rather than snapping or fading. Teardown for the grace-dismiss MAY defer the order-out until the shrink completes, but restore and feature-off teardown SHALL remain **synchronous** (the ghost-on-Space-switch path). - -#### Scenario: Crossing behind the notch reveals the rail -- **WHEN** the cursor crosses up behind the notch (into the notch band on a notched display), or slams to the physical top edge at top-center on a notchless/external display -- **THEN** the rail of parked-session cards — always including the "+ New chat" card — is revealed, emerging downward from the notch on a notched display, or hanging below the tab on a notchless/external display - -#### Scenario: Grazing the strip below the notch does not reveal -- **WHEN** the cursor moves across the resting strip just below the notch (without crossing up into the notch band) while the rail is hidden -- **THEN** the rail is NOT revealed (only crossing behind the notch triggers it) - -#### Scenario: A dwell gates the reveal -- **WHEN** a reveal dwell is configured and the cursor crosses behind the notch but leaves the trigger before the dwell elapses -- **THEN** the rail does not reveal; it reveals only once the cursor stays crossed behind the notch continuously for the dwell (a dwell of zero reveals immediately, and a still cursor still reveals) - -#### Scenario: The panel grows from a point and shrinks back (no fade) -- **WHEN** the rail is revealed and later grace-dismissed -- **THEN** it grows from a point behind the notch to the full dock (its border stretching out of the point, no opacity fade) and shrinks back into the point on a fluid spring anchored at the top edge, and a reveal arriving mid-shrink cancels the teardown - -#### Scenario: Opening or closing a conversation stretches the border between sizes -- **WHEN** a card is opened (or a new chat created) into the expanded conversation panel, or the conversation is collapsed back to the rail -- **THEN** the panel's border fluidly stretches between the rail size and the expanded size (rather than snapping or fading), the frame animation running to completion without a per-tick reposition snapping it - -#### Scenario: The dock expands with each session rather than scrolling -- **WHEN** a new session is added to the dock -- **THEN** the panel widens to hug all cards (the "+ New chat" card plus one per session) and the rail does not scroll (within the parked-session cap) - -#### Scenario: Cards sit balanced and clear of the notch -- **WHEN** the rail is revealed on a notched display -- **THEN** the panel is sized so the centered card row has symmetric padding of the notch height plus a small clearance on top and bottom — the cards clear the notch (none clipped behind it) and sit balanced, not shoved to the bottom - -#### Scenario: Moving up into the notch docks rather than dismisses -- **WHEN** the rail is shown and the cursor moves up into the notch band above the resting zone -- **THEN** the rail stays shown (the notch band is inside the contiguous live area), and it does not grace-dismiss - -#### Scenario: Leaving the zone and rail dismisses after a grace period in rail mode -- **WHEN** the panel is in rail mode and the cursor leaves the zone, the rail, and the notch band for longer than the grace period -- **THEN** the rail dismisses, ordered out synchronously - -#### Scenario: The reveal needs no new permission -- **WHEN** the cursor-reveal monitor is installed -- **THEN** it observes cursor moves passively and requires no Input Monitoring or other new permission - -### Requirement: One active generation now, with a K-ready scheduler seam for batching -The system SHALL schedule parked sessions through a pure scheduler seam that decides which session each generation slot serves. The seam SHALL expose a slot-count-parameterized request for runnable sessions, an advance-feedback callback, and an escalation callback. In this capability exactly **one** generation SHALL be active and the rest SHALL be queued; the scheduler's runnable-set request SHALL honor the caller's slot count so that a later **batched** runtime can fill **K** slots at once as a **drop-in** with **no change** to the seam. Runnable sessions SHALL exclude those blocked on the user (needs-you) and those whose scheduled next-run time has not arrived; a parked session's next-run time SHALL be **optional**, and a parked session with **no scheduled next-run time is dormant** (blocked on the user or awaiting reactivation) and SHALL NOT be runnable. Runnable sessions SHALL be ordered deterministically (oldest-waiting first). The scheduler SHALL be pure — time SHALL be an input — so its decisions are deterministically testable. The foreground active session SHALL always be served (the scheduler fills only the remaining slots). - -#### Scenario: Exactly one parked session is advanced at a time in this capability -- **WHEN** several sessions are parked and runnable and one generation slot is requested -- **THEN** the scheduler returns exactly one session, the oldest-waiting runnable one, and the rest stay queued - -#### Scenario: The seam serves K slots without changing -- **WHEN** a batched runtime requests up to K runnable sessions through the same seam -- **THEN** the scheduler returns up to K runnable sessions with no change to the seam's shape (batching is a drop-in) - -#### Scenario: Sessions blocked on the user, scheduled later, or dormant are not runnable -- **WHEN** a session is in needs-you, its next-run time has not yet arrived, or it is parked with no scheduled next-run time -- **THEN** the scheduler does not include it in the runnable set - -#### Scenario: Scheduling decisions are deterministic -- **WHEN** the scheduler is asked for runnable sessions with a given timestamp -- **THEN** its result depends only on the parked set and that injected timestamp, so it is reproducible in tests - -### Requirement: Durable parked-session store survives relaunch with a one-line resume -The system SHALL durably persist each parked session's conversation (encodable) keyed by its session identity, plus the lightweight rail/scheduler rows, so that parked sessions and the rail rebuild after relaunch. The store SHALL be the **single** owner of parked-conversation persistence (it does not duplicate the conversation type, which the conversation-runtime capability owns). Because sessions are durable **from their first message** (see "Sessions are born at the notch from the new-chat card"), a relaunch SHALL rebuild every session that had sent at least one message and had not yet expired or been deleted — including one created moments before quit with a message sent but no completed turn. An **empty** new chat (opened but never messaged) SHALL NOT be persisted and SHALL leave nothing to rebuild. File and coding failures SHALL be mapped at the store boundary into the capability's error type and surfaced bounded and non-blocking — never a raw thrown error and never silently dropped. - -#### Scenario: Parked sessions rebuild after relaunch -- **WHEN** the app relaunches with sessions previously parked -- **THEN** the rail rebuilds from the durable store, each card showing its title and state - -#### Scenario: A just-born session survives relaunch -- **WHEN** the user creates a new chat and quits the app before any assistant turn completes -- **THEN** after relaunch the session's card is present on the rail, expandable to its stored conversation - -#### Scenario: A store failure is observable, not silent -- **WHEN** persisting or reading a parked session fails -- **THEN** the failure is mapped into the capability's error type and surfaced as a bounded, non-blocking failed indicator, never a raw error and never silence - -### Requirement: Lifecycle — max parked count, idle-timeout sleep, and discard semantics -The system SHALL bound and age notch sessions: -- A configurable **maximum parked count** SHALL be enforced by evicting the **least-recently-updated idle** session when the count is exceeded; an **active**, **needs-you**, or actively-**thinking** session SHALL NEVER be evicted. -- **A settled conversational turn is NEVER terminal:** a session whose assistant turn completes while backgrounded SHALL become **idle with an unseen-result badge** — there SHALL be no "terminal completion" classification and no state that auto-dismisses a session because its turn finished. A turn that completes while its session is **expanded** leaves the session active in place (the user is reading it). -- **Expiry is opt-in and protects unseen work:** the **auto-dismiss countdown** SHALL default to **0 = never** (no session expires by age out of the box). When the user configures a positive countdown, a session SHALL be auto-dismissed only when it is **idle** (never active, needs-you, parked-with-pending-work, or thinking), has **no unseen results** (a zero badge — unseen results protect a session from expiry indefinitely), and its idle age exceeds the countdown. Auto-dismissal SHALL route through the same authoritative discard path as a manual deletion. -- **Deletion (discard)** SHALL cancel the session's pending generation via task cancellation (a cancellation is **not** a failure and SHALL NOT leave a failed badge) and remove its durable conversation. **Completed side effects SHALL NOT be rolled back** — work the session already committed (a written event, a moved file, a launched process) stays done; discard stops only future work. Deletion SHALL be reachable from the session's rail card and from its expanded conversation view. -- **No other removal path exists:** user deletion/purge, the opt-in expiry above, and the max-parked eviction SHALL be the ONLY operations that remove a session. - -#### Scenario: Exceeding the max evicts an idle session, never an active one -- **WHEN** the parked count exceeds the maximum and at least one idle session exists -- **THEN** the least-recently-updated idle session is evicted and no active, needs-you, or thinking session is evicted - -#### Scenario: A docked session's finished answer never dismisses it -- **WHEN** a session is collapsed mid-generation and its assistant turn later completes in the background -- **THEN** the session becomes idle with an unseen-result badge, remains on the rail and in the durable store, and no auto-dismiss occurs because the turn finished - -#### Scenario: No session expires at the default countdown -- **WHEN** the auto-dismiss countdown is at its default of 0 -- **THEN** no session is ever dismissed by age, regardless of how long it idles - -#### Scenario: Unseen results protect a session from a configured countdown -- **WHEN** a positive countdown is configured and an idle session with a non-zero unseen-result badge exceeds that age -- **THEN** the session is NOT dismissed; only an idle session with a zero badge past the countdown is dismissed, through the authoritative discard path - -#### Scenario: Discard cancels pending work but does not undo completed side effects -- **WHEN** the user deletes a session that has pending generation and an already-completed side effect -- **THEN** the pending generation is cancelled (not marked failed) and the session is removed, while the completed side effect remains and is not rolled back - -### Requirement: Parked-session errors use one taxonomy and are bounded and non-blocking -All parked-session failures SHALL be classified into the shared error taxonomy — reusing the existing runtime/task errors, plus at most one capability-specific error type for store/persistence cases the shared taxonomy cannot carry — and SHALL be surfaced through the single error translator as a clean headline with opt-in copyable details. A failure SHALL be an observable failed state (a failed badge / a failed restore with Retry), **never** an app-modal alert (which would freeze the Settings window) and **never** raw error text in a headline. - -#### Scenario: A failed restore is bounded and non-blocking -- **WHEN** restoring a parked session fails -- **THEN** the failure is shown as a bounded, non-blocking failed state with a clean headline and a Retry, and no app-modal alert appears - -#### Scenario: Raw error text never reaches a headline -- **WHEN** any parked-session error is surfaced -- **THEN** the headline is the clean translated message and any raw text is available only behind an opt-in disclosure or in logs - -### Requirement: Sessions are born at the notch from the new-chat card -The revealed rail SHALL always include a persistent **"+ New chat"** card — including when no sessions exist (an empty dock still reveals a purposeful surface). Activating it SHALL create a new conversational session (a fresh conversation with a stable session identity) and expand it in place for typing, but SHALL **NOT** persist it yet — an **empty** new chat is **never** written to the store and **never** shown as a rail card. The session SHALL become **durable on its first message**: sending the first turn writes the conversation to the durable store and registers it with the scheduler (so it survives relaunch from that moment and joins the dock, ordered most-recently-used). The message SHALL be saved even if the model is momentarily unavailable (the message is never dropped). If the new chat is **closed while still empty** (collapsed/closed without a message), it SHALL be **discarded** — no store row, no rail card, nothing to rebuild on relaunch. The notch SHALL be the **only** surface that creates conversational sessions: the launcher's AI command band never creates, parks, or transfers a session here (quick actions are one-shot and ephemeral — see `ai-command-band`). A session SHALL remain at the notch for its whole life — background-advancing, collapsing, and expanding in place — until it is removed by **expiry** (the auto-dismiss countdown) or **deletion** (discard); there SHALL be no operation that moves it to another surface. - -#### Scenario: New chat is not saved until the first message -- **WHEN** the user activates the "+ New chat" card on the rail -- **THEN** the panel expands in place ready for the first typed turn, but nothing is written to the store and no rail card is created; only when the first message is sent is the session persisted, registered with the scheduler, and docked - -#### Scenario: An empty new chat closed without a message is discarded -- **WHEN** the user opens a new chat and closes it (collapses/swipes it away) without sending a message -- **THEN** the session is discarded — it leaves no store row, no rail card, and nothing to rebuild on relaunch - -#### Scenario: The new-chat card is present on an empty dock -- **WHEN** the rail is revealed while no sessions exist -- **THEN** the rail shows the "+ New chat" card, and no empty-state dead end is presented - -#### Scenario: A session lives only at the notch -- **WHEN** a session exists in the notch dock -- **THEN** every interaction with it (expand, type, approve, collapse, delete) happens on the notch surface, and no affordance moves it onto the launcher canvas or any other surface - -### Requirement: A card expands in place into a notch-anchored conversation panel -Clicking a session card (or creating a new chat) SHALL expand the notch panel **in place** into a conversation view for that session — the same merged-notch panel and chrome, mode-switched from the rail, never a second panel. The thread SHALL open scrolled to its **latest turn** — an existing session with history lands at the **end** of its conversation, not at the top — and SHALL stay pinned to the newest turn as it streams. The expanded view SHALL render the session's thread (user and assistant turns, with the assistant's reasoning behind a collapsible section and streaming shown live), the ordered tool steps the routing loop has run, and a typed **composer**: Enter sends the turn; the panel SHALL become the **key window only while the composer field is focused** so keystrokes reach it, SHALL never become the main window, SHALL never activate the app, and SHALL drop key status when the conversation collapses so the previously frontmost app keeps focus. When the session's routing loop pauses awaiting a decision, the expanded view SHALL present the review as a card with explicit **Approve** and **Skip** buttons (the notch is a cursor-and-keyboard surface; the launcher's two-finger compass grammar is not imported). Each assistant answer SHALL offer a **Copy** affordance (the notch surface writes nothing into other apps). Exactly **one** session SHALL be expanded (foreground) at a time; expanding another card SHALL first collapse the current one. **Collapse** — via an explicit collapse affordance, Escape from the composer, or clicking the notch resting zone — SHALL persist the conversation back to the store, return the panel to rail mode, and hand the session back to background scheduling **without cancelling an in-flight turn** (the turn completes in the background and updates the card's badge). **An in-flight turn INCLUDES one paused at an approval:** collapsing while the routing loop awaits an Approve/Skip decision SHALL keep the suspended step alive (the session surfaces as needs-you), and re-expanding SHALL re-present the same approval card whose Approve/Skip resumes the original paused step — the turn is never restarted or silently dropped by a dock/expand round-trip. While a conversation is expanded, cursor departure SHALL NOT dismiss the panel (the grace-dismiss applies to rail mode only); feature-off and Space-switch teardown SHALL remain synchronous. - -#### Scenario: Expanding a card shows its thread and composer in place -- **WHEN** the user clicks a parked session's card on the rail -- **THEN** the same notch panel expands in place into that session's conversation view — thread, tool steps, and composer — with no second panel and no surface change - -#### Scenario: The panel is key only while the composer is focused -- **WHEN** the user focuses the composer, types a turn, and later collapses the conversation -- **THEN** keystrokes reach the composer while it is focused (the panel is key), the panel never becomes main and never activates the app, and on collapse key status is dropped so the previously frontmost app keeps focus - -#### Scenario: Enter sends and the assistant streams in place -- **WHEN** the user types a message in the expanded composer and presses Enter -- **THEN** the turn is appended to the session's conversation and the assistant's reply streams live into the thread, with reasoning behind the collapsible section - -#### Scenario: A paused tool step is resolved with buttons -- **WHEN** the expanded session's routing loop pauses awaiting a confirm/dangerous decision -- **THEN** the review is presented as a card with Approve and Skip buttons, and choosing one resumes the loop accordingly - -#### Scenario: Collapse returns the session to background scheduling mid-turn -- **WHEN** the user collapses the conversation while an assistant turn is still streaming -- **THEN** the panel returns to rail mode, the in-flight turn is not cancelled, and its completion updates the session's card badge through the scheduler — the session is NOT dismissed when the turn completes - -#### Scenario: Docking a paused approval survives the round-trip -- **WHEN** the user collapses the conversation while the routing loop is paused awaiting an approval, then later re-expands the session and chooses Approve -- **THEN** the suspended step resumes exactly where it paused (the turn is not restarted), and the loop continues to completion - -#### Scenario: An expanded conversation does not grace-dismiss on cursor leave -- **WHEN** a conversation is expanded and the cursor moves away from the notch area -- **THEN** the panel stays open (the grace-dismiss applies only to rail mode), and it closes only via an explicit collapse, feature-off, or synchronous teardown paths - -### Requirement: The expanded conversation speaks the fast-vs-soft trackpad grammar -While a conversation is **expanded** at the notch, the system SHALL recognize **two-finger flick** gestures from the trackpad, discriminated from reading-scrolls by the canonical flick classification (a dominant-axis travel floor, a dominant-axis **peak velocity** at or above the flick velocity threshold, and a **lift** arriving within the flick lift-window of the last fast frame — the same classification, thresholds, and tunables as the launcher canvas resolve): - -- A **fast flick UP** SHALL **minimize the conversation into the notch dock**: the session performs the standard collapse (the snapshot persists, background scheduling continues, and an in-flight turn is never cancelled), and the **panel closes straight** — it SHALL shrink from its expanded size **directly into the point behind the notch** in one motion, **without** first stretching to the rail size and then dismissing (no intermediate "dwell on the dock"). The closing panel SHALL show the **conversation itself** shrinking; it SHALL NOT flash the empty "new chat" state or the rail on the way out (the conversation stays bound through the shrink; the state-collapse happens once the panel is hidden). The session remains parked (its card returns on the next reveal). This differs from the explicit **collapse affordance / Escape**, which returns the panel to the **visible rail** (so another card can be opened). -- A **fast flick RIGHT** SHALL **purge-delete** the session (see the purge requirement below). -- **Fast DOWN and fast LEFT SHALL be reserved no-ops** (no action, no dismissal), so the grammar can grow without destructive defaults. -- A **soft swipe** in any direction (sub-threshold peak velocity, or a decelerated hold-then-lift outside the lift window) SHALL emit **no gesture** — two-finger scrolling of the thread remains fully native and untouched. Gesture recognition SHALL be **watch-only** (read passively from the multitouch feed), never consuming or altering the scroll events the panel receives. - -The grammar SHALL apply **only while a conversation is expanded** (never in rail mode), and SHALL NOT capture wider gestures: a **three-or-more-finger** contact SHALL behave exactly as if no conversation were expanded (the window switcher and launcher remain fully usable while a chat is open). The launcher's own modal gesture states (the AI preview canvas, the Files drill) SHALL take precedence when active. - -#### Scenario: A fast flick up minimizes the conversation to the dock -- **WHEN** a conversation is expanded and the user performs a fast two-finger flick up (travel floor crossed, peak velocity at or above the threshold, prompt lift) -- **THEN** the conversation collapses into its notch dock card via the standard collapse — persisted, still background-scheduled, an in-flight turn not cancelled — and the panel closes straight, shrinking from the expanded size directly into the point behind the notch (no rail-size intermediate), leaving the session parked for the next reveal - -#### Scenario: A soft scroll never minimizes or deletes -- **WHEN** a conversation is expanded and the user scrolls the thread with a slow two-finger scrub, or pauses before lifting -- **THEN** no gesture fires and the thread scrolls natively, exactly as without the grammar - -#### Scenario: The switcher still works while a chat is open -- **WHEN** a conversation is expanded and the user performs a three-finger switcher swipe or a four-finger launcher swipe -- **THEN** the switcher/launcher behave exactly as if no conversation were expanded (the flick grammar watches two-finger excursions only) - -#### Scenario: Fast down and fast left are reserved -- **WHEN** a conversation is expanded and the user performs a fast two-finger flick down or left -- **THEN** nothing happens (no collapse, no delete, no dismissal) - -### Requirement: The purge-delete gesture removes the session with no trace -A **fast two-finger flick RIGHT** on the expanded conversation SHALL delete the session **completely and without trace**: the pending generation is cancelled (a cancellation is not a failure), the durable conversation and rail row are removed through the authoritative discard path, the bound engine is dropped, **and every audit-log record attributed to that session is purged** — from the in-memory ring and from the durable audit file. The purge path itself SHALL NOT write any log line or record referencing the session. Completed side effects in the outside world (a written calendar event, a moved file) are NOT rolled back — purge erases the app's own records of the session, not the world. The purge SHALL be **user-initiated only** (this gesture); the plain delete affordances (the card's context menu, the expanded header's delete) SHALL keep the standard discard, which leaves the audit ledger intact. - -#### Scenario: A fast flick right purges the session everywhere -- **WHEN** a conversation is expanded and the user performs a fast two-finger flick right -- **THEN** the panel dismisses, the pending generation is cancelled, the durable conversation and rail row are removed, and the session's audit records are gone from both the in-memory ring and the durable audit file — with no new log line about the session - -#### Scenario: A purged session leaves nothing after relaunch -- **WHEN** a session is purge-deleted and the app relaunches -- **THEN** no card, no conversation, and no audit record attributed to that session can be found anywhere - -#### Scenario: A plain delete keeps the ledger -- **WHEN** the user deletes a session via the card's context menu or the expanded header's delete affordance -- **THEN** the session's conversation and row are removed but its audit-log records remain in the ledger - - -### Requirement: A background driver advances runnable sessions -The system SHALL actually drive the scheduler seam: on a coarse repeating tick (and once after launch normalization), the driver SHALL request the runnable set (one slot in this capability), rebuild a detached engine from the durable conversation, mark the served session's next-run time cleared (so a second tick cannot double-serve it), and advance the session by running its **pending turn** — the stored conversation whose last message awaits an assistant reply — through the same turn machinery, settle path, and badge classification as a collapsed foreground turn. The driver SHALL skip its pass entirely while any turn is in flight (the foreground or a detached turn owns the single slot). A failed advance SHALL report through the scheduler's advance-feedback callback (re-parking with a scheduled retry), never silently; a completed advance SHALL leave the session idle with an unseen-result badge. - -#### Scenario: A recovered session advances in the background -- **WHEN** a parked session is runnable (its scheduled next-run time has arrived) and no turn is in flight -- **THEN** the driver rebuilds its engine from the durable conversation, runs the pending turn detached, and the settled answer leaves the session idle with an unseen-result badge - -#### Scenario: The driver never double-serves or preempts -- **WHEN** the driver ticks while a foreground or detached turn is already in flight, or ticks twice while an advance is running -- **THEN** it serves no session (the in-flight turn owns the slot) and the advancing session is not served a second time - -#### Scenario: A failed advance re-parks with a retry, never silence -- **WHEN** a background advance fails -- **THEN** the failure reports through the advance-feedback callback, the session re-parks with a scheduled retry, and the rail reflects the state — the session is not dismissed and the failure is not silent - -### Requirement: Relaunch normalizes rows and recovers interrupted turns -At startup, the system SHALL normalize the durable rows so no session is stranded: a row persisted as **active** (the app quit while the session was expanded or its turn was detached in flight) SHALL become **parked and scheduled to run now** when its conversation's last message awaits an assistant reply (so the background driver re-runs the interrupted turn), and **idle** otherwise; a **needs-you** row SHALL stay needs-you (blocked on the user, not runnable); a row persisted under the retired terminal state by an older build SHALL decode as **idle** (never dropped, never dismissed). Re-running a recovered turn MAY re-execute auto-tier tool steps (audited, restore-era convention); confirm/dangerous steps still gate. - -#### Scenario: A quit mid-response resumes after relaunch -- **WHEN** the app quits while a session's assistant turn is streaming and the app later relaunches -- **THEN** the session's row is normalized to parked-and-scheduled, the background driver re-runs the pending turn, and the answer lands as an unseen result — the session is never left permanently stuck or deleted - -#### Scenario: Stale rows never strand or vanish -- **WHEN** the store loads rows persisted as active with no pending turn, as needs-you, or under the retired terminal state -- **THEN** they normalize to idle, stay needs-you, and decode as idle respectively — every previously stored session is still present after relaunch diff --git a/openspec/specs/ai-subagents/spec.md b/openspec/specs/ai-subagents/spec.md deleted file mode 100644 index 1915273..0000000 --- a/openspec/specs/ai-subagents/spec.md +++ /dev/null @@ -1,38 +0,0 @@ -# ai-subagents Specification - -## Purpose -Define subagents as routable tools: fixed, bounded, fresh-context sub-tasks the agent loop can delegate to — context hygiene on a single-GPU machine, not parallelism. A subagent runs a registered template in a brand-new conversation (its own system prompt, none of the orchestrator's history) within the orchestrator's in-flight turn, and only its summary re-enters the orchestrator's thread. There is no dynamic or recursive spawning. - -## Requirements -### Requirement: Subagents are fixed, bounded templates — never dynamic spawning -The system SHALL model a subagent as a **named, fixed template** (a system prompt plus a bounded turn budget) drawn from a **small registered set**. There SHALL be NO open-ended, model-invented, or recursive subagent spawning: the model can only invoke the registered templates, a subagent SHALL NOT itself invoke subagents, and the template set SHALL be injectable so it can grow deliberately (e.g. skills-derived later) without changing the seam. - -#### Scenario: Only registered templates are invocable -- **WHEN** the routing loop offers tools for a turn -- **THEN** exactly the registered subagent templates appear (as `subagent:` tools), and no path exists for the model to define or spawn an unregistered subagent - -#### Scenario: No recursion -- **WHEN** a subagent runs its sub-task -- **THEN** its own execution offers no subagent tools (a subagent cannot spawn subagents) - -### Requirement: A subagent runs as a routable tool step in a fresh conversation -Each registered subagent SHALL be exposed to the agent loop as a **routable tool** with an `auto` write policy (CONTAINED — it is read-only with respect to the orchestrator's world; it performs no external side effects in this capability). Invoking it SHALL open a **fresh conversation** — a brand-new session identity seeded with the template's own system prompt and the routed input, empty of the orchestrator's history — and run it on the session's runtime **within the orchestrator's in-flight turn** (same task, same cancellation: cancelling the turn cancels the subagent; no new concurrency surface). The subagent step SHALL be audited like any other tool step. - -#### Scenario: A subagent step runs in isolation -- **WHEN** the loop routes a `subagent:` step with an input -- **THEN** the sub-task runs in a fresh conversation seeded only with the template's system prompt and that input — none of the orchestrator's history is visible to it - -#### Scenario: Cancelling the turn cancels the subagent -- **WHEN** the user discards the turn while a subagent step is running -- **THEN** the subagent's generation is cancelled with the turn (a cancellation, not a failure) - -### Requirement: Only the summary re-enters the orchestrator -A subagent SHALL return **only a summary** to the orchestrator: the step's result carries the sub-task's final text, which re-enters the orchestrator's thread as the tool step's message — the orchestrator's context SHALL NEVER absorb the subagent's intermediate turns, reasoning, or system prompt (the context-hygiene contract that justifies subagents on a single-GPU machine). A subagent failure SHALL surface as a **failed tool step** carrying a clean headline from the shared error taxonomy — never raw error text, never a fabricated summary. - -#### Scenario: The orchestrator sees one summary message -- **WHEN** a subagent step completes -- **THEN** exactly one tool message containing the subagent's summary is appended to the orchestrator's context, and none of the sub-conversation's other content appears there - -#### Scenario: A subagent failure is a clean failed step -- **WHEN** the subagent's generation fails -- **THEN** the step resolves as failed with a clean translated headline (no raw error text), and the loop handles it like any failed step — never presenting a fabricated summary diff --git a/openspec/specs/computer-use-tools/spec.md b/openspec/specs/computer-use-tools/spec.md deleted file mode 100644 index 9210e56..0000000 --- a/openspec/specs/computer-use-tools/spec.md +++ /dev/null @@ -1,88 +0,0 @@ -# computer-use-tools Specification - -## Purpose -TBD - created by archiving change add-voice-computer-use-agent. Update Purpose after archive. -## Requirements -### Requirement: AX-first reading with bounded, honest snapshots -The system SHALL read windows through the accessibility tree as the primary sense: `read_window` SHALL produce a semantic snapshot — extracted text plus an enumerated element list (role, label, value preview, actionability) — bounded in depth and count, with truncation reported honestly (a `truncated` flag, never a silently-partial tree presented as complete). Reading SHALL run off the main thread with a timeout so a beach-balling target app never freezes a turn. A window that exposes no usable tree SHALL produce an honest "cannot read this window" outcome (vision capture remains the explicit fallback), never a fabricated result. - -#### Scenario: A terminal window is read semantically -- **WHEN** `read_window` targets a standard text-bearing window -- **THEN** the step result carries its extracted text and an element list, with no screenshot or vision inference involved - -#### Scenario: An AX desert is honest -- **WHEN** `read_window` targets a window exposing no usable accessibility content -- **THEN** the step reports it cannot read the window (clean headline) rather than returning fabricated or coordinate-based content - -### Requirement: Acts are constrained to enumerated element IDs — never coordinates, never fabricated -Every acting tool (`click_element`, `type_text`) SHALL accept ONLY a stable element ID that resolves against the most recent snapshot of that window (a per-window epoch). Stale or unknown IDs SHALL fail cleanly (`staleElement`) instructing a re-read — the model can only act on elements that actually exist. The tool schemas SHALL contain NO coordinate parameters, and the system SHALL post no synthetic mouse movement or coordinate clicks. This extends the tool-routing "degrade, never fabricate" rule one level down: an unresolvable target is a clean failure, never a guess. - -#### Scenario: A stale ID never mis-clicks -- **WHEN** an act names an element ID from an outdated snapshot after the window changed -- **THEN** the act fails with a stale-element outcome telling the loop to re-read, and nothing is clicked - -#### Scenario: No coordinate surface exists -- **WHEN** the acting tools' schemas are inspected -- **THEN** no coordinate/point parameter exists on any of them - -### Requirement: Every act verifies, and an unverified act is a failure -After `click_element` / `type_text`, the primitive SHALL re-read the affected element or subtree and report whether the expected change is observable. A side effect that cannot be verified SHALL become a `.failed` step with a clean headline — never a false "Done". Verification SHALL be part of the primitive (not a separate optional tool), so no code path can act without it. - -#### Scenario: A click that didn't land is reported -- **WHEN** `click_element` presses an element but the re-read shows no observable change where one was expected -- **THEN** the step settles `.failed` with a clean explanatory headline, and the loop/user sees the truth - -### Requirement: Window focus goes through the switcher's own commit path -`focus_window` SHALL resolve targets from the switcher's window enumeration and SHALL raise through the SAME hardened commit path the trackpad and ⌘-Tab use (`raiseCommitted`: SkyLight handshake, minimized restore, Stage Manager guards) — the agent is a third caller of existing machinery, not a parallel raise implementation. - -#### Scenario: The agent focuses a background window reliably -- **WHEN** `focus_window` targets a background app's window by app + title hint -- **THEN** the window is raised via the existing commit path and the step reports the focused window's identity - -### Requirement: Acts respect the write-policy gate, with a per-conversation auto-approve mode -Reading and focusing SHALL be `.auto`-tier; acting (`click_element`, `type_text`, and turning auto mode ON) SHALL be `.confirm`-tier through the EXISTING approval gate. A per-conversation `autoApprove` grant — set from the initial command's parsed intent, a routable `set_auto_mode` tool, or a visible surface toggle — SHALL lift `.confirm` steps to immediate execution for THAT conversation only. Enabling auto mode SHALL itself always require confirmation (the one approval that cannot be skipped); disabling SHALL be instant. Every auto-executed act SHALL be narrated (spoken when voice is active, always visible in the step list) — silence never hides an act. - -#### Scenario: Auto mode executes acts hands-free but narrated -- **WHEN** a conversation has auto-approve granted and the loop routes a `type_text` -- **THEN** the act executes without a pause and its summary is narrated/visible as it happens - -#### Scenario: Granting auto mode is itself gated -- **WHEN** the model routes `set_auto_mode(on)` without a prior user grant this conversation -- **THEN** the step pauses at the approval gate; only the user's approval enables it - -### Requirement: The agent loop is time-bounded per step and per turn -Each tool dispatch SHALL race a per-step wall-clock timeout (configurable, default 30 s): a timed-out step SHALL cancel its work and settle `.failed` with a clean "timed out" headline (never a misleading network/server error). Each turn SHALL carry a total deadline (configurable, default 180 s) checked between steps; exceeding it SHALL terminate the loop through the existing cap-reached fallback with an honest partial summary. Cancellation (user abort, barge-in) SHALL remain a discard, not a failure. - -#### Scenario: A hung tool cannot freeze a turn -- **WHEN** a tool step exceeds the step timeout (e.g. an unresponsive AX target) -- **THEN** the step is cancelled and settles `.failed("… timed out")`, and the loop proceeds to its fallback behavior - -#### Scenario: A runaway turn ends honestly -- **WHEN** a turn's accumulated wall-clock exceeds the turn deadline -- **THEN** the loop terminates with the cap-reached fallback and an honest summary of what was and wasn't done - -### Requirement: Any human trackpad touch aborts agent action instantly -While the agent is acting (`isActing`), the system SHALL treat ANY human trackpad contact as an immediate abort: the in-flight tool task and turn are cancelled (a discard), synthetic input stops, and the abort is acknowledged (visible, and spoken when voice is active). Agent-posted synthetic keyboard events SHALL be tagged at the event source and ignored by the app's own gesture/event taps, so the agent's typing can never be misread as human gestures or trigger recognizers. A visible indicator SHALL show while the agent has the wheel. - -#### Scenario: The human always wins the input -- **WHEN** the user touches the trackpad while the agent is mid-act -- **THEN** the act and turn cancel immediately as a discard, and the indicator clears - -#### Scenario: Agent typing never triggers the app's own recognizers -- **WHEN** `type_text` posts synthetic keystrokes -- **THEN** the app's event taps identify and ignore the tagged events (no gesture, no ⌘-Tab interception, no recognizer state change) - -### Requirement: Computer-use failures join the single error taxonomy -AX failures SHALL be classified into an `AXActionError` taxonomy (`LocalizedError`, parallel to `FileActionError`): not permitted, app not responding, window gone, stale element, element not actionable, verify failed — mapped at the `AXUIElement` boundary, routed through `AIError.message(for:)`, surfaced bounded + non-blocking. Raw AX/OS error text SHALL appear only as opt-in details. - -#### Scenario: A revoked Accessibility grant is a clean failure -- **WHEN** an act runs while the Accessibility permission is missing/revoked -- **THEN** the step settles `.failed` with a clean "not permitted" headline and a pointer to permissions — never raw AX error text, never a modal - -### Requirement: The tool surface is flag-gated and additive -The computer-use tools SHALL exist only while the `computerUseEnabled` opt-in (default OFF, under the AI master gate) is on — the registry re-queries contributors live, so toggling takes effect immediately without restart. The tools SHALL be additive: no existing tool, command, or gesture behavior changes when the flag is off. - -#### Scenario: Off means absent -- **WHEN** `computerUseEnabled` is off and the route candidates are gathered -- **THEN** no computer-use tool appears among them - diff --git a/openspec/specs/configuration-hub/spec.md b/openspec/specs/configuration-hub/spec.md index 82d7670..4ef2a01 100644 --- a/openspec/specs/configuration-hub/spec.md +++ b/openspec/specs/configuration-hub/spec.md @@ -2,16 +2,16 @@ ## Purpose -Define the single, unified configuration **Hub** window that is the only surface for configuring every feature: an Overview landing page of master toggles, grouped-sidebar navigation (Content → Bands; Features → Switcher/Launcher/Clipboard/AI; System → Setup/General), per-feature pages that preserve all tunables and persistence, a Setup page for permissions and native-gesture opt-ins, a Bands page that edits only authored bands, and a Liquid-Glass/material presentation consistent with the runtime overlays. All former configuration surfaces (Settings, Favorites editor, Setup/Onboarding, AI-command editor) fold into this one window. +Define the single, unified configuration **Hub** window that is the only surface for configuring every feature: an Overview landing page of master toggles, grouped-sidebar navigation (Content → Bands; Features → Switcher/Launcher/Clipboard; System → Setup/General), per-feature pages that preserve all tunables and persistence, a Setup page for permissions and native-gesture opt-ins, a Bands page that edits only authored bands, and a Liquid-Glass/material presentation consistent with the runtime overlays. All former configuration surfaces (Settings, Favorites editor, Setup/Onboarding) fold into this one window. ## Requirements ### Requirement: Unified configuration Hub window -The system SHALL provide a single configuration **Hub** window that is the only surface for configuring every feature. Opening any configuration entry point (from the status menu or from in-app deep links) SHALL open this one window — there SHALL be no separate Settings, Favorites, Setup, or AI-command-editor window or sheet. The Hub SHALL be a single reusable window: re-opening it SHALL bring the existing window forward rather than creating another, and its frame SHALL persist across launches. +The system SHALL provide a single configuration **Hub** window that is the only surface for configuring every feature. Opening any configuration entry point (from the status menu or from in-app deep links) SHALL open this one window — there SHALL be no separate Settings, Favorites, or Setup window or sheet. The Hub SHALL be a single reusable window: re-opening it SHALL bring the existing window forward rather than creating another, and its frame SHALL persist across launches. The one exception is the **First Touch wizard**: a transient first-run/replay window that is an onboarding performance, not a configuration surface. Every preference the wizard writes SHALL be the same persisted preference the Hub owns, and the wizard SHALL NOT host any configuration capability beyond its onboarding steps — the Hub remains the only place to configure the app. #### Scenario: One window for all configuration - **WHEN** the user opens configuration from the status menu -- **THEN** the Hub window opens, and no separate Settings, Favorites, Setup, or AI-command window exists +- **THEN** the Hub window opens, and no separate Settings, Favorites, or Setup window exists #### Scenario: Re-opening reuses the same window - **WHEN** the Hub is already open and the user triggers it again @@ -26,11 +26,11 @@ The one exception is the **First Touch wizard**: a transient first-run/replay wi - **THEN** it writes the identical persisted preference as the corresponding Hub page, and any further configuration of that feature happens in the Hub ### Requirement: Overview landing page with feature master toggles -The Hub SHALL present an **Overview** landing page that shows every feature's master enable toggle at a glance — the window switcher, Space-row switching, the four-finger launcher, clipboard history, and AI commands — and SHALL let the user turn each feature on or off directly from this page. Each feature row SHALL deep-link to that feature's detail page. Toggling a feature on the Overview SHALL write the same persisted preference as toggling it on its detail page. +The Hub SHALL present an **Overview** landing page that shows every feature's master enable toggle at a glance — the window switcher, Space-row switching, the four-finger launcher, and clipboard history — and SHALL let the user turn each feature on or off directly from this page. Each feature row SHALL deep-link to that feature's detail page. Toggling a feature on the Overview SHALL write the same persisted preference as toggling it on its detail page. #### Scenario: All master toggles visible at a glance - **WHEN** the user opens the Hub on the Overview page -- **THEN** the on/off state of the switcher, Space-row switching, launcher, clipboard, and AI features is shown together +- **THEN** the on/off state of the switcher, Space-row switching, launcher, and clipboard features is shown together #### Scenario: Toggle from the Overview - **WHEN** the user flips a feature's toggle on the Overview page @@ -41,11 +41,11 @@ The Hub SHALL present an **Overview** landing page that shows every feature's ma - **THEN** the Hub navigates to that feature's detail page ### Requirement: Grouped sidebar navigation -The Hub SHALL organize its pages in a sidebar grouped as: an **Overview** entry; a **Content** group containing **Bands**; a **Features** group containing **Switcher**, **Launcher**, **Clipboard**, and **AI**; and a **System** group containing **Setup** and **General**. The sidebar SHALL be a compact icon-only rail (each destination shown as its icon, with its name as a tooltip, the groups separated by dividers) to conserve horizontal space, and SHALL tint the selected destination. The sidebar SHALL provide a button to expand it to show icon + label and collapse it back to icons, transitioning smoothly (animated). Each destination's full row (not merely its glyph) SHALL be the click target. Selecting a sidebar entry SHALL show that page in the detail area. A disabled feature SHALL keep its page reachable (its controls shown disabled) so it can be re-enabled. Space-row switching is a sub-feature of the Switcher and SHALL appear as sections on the **Switcher** page (Space-row switching and Fixed order) rather than as its own sidebar destination. +The Hub SHALL organize its pages in a sidebar grouped as: an **Overview** entry; a **Content** group containing **Bands**; a **Features** group containing **Switcher**, **Launcher**, and **Clipboard**; and a **System** group containing **Setup** and **General**. The sidebar SHALL be a compact icon-only rail (each destination shown as its icon, with its name as a tooltip, the groups separated by dividers) to conserve horizontal space, and SHALL tint the selected destination. The sidebar SHALL provide a button to expand it to show icon + label and collapse it back to icons, transitioning smoothly (animated). Each destination's full row (not merely its glyph) SHALL be the click target. Selecting a sidebar entry SHALL show that page in the detail area. A disabled feature SHALL keep its page reachable (its controls shown disabled) so it can be re-enabled. Space-row switching is a sub-feature of the Switcher and SHALL appear as sections on the **Switcher** page (Space-row switching and Fixed order) rather than as its own sidebar destination. #### Scenario: Sidebar groups - **WHEN** the user opens the Hub -- **THEN** the sidebar shows Overview, a Content group with Bands, a Features group with Switcher/Launcher/Clipboard/AI, and a System group with Setup/General, as a compact icon-only rail with names as tooltips +- **THEN** the sidebar shows Overview, a Content group with Bands, a Features group with Switcher/Launcher/Clipboard, and a System group with Setup/General, as a compact icon-only rail with names as tooltips - **AND** Space-row switching appears as sections on the Switcher page, not as its own sidebar entry #### Scenario: Disabled feature page still reachable @@ -61,7 +61,7 @@ Every tunable and control that existed in the former Settings window SHALL be pr #### Scenario: Reset semantics are unchanged - **WHEN** the user resets to defaults from the Hub -- **THEN** the same tunables reset and the same opt-in preferences (gesture relocations, clipboard and AI opt-ins, excluded apps, selected model) are preserved exactly as before +- **THEN** the same tunables reset and the same opt-in preferences (gesture relocations, the clipboard opt-in, excluded apps) are preserved exactly as before ### Requirement: Setup page hosts permissions and native-gesture opt-ins The Hub SHALL provide a **Setup** page that hosts the permissions status and guidance and the native-gesture opt-ins for ongoing (post-onboarding) use. The configuration entry point for permissions and setup SHALL be this page. The Setup page SHALL also offer the First Touch wizard entry: **Resume the welcome tour** while first-run onboarding is incomplete, and **Replay the welcome tour** after completion. @@ -75,11 +75,11 @@ The Hub SHALL provide a **Setup** page that hosts the permissions status and gui - **THEN** a resume entry is offered if onboarding is incomplete, or a replay entry if it is complete ### Requirement: Bands page edits only authored bands -The Hub SHALL provide a **Bands** page that is the single surface for arranging launcher content. The Bands page SHALL edit only **authored** bands (the user's favorites bands, which now include AI-command items). It SHALL NOT present **live** bands — bands whose items are auto-populated rather than authored, such as the clipboard band — for item-level editing; such features are configured on their own feature page and projected into the launcher at runtime. +The Hub SHALL provide a **Bands** page that is the single surface for arranging launcher content. The Bands page SHALL edit only **authored** bands (the user's favorites bands). It SHALL NOT present **live** bands — bands whose items are auto-populated rather than authored, such as the clipboard band — for item-level editing; such features are configured on their own feature page and projected into the launcher at runtime. #### Scenario: Authored bands are editable on the Bands page - **WHEN** the user opens the Bands page -- **THEN** the user's favorites bands and their items (including AI commands) are shown and editable +- **THEN** the user's favorites bands and their items are shown and editable #### Scenario: The clipboard live band is not edited on the Bands page - **WHEN** the user opens the Bands page with clipboard history enabled @@ -110,7 +110,7 @@ The Hub's **General** page SHALL present its preference settings — **Self-heal ### Requirement: General page Danger zone The Hub's **General** page SHALL provide a "Danger zone" section with selective, explicit reset controls: -- Four opt-in selectors, all default off, each gating one deletion category: **App data & settings** (the app's preferences domain, Application Support data excluding the AI model weights, and saved window state), **Caches**, **AI models** (the on-disk weights, with the AI opt-in turned off first), and **Permissions** (a TCC reset for every service the app can hold). The selectors SHALL be presented as a 2×2 grid of full-body toggle-cards: the whole card is the click target, and a selected card is visually highlighted (distinct from a plain on/off switch). +- Opt-in selectors, all default off, each gating one deletion category: **App data & settings** (the app's preferences domain, Application Support data, and saved window state), **Caches**, and **Permissions** (a TCC reset for every service the app can hold). The selectors SHALL be presented as a grid of full-body toggle-cards: the whole card is the click target, and a selected card is visually highlighted (distinct from a plain on/off switch). - A destructive **Clear selected** action that SHALL be disabled while no category is selected and SHALL require an explicit confirmation enumerating exactly what will happen before anything is deleted. - WHEN App data & settings is selected and any native-gesture/Spaces backup exists, the relocations SHALL be restored FIRST (and the confirmation SHALL say so) — the wipe must never delete the backups while leaving the system relocated. - WHEN App data & settings or Permissions was cleared, the app SHALL relaunch itself so the fresh process reads the cleared state (a data wipe re-enters first-run onboarding); cache/model-only clears SHALL report a non-blocking summary and stay running. @@ -121,8 +121,8 @@ The Hub's **General** page SHALL provide a "Danger zone" section with selective, - **THEN** the Clear action is disabled and nothing is deleted #### Scenario: Selective clear honors the selection -- **WHEN** the user selects only Caches and AI models and confirms -- **THEN** only the cache directories and the model weights are removed (the AI opt-in turning off first), preferences and permissions are untouched, and the app keeps running with a summary +- **WHEN** the user selects only Caches and confirms +- **THEN** only the cache directories are removed, preferences and permissions are untouched, and the app keeps running with a summary #### Scenario: Data wipe restores gestures first - **WHEN** App data & settings is selected while a trackpad relocation backup exists and the user confirms @@ -140,81 +140,3 @@ The Hub's **General** page SHALL provide a "Danger zone" section with selective, - **WHEN** the Danger zone is shown - **THEN** the four category selectors appear as a 2×2 grid of full-body toggle-cards where clicking anywhere on a card toggles its selection and the selected card is highlighted -### Requirement: Files page hosts roots, appearance, and behavior - -The Hub SHALL provide a **Files** page (reachable from the grouped sidebar) that hosts the Files band's configuration: the **roots editor** (add / remove / reorder the local root folders that form the band's entry column), the **appearance** controls (column width / density, tint, and an icon-vs-preview choice), and the **behavior** controls (entry **sort order**, the default-open action, and which metadata a row shows). The page SHALL include the Files band **opt-in** master toggle. All controls SHALL **persist** their values (live-applied, like the other feature pages) and SHALL use the shared **Liquid Glass** presentation consistent with the rest of the Hub. The roots editor SHALL accept **local folders only**. - -#### Scenario: The Files page is reachable and toggles the band -- **WHEN** the user opens the Hub and selects the Files page -- **THEN** the page shows the Files opt-in toggle plus the roots, appearance, and behavior controls - -#### Scenario: Editing roots updates the entry column -- **WHEN** the user adds, removes, or reorders a root on the Files page -- **THEN** the Files band's entry column reflects the change on the next launcher open - -#### Scenario: Appearance and behavior changes persist and live-apply -- **WHEN** the user changes column density, tint, sort order, or the default-open action -- **THEN** the change persists across launches and applies to the Files band - -#### Scenario: Only local folders can be added as roots -- **WHEN** the user attempts to add a root -- **THEN** only local folders are accepted (network / iCloud-placeholder locations are rejected) - -### Requirement: The Files page configures the action menu and lift action - -The Hub's **Files page** SHALL let the user configure the Files-band action menu and the drill's lift action. It SHALL provide: - -- a **per-type menu editor** (separate for **file** and **folder**) to **add, remove, and reorder** items from the action catalog (the defaults plus the opt-in extras — Reveal in Finder, Add to Favorites, Open in ‹editor›, Copy name); -- a **terminals/editors curation** control listing the **auto-detected** installed tools, each individually enable-able; -- a control to set the Files **lift action** (deliver / open / open-menu), edited through the existing gesture-binding editor used by the other remappable surfaces. - -Edits SHALL persist (see *tunable-settings*) and SHALL take effect on the next time the band is shown. The page SHALL keep the existing previewed-section header and its other Files controls unchanged. - -#### Scenario: Reordering the file menu persists - -- **WHEN** the user reorders the **file** action menu on the Files page -- **THEN** the next time the file action menu opens it reflects the new order, across launches - -#### Scenario: Curating terminals controls the menu - -- **WHEN** the user disables a detected terminal on the Files page -- **THEN** that terminal no longer appears in the folder action menu - -#### Scenario: Setting the lift action - -- **WHEN** the user sets the Files lift action to "open" -- **THEN** a lift on a highlighted entry opens it (instead of delivering), and the default can be restored - -#### Scenario: File and folder editors are independent - -- **WHEN** the user edits the folder menu -- **THEN** the file menu editor and its result are unaffected - -### Requirement: The AI page hosts the Background autonomy whitelist and audit log -The Hub's **AI** feature page SHALL provide a **Background autonomy** section with two surfaces: - -- a **whitelist editor** — add, remove, and review the trusted **folder path prefixes** (picked as **local folders only**) and the trusted **command patterns** that let a parked agent run a `confirm` write automatically. The editor SHALL default empty (a fresh install trusts nothing arbitrary), SHALL **persist** its values with the same keys/defaults/reset semantics as the other AI opt-ins, and SHALL state plainly that whitelisting a folder/command makes matching writes run in the background, and that dangerous operations (delete, overwrite-existing, arbitrary shell) are never made automatic by the whitelist. -- an **audit log viewer** — a reverse-chronological "what your agents did while you were away" ledger of recent background and foreground tool steps, each showing the tool, a redacted arguments summary, the effective tier, the outcome (a failure shown as a clean headline with an opt-in details disclosure), and a timestamp, distinguishing background actions from foreground ones. - -Both surfaces SHALL use the shared Liquid Glass presentation consistent with the rest of the Hub. A failure to persist or load the audit log SHALL surface as a **bounded, non-blocking** banner (clean headline, opt-in details), **never** an app-modal alert. - -#### Scenario: The Background autonomy section is reachable on the AI page -- **WHEN** the user opens the Hub and selects the AI feature page -- **THEN** a Background autonomy section shows the whitelist editor (trusted folders + command patterns) and the audit log viewer - -#### Scenario: Editing the whitelist persists and live-applies -- **WHEN** the user adds a trusted folder or command pattern and removes another -- **THEN** the change persists across launches, is preserved by a reset-to-defaults like the other AI opt-ins, and the agent's effective-tier resolution reflects it on the next step - -#### Scenario: Only local folders can be added as trusted prefixes -- **WHEN** the user adds a trusted folder -- **THEN** only a local folder is accepted (network / iCloud-placeholder locations are rejected) - -#### Scenario: The audit viewer reads the ledger -- **WHEN** the user opens the audit log viewer after the agent has worked -- **THEN** the recent tool steps appear in reverse-chronological order with their tool, redacted args, effective tier, outcome, and timestamp, marking which ran in the background - -#### Scenario: An audit store failure is non-blocking -- **WHEN** loading or persisting the audit log fails -- **THEN** the viewer shows a bounded banner with a clean headline (details behind an opt-in disclosure) and the rest of the page stays usable, with no app-modal alert - diff --git a/openspec/specs/favorites-editor/spec.md b/openspec/specs/favorites-editor/spec.md index 6cf7817..6809cab 100644 --- a/openspec/specs/favorites-editor/spec.md +++ b/openspec/specs/favorites-editor/spec.md @@ -2,7 +2,7 @@ ## Purpose -Define the band/item editing canvas — the Hub's **Bands** page — that browses and sources launch items by type (including AI commands), arranges them into context bands on a canvas, and configures per-item and per-band appearance and strategy, with edits persisting immediately to the launcher. +Define the band/item editing canvas — the Hub's **Bands** page — that browses and sources launch items by type, arranges them into context bands on a canvas, and configures per-item and per-band appearance and strategy, with edits persisting immediately to the launcher. ## Requirements @@ -18,7 +18,7 @@ The system SHALL provide a band/item editing canvas as the Hub's **Bands** page - **THEN** the launcher reflects the change ### Requirement: Source items by type via a browsing sidebar -The Bands page SHALL provide a source picker that lists item-source categories by type (applications, shortcuts, paths, URLs, presets, scripts, **AI command**, **Claude Project**, **Open in Terminal**). The source picker SHALL appear inline under the expanded band in the bands column — the expanded band is the add target — and everything chosen SHALL be added directly into that band. Browse-style categories (applications, shortcuts, actions, AI command, presets) SHALL open a scrollable browseable list of candidates; selecting a candidate SHALL add it to the currently targeted context band. Immediate-add categories — **URLs**, **Scripts**, **Files & Folders**, **Claude Project**, and **Open in Terminal** — SHALL add an item directly rather than presenting a fill-in-first form: choosing URLs or Scripts SHALL add a blank item of that kind; choosing Files & Folders SHALL prompt for a path and then add the item; choosing Claude Project or Open in Terminal SHALL prompt for a folder and then add the item. In all cases the value-bearing fields (a link's URL/open-with/window, a script's body, a file's path, a Claude Project's folder and command, an Open-in-Terminal item's folder and command) SHALL be edited in the item panel after adding — not in the source picker. The newly added item SHALL be selected, and the editor SHALL place the keyboard focus on its first relevant field (the URL field for a link, the body for a script, the name for a file) for fast entry. The picker SHALL provide a way to return from a browse list to the category index. +The Bands page SHALL provide a source picker that lists item-source categories by type (applications, shortcuts, paths, URLs, presets, scripts, **Claude Project**, **Open in Terminal**). The source picker SHALL appear inline under the expanded band in the bands column — the expanded band is the add target — and everything chosen SHALL be added directly into that band. Browse-style categories (applications, shortcuts, actions, automations, presets) SHALL open a scrollable browseable list of candidates; selecting a candidate SHALL add it to the currently targeted context band. Immediate-add categories — **URLs**, **Scripts**, **Files & Folders**, **Claude Project**, and **Open in Terminal** — SHALL add an item directly rather than presenting a fill-in-first form: choosing URLs or Scripts SHALL add a blank item of that kind; choosing Files & Folders SHALL prompt for a path and then add the item; choosing Claude Project or Open in Terminal SHALL prompt for a folder and then add the item. In all cases the value-bearing fields (a link's URL/open-with/window, a script's body, a file's path, a Claude Project's folder and command, an Open-in-Terminal item's folder and command) SHALL be edited in the item panel after adding — not in the source picker. The newly added item SHALL be selected, and the editor SHALL place the keyboard focus on its first relevant field (the URL field for a link, the body for a script, the name for a file) for fast entry. The picker SHALL provide a way to return from a browse list to the category index. #### Scenario: Browse applications - **WHEN** the user opens the Applications category in the source picker @@ -37,10 +37,6 @@ The Bands page SHALL provide a source picker that lists item-source categories b - **WHEN** the user chooses the Scripts source, or the Files & Folders source and picks a path - **THEN** the item is added and selected, and its body (script) or name (file) is focused in the item panel for editing -#### Scenario: Add an AI command from the source picker -- **WHEN** the user chooses the AI command source -- **THEN** a new AI-command item is added to the currently targeted band and selected for inline editing - #### Scenario: Add a Claude Project from the source picker - **WHEN** the user chooses the Claude Project source and picks a folder - **THEN** a Claude Project item bound to that folder is added to the currently targeted band, titled with the folder name, and selected — with its folder editable in the item panel @@ -53,25 +49,6 @@ The Bands page SHALL provide a source picker that lists item-source categories b - **WHEN** the user is in a category's browse list and chooses back - **THEN** the picker returns to the category index -### Requirement: Author and edit AI commands inline as band items -The Bands page SHALL let the user author and edit an AI command **inline**, as a band item, exposing every field of the command model — name, icon, tint, input source, prompt template (with token insertion), output target (and, for task/send-to targets, the task kind / destination), model selector, and confirm-before-run. Editing an AI-command item SHALL persist immediately into the Favorites record, and the command SHALL be movable between bands like any other item. There SHALL be no separate AI-command editor window or sheet. - -#### Scenario: Edit an AI command in the item inspector -- **WHEN** the user selects an AI-command item on the Bands page -- **THEN** the inspector shows its full editable fields (name, icon, tint, input, prompt template, output, model, confirm-before-run) - -#### Scenario: AI command edits persist into Favorites -- **WHEN** the user changes an AI command's prompt template or output target on the Bands page -- **THEN** the change is saved into the Favorites record immediately and used on the next firing - -#### Scenario: An AI command moves between bands -- **WHEN** the user moves an AI-command item from one band to another -- **THEN** the command now belongs to the destination band and appears there in the launcher - -#### Scenario: No separate AI command editor -- **WHEN** the user wants to edit AI commands -- **THEN** they do so on the Bands page, and no standalone AI-command editor window or sheet exists - ### Requirement: Arrange items by context band on the canvas The editor canvas SHALL present the context bands as the user will swipe them, allowing: reordering items within a band by drag, reordering bands by drag, removing an item, creating a new band, and choosing which band is the active add target. The canvas SHALL be the same structure the launcher navigates. The canvas SHALL use two columns: a merged **bands column** and an always-visible **items column**. The bands column SHALL list the bands; clicking a band SHALL select it (its items appear in the items column) AND expose that band's source picker inline under its row, while the selected band's settings (icon, color, default app strategy) SHALL be pinned at the bottom of the same column. Re-clicking the selected band SHALL deselect it entirely — no band highlighted, its sources and settings hidden, and the items column showing its empty/placeholder state — the same clean state as first opening the Hub (no band SHALL be auto-selected on open). Clicking another band SHALL switch the selection and expose that band's sources. While a band is selected, the items column SHALL show its items grid (and the selected item's editor) — the items are not hidden behind a disclosure. diff --git a/openspec/specs/files-action-menu/spec.md b/openspec/specs/files-action-menu/spec.md deleted file mode 100644 index 7693f97..0000000 --- a/openspec/specs/files-action-menu/spec.md +++ /dev/null @@ -1,204 +0,0 @@ -# files-action-menu Specification - -## Purpose -TBD - created by archiving change add-files-band-actions. Update Purpose after archive. -## Requirements -### Requirement: An action menu over the highlighted file or folder - -The Files band SHALL offer an **action menu** for the highlighted entry, summoned by the configured **menu excursion** (default: the **`+1`-finger lift**). The menu SHALL be offered for **both files and folders** (today's Open-With trigger is files-only; the menu replaces it and works on folders too). The menu SHALL be a **scrubbable, lift-to-commit** surface that rides the existing drill grammar: while it is open, **highlight (vertical) scrubs the menu items**, **lift commits** the highlighted item, and the **four-finger discard backs out** of the menu (returning to folder navigation) without resolving. Entering the menu and backing out of it SHALL **re-arm the drill** so navigation never goes inert. - -#### Scenario: The menu opens on a folder - -- **WHEN** a folder is highlighted and the user performs the menu excursion -- **THEN** an action menu for that folder appears (it is no longer empty on folders) - -#### Scenario: The menu opens on a file - -- **WHEN** a file is highlighted and the user performs the menu excursion -- **THEN** an action menu for that file appears - -#### Scenario: Scrub and commit - -- **WHEN** the menu is open -- **THEN** vertical scrubbing moves the highlighted item and a lift commits the highlighted item - -#### Scenario: Discard backs out of the menu - -- **WHEN** the menu is open and the user performs the four-finger discard -- **THEN** the menu closes, no item is committed, folder navigation resumes, and the drill is re-armed - -### Requirement: Default menu contents per entry type - -The action menu's **default** contents SHALL be, in order: - -- **File:** Copy as path · Copy · Cut · Paste · Open in ▸ · Delete -- **Folder:** Copy as path · Copy · Cut · Paste · ‹installed terminals…› · Open in ▸ · Delete - -**Cut** and **Delete** are defaults for **both** entry types, with **Delete last** (set apart from the everyday actions). The ‹installed terminals…› group SHALL appear for **folders** and expand to one row per installed, enabled terminal. Items whose preconditions are unmet SHALL be hidden (for example, **Paste** SHALL be hidden when the pasteboard holds no file reference). The default lists SHALL be exactly these; any deviation is a user customization (see *configuration-hub* / *tunable-settings*). - -#### Scenario: Folder default menu lists terminals - -- **WHEN** the default menu opens on a folder and at least one terminal is installed and enabled -- **THEN** the menu shows Copy as path, Copy, Cut, Paste, a row per installed terminal, Open in ▸, and Delete in that order - -#### Scenario: File default menu omits terminals - -- **WHEN** the default menu opens on a file -- **THEN** the menu shows Copy as path, Copy, Cut, Paste, Open in ▸, and Delete — with no terminals group - -#### Scenario: Paste hides without a file on the clipboard - -- **WHEN** the menu opens and the pasteboard holds no file reference -- **THEN** the Paste item is not shown - -### Requirement: Copy as path writes the path to clipboard history - -The **Copy as path** item SHALL write the entry's **standardized absolute path** as text into the **clipboard history** (the persisted clipboard store) **and** the live pasteboard, so the path is both pasteable now and recallable from the Clipboard band. The history insert SHALL NOT be double-captured as a separate live-pasteboard event (self-write suppression). - -#### Scenario: Path is on the clipboard and in history - -- **WHEN** the user commits Copy as path on an entry -- **THEN** the entry's absolute path is on the live pasteboard and recorded once in clipboard history - -### Requirement: Copy writes the file or folder object to the pasteboard - -The **Copy** item SHALL write the entry's **file reference** to the pasteboard (the object, not its path text), so that a subsequent paste in Finder performs the real file/folder copy. Copy SHALL apply to both files and folders. - -#### Scenario: Copy then paste-in-Finder copies the object - -- **WHEN** the user commits Copy on a file or folder and later pastes in a Finder window -- **THEN** the actual file or folder is copied there (the pasteboard carried the object, not just a path string) - -### Requirement: Paste copies the clipboard's file into the target folder (bounded, keep-both) - -The **Paste** item SHALL copy the **file reference(s)** currently on the pasteboard **into** the target folder. The target SHALL be the highlighted **folder**, or — for a highlighted **file** — that file's **containing** folder. On a name conflict the copy SHALL **keep both** by auto-renaming the incoming copy (it SHALL NOT overwrite, move, or delete anything). Paste SHALL operate on **file URLs only** in this version; other pasteboard content SHALL NOT be pasted. A failed paste SHALL map at the boundary into the band's error taxonomy and surface as a **bounded, non-blocking** card with retry/dismiss — never an app-modal alert, never raw error text in a headline, and never a false success. - -#### Scenario: Paste copies into the folder - -- **WHEN** the pasteboard holds a file and the user commits Paste on a highlighted folder -- **THEN** that file is copied into the folder - -#### Scenario: Paste into a file targets its parent - -- **WHEN** the pasteboard holds a file and the user commits Paste on a highlighted file -- **THEN** the file is copied into that highlighted file's containing folder - -#### Scenario: Conflicts keep both, never overwrite - -- **WHEN** a paste target already contains an item with the same name -- **THEN** the incoming copy is auto-renamed (keep both); nothing is overwritten, moved, or deleted - -#### Scenario: A failed paste surfaces a bounded message - -- **WHEN** a paste cannot complete (for example permission denied or the source was removed) -- **THEN** the navigator shows a clean, bounded failure with retry/dismiss and nothing is silently dropped - -### Requirement: Open in presents a scrubbable app grid - -The **Open in ▸** item SHALL present a **scrubbable grid** of applications (the app-drawer presentation), navigated by the trackpad in **both axes** and **resolved on lift** (the highlighted app opens the entry). For a **file**, the grid SHALL list the applications capable of opening that file, with the default app indicated (today's Open-With set). For a **folder**, the grid SHALL list folder-openers (Finder plus the curated editors/terminals). The grid SHALL remain **pure-trackpad** — no keypress, and the overlay SHALL NOT become key/main for it. - -#### Scenario: Open in for a file lists capable apps as a grid - -- **WHEN** the user opens "Open in ▸" on a file -- **THEN** a scrubbable grid of the apps that can open that file appears, the default indicated, and a lift opens the file with the highlighted app - -#### Scenario: Open in for a folder lists folder-openers - -- **WHEN** the user opens "Open in ▸" on a folder -- **THEN** the grid lists Finder and the curated editors/terminals (a folder has no default-app list of its own) - -#### Scenario: The grid takes no keypresses - -- **WHEN** the app grid is open -- **THEN** it is navigated and resolved entirely by trackpad; the overlay never becomes key/main to capture keys - -### Requirement: Open in a terminal opens the folder as the working directory - -An installed-terminal item (and an "Open in ‹editor›" item) SHALL open the **folder** rooted as that tool's **working directory**. For a highlighted **file**, the tool SHALL open the file's **containing** folder. The set of offered terminals/editors SHALL be **auto-detected** from installed applications and **user-curated**. - -#### Scenario: Open folder in a terminal sets CWD - -- **WHEN** the user commits an installed-terminal item on a folder -- **THEN** that terminal opens with the folder as its working directory - -#### Scenario: Terminal on a file uses its parent folder - -- **WHEN** the user commits a terminal item on a highlighted file -- **THEN** the terminal opens rooted at the file's containing folder - -### Requirement: The menu contents and order are user-configurable per type - -The action menu SHALL draw from a **catalog** of actions — the per-type defaults plus opt-in extras: **Reveal in Finder**, **Add to Favorites** (insert the entry as a launcher favorite), **Open in ‹editor›**, and **Copy name**. The user SHALL be able to **add, remove, and reorder** items **independently per type** (file vs folder). Removing an item SHALL never remove a load-bearing safety (discard/back-out is always available). The configured menus SHALL persist (see *tunable-settings*). - -#### Scenario: User adds an extra and reorders - -- **WHEN** the user adds "Add to Favorites" to the file menu and moves it to the top -- **THEN** the file action menu shows it first, persistently across launches - -#### Scenario: File and folder menus are configured independently - -- **WHEN** the user customizes the folder menu -- **THEN** the file menu is unaffected, and vice versa - -#### Scenario: Add to Favorites bridges into the launcher - -- **WHEN** the user commits "Add to Favorites" on an entry -- **THEN** that file or folder is inserted as a launcher favorite item - -### Requirement: Cut moves on the next Paste - -The action menu SHALL offer a **Cut** action for a highlighted file or folder. Choosing Cut SHALL write the entry's file reference to the pasteboard and **mark it as cut** (recording the pasteboard's state at that moment). The menu's **Paste** action SHALL then be **dual-mode**: when the live pasteboard is **still that cut**, Paste SHALL **move** the marked entry into the target folder (a highlighted folder, or a highlighted file's containing folder); otherwise Paste SHALL **copy** as before. Both modes SHALL **keep both** on a name conflict (auto-rename) and SHALL NEVER overwrite. A completed move SHALL clear the cut mark. An intervening **Copy** (or any other pasteboard write) SHALL **supersede** the cut, so a subsequent Paste copies rather than moves. Cut SHALL be a default action for **both** files and folders. - -#### Scenario: Cut then Paste moves the entry - -- **WHEN** the user commits Cut on an entry, then commits Paste over another folder while the pasteboard is unchanged -- **THEN** the entry is **moved** into that folder (removed from its original location), keeping both names on conflict, and the cut mark is cleared - -#### Scenario: Copy then Paste still copies - -- **WHEN** the user commits Copy on an entry, then commits Paste over a folder -- **THEN** the entry is **copied** into that folder (the original stays), keeping both names on conflict - -#### Scenario: A Copy between Cut and Paste supersedes the cut - -- **WHEN** the user commits Cut, then commits Copy on a different entry, then commits Paste -- **THEN** Paste **copies** the most recently copied entry (the earlier cut no longer moves) - -### Requirement: Delete moves to the Trash - -The action menu SHALL offer a **Delete** action for a highlighted file or folder that moves it to the **Trash** (recoverable from Finder). It SHALL NOT permanently remove the entry. A failed delete SHALL surface as the existing **bounded, non-blocking** failure row (a clean headline + opt-in copyable details + Retry/Dismiss), never an app-modal alert and never raw error text in the headline. Delete SHALL be a default action for **both** files and folders, and SHALL be committed by the same **dwell-armed lift** as every other menu row (the deliberate rest-to-arm is the confirmation — there is no separate confirmation dialog). - -#### Scenario: Delete trashes the entry - -- **WHEN** the user commits Delete on a highlighted entry -- **THEN** the entry is moved to the Trash (recoverable) and is not permanently removed - -#### Scenario: A failed delete is observable, never silent - -- **WHEN** a Delete cannot complete (e.g. permission denied) -- **THEN** a bounded, non-blocking failure row appears with a clean headline and Retry — never an alert, never raw error text - -#### Scenario: Delete requires the dwell-arm - -- **WHEN** the user scrubs onto Delete and lifts before the row has armed -- **THEN** nothing is deleted and the overlay dismisses (the dwell-arm is the deliberate confirm) - -### Requirement: The action menu and its Open-With grid fit their content - -The action menu **and** its "Open in ▸" Open-With grid SHALL each size to their content — **width** derived from the widest row label (bounded to a sensible minimum and maximum) and **height** derived from the row count (header + rows + padding, bounded by a safety cap) — rather than rendering at a fixed size with empty space. A label too long for the maximum width SHALL truncate within it. The single sliding selection highlight SHALL continue to span the popup's width. - -#### Scenario: A short popup is compact - -- **WHEN** the action menu shows a few short rows (e.g. Copy · Cut · Delete), or the Open-With grid lists a couple of apps -- **THEN** the popup is sized to fit those rows in both dimensions, not padded out to a fixed width/height - -#### Scenario: A long label stays bounded - -- **WHEN** a row label is longer than the popup's maximum width -- **THEN** the label truncates within the bounded width rather than overflowing - -#### Scenario: Height tracks the row count - -- **WHEN** a popup has few rows -- **THEN** its height is just the header plus those rows (no empty vertical space), expanding with more rows up to the safety cap - diff --git a/openspec/specs/files-band/spec.md b/openspec/specs/files-band/spec.md deleted file mode 100644 index 9f20ed3..0000000 --- a/openspec/specs/files-band/spec.md +++ /dev/null @@ -1,234 +0,0 @@ -# files-band Specification - -## Purpose -TBD - created by archiving change files-band. Update Purpose after archive. -## Requirements -### Requirement: Opt-in Files band injected into the launcher - -The app SHALL provide a **Files band** as an opt-in (default **off**) that, when enabled, is injected into the four-finger launcher as a **synthetic band** — appended at launcher-open time like the Clipboard band and **never persisted** into the authored favorites. Enabling the Files band SHALL NOT relocate any native gesture, SHALL NOT require a re-login, and SHALL NOT request any new permission; it reads the local filesystem on demand. The opt-in SHALL take effect immediately (no `is…Effective` gate): toggling it on injects the band on the next launcher open, toggling it off removes it. - -#### Scenario: Disabled by default -- **WHEN** the app is first run with no prior configuration -- **THEN** the Files band is absent from the launcher and no filesystem is read - -#### Scenario: Enabling injects the band immediately -- **WHEN** the user turns the Files band opt-in on -- **THEN** the next time the launcher opens, the Files band appears as a band, with no re-login and no new permission prompt - -#### Scenario: The band is never written into authored favorites -- **WHEN** the Files band is enabled and shown -- **THEN** it is appended at launcher-open time and is not saved into the user's authored bands store - -### Requirement: Configured roots with remembered locations - -The Files band SHALL open onto a set of **user-configured local root folders** (its entry column). Each root SHALL track the **deepest location** the user last navigated to within it. A **"remember and reopen the last folder" toggle** (default ON) SHALL govern whether that remembered location is *used to land*: - -- When the toggle is **ON**, the band SHALL **open displaying** the remembered deepest folder (restored at open, with its ancestor chain reconstructed so ascending walks back up), and **re-entering a root** (descending into it from the roots list) SHALL restore that root's remembered location. -- When the toggle is **OFF**, the band SHALL **open on the roots list**, and descending into a root SHALL land on that root's **top level** — it SHALL NOT jump to the remembered location. The deepest-location *tracking* SHALL continue regardless of the toggle, so turning it back ON restores correctly. - -The displayed column and the navigation state SHALL always agree (no visual/state desync): what the band shows on open is exactly where horizontal navigation begins. Navigating fully out (ascending past a root's top level) SHALL return to the roots list. Roots SHALL reference **local** folders only. - -#### Scenario: Entry shows the configured roots -- **WHEN** the user lands on the Files band with the remember toggle OFF -- **THEN** the current column lists the configured root folders - -#### Scenario: A root remembers where you left off (toggle ON) -- **WHEN** the remember toggle is ON and the user descends several levels inside a root, leaves the band, and later re-enters that root -- **THEN** the navigator restores the folder the user was last in within that root - -#### Scenario: Toggle OFF never jumps to the last-visited folder -- **WHEN** the remember toggle is OFF and the user descends into a root from the roots list -- **THEN** the navigator lands on that root's top level, not the previously-visited deeper folder - -#### Scenario: Backing out returns to the roots list -- **WHEN** the user ascends from a root's top level -- **THEN** the current column returns to the roots list - -### Requirement: On-demand local directory listing - -The Files band SHALL list a folder's contents by reading the **live local filesystem on demand** (no background indexing, no recording, no persisted file cache). Listing SHALL be performed **off the main thread** so navigation never blocks the UI, and SHALL request only the resource values it needs (is-directory, modification date, regular-file). Each listed entry SHALL carry a **stable identity derived from its absolute path** so repeated listings and live directory changes do not cause the selection highlight to strobe or jump. Entries SHALL be ordered by a configurable **sort order**. Network and iCloud-placeholder locations are out of scope; only local entries are listed. - -#### Scenario: Folder contents are read live -- **WHEN** the user descends into a folder -- **THEN** that folder's current local contents are listed, reflecting the filesystem at that moment - -#### Scenario: Listing does not block the UI -- **WHEN** a large folder is opened -- **THEN** the navigator remains responsive while its contents load - -#### Scenario: Stable identity prevents strobing -- **WHEN** a folder is re-listed (on re-entry, or because a file changed on disk) -- **THEN** entries keep stable identities by path and the highlight does not flicker or jump - -### Requirement: Column navigation model - -The Files band SHALL maintain a **navigation stack**: an ordered list of **ancestor folders**, a **current folder** whose entries are listed, and a **highlighted entry** within it. Descending into a highlighted folder SHALL push the prior current folder onto the ancestors and make the highlighted folder current; ascending SHALL pop the deepest ancestor back to current. The highlighted entry's **preview target** SHALL be the entry itself — a file previews itself, a folder previews its own contents — so descending into a folder is equivalent to promoting that folder's preview to the current column. - -#### Scenario: Descend pushes an ancestor and makes the child current -- **WHEN** a folder is highlighted and the user descends -- **THEN** the prior current folder becomes the deepest ancestor and the highlighted folder becomes the current folder - -#### Scenario: Ascend pops back to the parent -- **WHEN** the user ascends -- **THEN** the deepest ancestor becomes the current folder again and the prior current folder is dropped - -#### Scenario: Highlighting a file targets the file for preview -- **WHEN** a file entry is highlighted -- **THEN** the preview target is that file - -#### Scenario: Highlighting a folder targets its contents for preview -- **WHEN** a folder entry is highlighted -- **THEN** the preview target is that folder's contents, which descending would promote to the current column - -### Requirement: Open a file or folder in the current Space - -Opening a highlighted entry SHALL open it for real: a **file** opens in its default application, a **folder** opens as a real Finder window. Open is an **action reached from the action menu** (and is the action a user MAY rebind the lift to); it is **no longer the default lift resolution** — the default lift **delivers** the entry to the captured front app (see *files-contextual-delivery*). The opened window SHALL appear on the **current Space** (it SHALL NOT teleport the user to another Space or app), and the open SHALL target the **front application captured when the launcher opened**, never whichever app is frontmost at the instant of firing (the overlay is non-activating). A failed open SHALL surface as an observable failure (see *Failures are observable, never silent*), never a silent false success. - -#### Scenario: Opening a file launches its default app on the current Space - -- **WHEN** the user commits "Open" on a highlighted file -- **THEN** the file opens in its default application and the resulting window appears on the current Space - -#### Scenario: Opening a folder opens a Finder window - -- **WHEN** the user commits "Open" on a highlighted folder -- **THEN** that folder opens as a Finder window on the current Space - -#### Scenario: Open targets the captured front app, not this overlay - -- **WHEN** an open is committed from the non-activating overlay -- **THEN** the action targets the app that was frontmost when the launcher opened, not the overlay - -#### Scenario: Open is not the default lift - -- **WHEN** the user lifts on a highlighted entry with the default lift binding -- **THEN** the entry is delivered to the captured front app (not opened); Open is performed only when chosen from the action menu or when lift is explicitly rebound to open - -### Requirement: Open With the relevant applications - -The Files band SHALL offer **Open With** as the **"Open in ▸"** item of the action menu, reachable for **both files and folders**. For a **file** it SHALL list **only the applications capable of opening that file** (not the full installed-apps list), with the file's **default application indicated**; for a **folder** it SHALL list folder-openers (Finder plus the curated editors/terminals). The candidate list SHALL be presented as a **scrubbable grid** (the app-drawer presentation) navigated by trackpad in both axes and resolved on lift. Choosing an application SHALL open the entry with that application, on the current Space, targeting the captured front-app context. The file's relevant-apps list SHALL be derived from the system's association of applications to that file and SHALL be computed **on demand**. - -#### Scenario: Open-With lists only capable apps for a file - -- **WHEN** the user opens "Open in ▸" on a highlighted file -- **THEN** the presented grid contains only applications that can open that file, with the default app indicated - -#### Scenario: Choosing an app opens the file with it - -- **WHEN** the user highlights an application in the grid and lifts -- **THEN** the file opens with the chosen application on the current Space - -#### Scenario: Open-With is now offered for folders too - -- **WHEN** the highlighted entry is a folder -- **THEN** "Open in ▸" is offered and lists folder-openers (Finder and the curated editors/terminals), rather than being absent - -### Requirement: Defusable open - -A committed open SHALL be **defusable** until it actually launches: it SHALL be held (briefly, or pending an explicit commit) so that a **discard** issued before it fires **cancels it and opens nothing**. Defusing SHALL **never terminate an already-running application** — it only prevents a not-yet-fired open. Once the target has actually been opened, a later discard SHALL NOT attempt to kill it. - -#### Scenario: Discard before launch opens nothing -- **WHEN** the user commits an open and then discards before the open has fired -- **THEN** nothing is opened and the navigator resolves with no side effect - -#### Scenario: Defusing never kills a running app -- **WHEN** the user discards an open whose target application was already running -- **THEN** the running application is left untouched (defuse cancels only the pending open) - -### Requirement: Failures are observable, never silent - -All Files-band side effects (directory listing, open, Open-With) SHALL map any underlying filesystem or workspace error into the app's shared error taxonomy **at the layer boundary**, and SHALL surface a failure as a **bounded, non-blocking** state carrying a clean headline — never an app-modal alert, never raw error text in a headline, and never a false "opened" when the side effect did not land. A failed open SHALL leave the navigator in an observable failed state from which the user can retry or discard. - -#### Scenario: A failed open surfaces a clean, bounded message -- **WHEN** an open fails (for example the file was removed, or no application can open it) -- **THEN** the navigator shows a clean, bounded failure message (not raw error text, not an app-modal alert) and offers retry or discard - -#### Scenario: A listing error does not crash or hang -- **WHEN** a folder cannot be read (for example permission denied) -- **THEN** the navigator surfaces a bounded message in place and remains responsive - -### Requirement: Local-only, non-destructive scope - -The Files band SHALL remain **local-only** and SHALL NOT read network or iCloud-placeholder locations. Its filesystem mutations SHALL be bounded to **recoverable, non-overwriting** operations — it SHALL never cause **irreversible data loss**. Specifically the band MAY: - -- **copy a file into** a folder (Paste after Copy) — **keep-both** on conflict (auto-rename); -- **move a file into** a folder (Paste after **Cut**) — **keep-both** on conflict (auto-rename), relocating the source (not destroying it); -- **move an entry to the Trash** (**Delete**) — recoverable from the Finder Trash. - -The band SHALL NOT **permanently delete** (it trashes, never `removeItem`), **overwrite** (every copy/move keeps both on conflict), **rename in place**, or **tag** any file or folder. All navigation, delivery, copy-to-clipboard, and open actions remain non-mutating. These are the only widenings of the original navigate-and-open scope, and each is recoverable. - -#### Scenario: Mutations are bounded to copy-in, move-in, and trash - -- **WHEN** the user acts in the Files band -- **THEN** the only operations that touch the filesystem are copy-into-folder, move-into-folder (Cut→Paste), and move-to-Trash (Delete) - -#### Scenario: No permanent delete or overwrite is exposed - -- **WHEN** the user navigates the Files band -- **THEN** no available action permanently deletes (Delete trashes, recoverably), overwrites (copies/moves keep both on conflict), renames in place, or tags any file or folder - -#### Scenario: Network and iCloud locations stay out of scope - -- **WHEN** the user configures or navigates roots -- **THEN** only local entries are listed; network and iCloud-placeholder locations are not read - -### Requirement: Dwell-to-arm gates Files-band resolution - -The Files band SHALL arm by **dwell**, like every other launcher surface (mirroring *launcher-overlay → Dwell-to-arm with feedback*). Resting the highlight on a row — in the navigator **or** in any sub-column (the action menu, the Open-With picker, the "Open in ▸" app grid) — for at least the configured dwell-to-arm duration SHALL **arm** that row. Arming SHALL be signalled by the existing best-effort haptic **arm tick** and a visual **charge-ring** that fills over the dwell duration and locks when armed. Moving the highlight — a highlight step, a depth descend/ascend, an async re-list that shifts the highlighted row, or a sub-column move — SHALL **reset the dwell and disarm**. Holding at the trackpad edge (auto-drill / highlight auto-repeat) SHALL re-charge on every step, so it never arms mid-scroll. Adding the `+1` finger SHALL NOT reset the dwell (it does not change the highlighted item); **entering** a sub-column SHALL begin a fresh dwell on its first row. - -This **supersedes** the band's prior resolve-on-lift-without-arming behavior, and supersedes the band's "add no new haptics" note **for the arm moment only** — the arm tick is the product's existing single haptic ("moments of arrival"), not a new pattern; no per-scrub, per-descend, or per-commit haptics are added. The dwell duration is the same `dwellToArmDuration` that governs the rest of the launcher (no Files-specific setting). - -#### Scenario: Dwell arms the highlighted row - -- **WHEN** the highlight rests on a Files row for at least the dwell duration -- **THEN** the row becomes armed, the arm haptic fires (if available), and the charge-ring shows armed - -#### Scenario: Charge-ring tracks partial dwell - -- **WHEN** the highlight has rested on a row for less than the dwell duration -- **THEN** the charge-ring is partially filled and the row is not armed - -#### Scenario: Moving the highlight disarms - -- **WHEN** a row is armed and the user steps the highlight, descends/ascends, or scrubs a sub-column to another row -- **THEN** the previous row disarms, its ring empties, and the new row begins its own dwell - -#### Scenario: Auto-drill never arms mid-scroll - -- **WHEN** the user holds at the trackpad edge and the tree auto-drills (or the highlight auto-repeats) -- **THEN** the dwell re-charges on every step and no row arms until the motion settles - -#### Scenario: Adding the +1 finger preserves the arm - -- **WHEN** a row is armed and the user adds the `+1` finger without moving the highlight -- **THEN** the row stays armed (the dwell is not reset) - -### Requirement: Files lift fires only when armed - -A **committing** Files lift SHALL fire **only when the highlighted row is armed**; if no row is armed, lifting SHALL **dismiss the overlay** without acting (mirroring *launcher-overlay → Lift fires only when armed*). This SHALL apply to every committing resolution — the default lift (**deliver** to the captured front app, or **open** when the lift is rebound), the `+1`-finger lift (**open the action menu**), and a **lift that commits a sub-column row** (an action-menu row, an Open-With / app-grid app). A quick scrub-and-lift (no dwell) SHALL therefore never deliver, open, open the menu, or commit a row. The four-finger **discard** (back-out) SHALL **never** be gated by arm — it backs out one level (or dismisses) armed or not, and SHALL NOT terminate a running application (the *Defusable open* rule is unchanged). The arm gate SHALL sit **before** the action fires, so the existing defuse window and observable-failure behavior are unchanged. - -#### Scenario: Armed lift acts - -- **WHEN** a Files row is armed and the fingers lift -- **THEN** the committing action fires (deliver / open / open-menu / commit the row) and the overlay resolves as before - -#### Scenario: Unarmed lift dismisses - -- **WHEN** the fingers lift while no Files row is armed -- **THEN** the overlay hides and nothing is delivered, opened, or committed - -#### Scenario: Scrub-and-lift never delivers - -- **WHEN** the user scrubs onto a file and lifts before the dwell completes -- **THEN** nothing is delivered to the front app and the overlay dismisses - -#### Scenario: The +1-finger menu requires an armed row - -- **WHEN** the user adds the `+1` finger and lifts on a row that has not armed -- **THEN** the action menu does not open and the overlay dismisses - -#### Scenario: Discard is never gated by arm - -- **WHEN** the user issues the four-finger discard while no row is armed -- **THEN** the back-out / dismiss happens normally and no running application is terminated - diff --git a/openspec/specs/files-contextual-delivery/spec.md b/openspec/specs/files-contextual-delivery/spec.md deleted file mode 100644 index 19aa9ff..0000000 --- a/openspec/specs/files-contextual-delivery/spec.md +++ /dev/null @@ -1,75 +0,0 @@ -# files-contextual-delivery Specification - -## Purpose -TBD - created by archiving change add-files-band-actions. Update Purpose after archive. -## Requirements -### Requirement: Lift delivers the highlighted entry to the captured front app - -The Files band's **default lift resolution** SHALL be to **deliver** the highlighted entry to the **front application captured when the launcher opened** (the overlay is non-activating), never to whichever app is frontmost at the instant of firing. Delivery SHALL write the entry to the pasteboard and synthesize a **paste** into that captured app. Open-to-default SHALL remain available as an action (reached from the action menu), but SHALL NOT be the default lift outcome; the lift action SHALL be user-configurable (a user MAY rebind lift back to open). Delivery SHALL target the **current Space** and SHALL NOT teleport the user to another Space or app. - -#### Scenario: Landing on a file delivers it to where you came from - -- **WHEN** the user lifts on a highlighted file with the default lift binding -- **THEN** the file is delivered (pasted) into the app that was frontmost when the launcher opened, not into the overlay and not into whatever became frontmost at firing - -#### Scenario: Open is still reachable, just not the default lift - -- **WHEN** the user wants to open a file in its default app -- **THEN** "Open" is available from the action menu, and the lift binding MAY be set back to open — delivery is the default only - -### Requirement: macOS routes the representation per target, with no context detection - -Delivery SHALL write **both** representations of the entry in a single pasteboard item: the entry's **file reference** (`fileURL`) **and** its **standardized absolute path** as a string. The app SHALL NOT inspect the front app to decide which form to send; the receiving app SHALL select the representation it understands. A **text destination** (text field, terminal, code editor) SHALL receive the **path string**; a **Finder window** SHALL receive the **file** (copying it into the displayed folder). - -#### Scenario: Path lands in a text field - -- **WHEN** the captured front app is a terminal or text editor and a file is delivered -- **THEN** the file's absolute path is inserted as text at the cursor - -#### Scenario: File lands in a Finder folder - -- **WHEN** the captured front app is a Finder window showing a folder and a file is delivered -- **THEN** the file is copied into that folder (Finder consumes the file reference) - -#### Scenario: No front-context probing - -- **WHEN** any delivery occurs -- **THEN** the same dual-representation item is written for every target; the routing is the receiver's choice, not an app-side detection step - -### Requirement: Delivery preserves the user's clipboard - -Delivery SHALL **snapshot** the user's current clipboard, write the delivery item, synthesize the paste, and then **restore** the prior clipboard, so that delivering an entry never clobbers what the user had copied. The restore SHALL cover non-text clipboard contents (images, file references) as well as text. - -#### Scenario: Clipboard is intact after delivery - -- **WHEN** the user has content on the clipboard, delivers a file from the band, and then pastes manually elsewhere -- **THEN** their original clipboard content is what pastes — the delivery did not overwrite it - -### Requirement: Delivery into an open or save panel navigates it to the path - -WHEN the captured front context is a detected **open/save panel** (a file picker), delivery SHALL **navigate the panel to the entry's path** (drive the panel's go-to-folder path entry) rather than paste a file reference, so the picker lands on that location. Detection SHALL be conservative: if the front context is **not** confidently a file picker, delivery SHALL fall back to the dual-representation pasteboard contract so the common text/Finder cases never misfire. - -#### Scenario: A file picker jumps to the delivered path - -- **WHEN** an open/save panel is the detected front context and the user delivers a file -- **THEN** the panel navigates to that file's location - -#### Scenario: Uncertain detection falls back to the contract - -- **WHEN** the front context cannot be confidently identified as a file picker -- **THEN** delivery uses the dual-representation pasteboard paste (path-or-file by receiver), never a misdirected picker action - -### Requirement: Delivery is observable, never a false success - -Delivery SHALL surface a **bounded, non-blocking** failure (the band's existing failure surface, clean headline, no app-modal alert, no raw error text in a headline) for the parts it **can** observe: there is **no** captured front app, or the pasteboard write fails. The app SHALL NOT claim a confirmed "Done" for the synthesized keystroke itself (which it cannot verify landed) — it reports delivery **attempted** and never fabricates a confirmation it cannot obtain. A delivery that fails an observable step SHALL leave the navigator in a state from which the user can retry or discard. - -#### Scenario: No front app surfaces a bounded failure - -- **WHEN** there is no captured front application to deliver into -- **THEN** the navigator shows a clean, bounded message and offers retry or discard, rather than silently doing nothing or claiming success - -#### Scenario: The keystroke is not falsely confirmed - -- **WHEN** a paste is synthesized into a front app that has no text target -- **THEN** the app does not display a confirmed "Done" for the keystroke (it cannot observe the landing), and it never shows a false success - diff --git a/openspec/specs/first-run-onboarding/spec.md b/openspec/specs/first-run-onboarding/spec.md index 1a27ea3..c741111 100644 --- a/openspec/specs/first-run-onboarding/spec.md +++ b/openspec/specs/first-run-onboarding/spec.md @@ -94,16 +94,12 @@ On completion the system SHALL record a single first-run-completed flag and SHAL - **THEN** those acts show their done states and no setting is written unless the user makes a new choice ### Requirement: Optional features are offered honestly and lazily -The wizard SHALL offer clipboard history, AI commands, and keyboard language as optional cards that state each feature's true cost plainly (clipboard records copied content locally; AI requires a one-time multi-gigabyte on-device model download and Apple Silicon; keyboard language needs no permission or re-login). Toggles SHALL write the same persisted preferences as the Hub pages. The wizard SHALL NOT request Calendar, Reminders, or Contacts permissions, SHALL NOT block on the AI model download, and SHALL treat skipping every optional feature as a first-class path to completion. +The wizard SHALL offer clipboard history and keyboard language as optional cards that state each feature's true cost plainly (clipboard records copied content locally; keyboard language relocates nothing and needs no new permission), all default off; enabling one never blocks the wizard's flow. #### Scenario: Skipping everything completes cleanly - **WHEN** the user declines all optional features - **THEN** the wizard reaches completion with the core switcher configured and nothing else changed -#### Scenario: AI enablement does not block -- **WHEN** the user enables AI commands in the wizard -- **THEN** the model download proceeds in the background via the existing machinery and the wizard advances without waiting - ### Requirement: The curtain offers permanence The wizard's final act SHALL offer Open at Login (using the existing registration path, including its /Applications failure guidance), show where the app lives (the menu-bar mark) and where configuration lives (the Hub), and present a clear completion state. diff --git a/openspec/specs/gesture-recognition/spec.md b/openspec/specs/gesture-recognition/spec.md index 6c3cd1e..d3792b3 100644 --- a/openspec/specs/gesture-recognition/spec.md +++ b/openspec/specs/gesture-recognition/spec.md @@ -199,44 +199,3 @@ Across navigation surfaces, adding a contact **above the current relaxed baselin - **WHEN** the action-menu intent has already resolved for the session and the fingers re-lift - **THEN** no further intent is emitted until the surface re-arms the session -### Requirement: Files-drill modal sub-state - -While the Files band's column navigator is open, the recognizer SHALL enter a **files-drill modal sub-state** that, like the AI canvas-resolution mode, **bypasses the normal finger-count latch**: gesture handling SHALL be routed to the drill tracker **before any idle re-latch**, so a fresh contact during drill-in SHALL NOT open the switcher or a second launcher on top of the navigator. The sub-state SHALL be **entered when the navigator opens** (signalled by the launcher controller) and **cleared when it resolves or hides**. - -While in the sub-state the recognizer SHALL: -- remain active while **two or more** contacts remain, measuring travel from the **centroid of the remaining contacts**, and **re-baseline** the reference origin (clearing any sub-step carry) on **every contact-count change**, so a leaving or landing finger emits no spurious step; -- emit a **depth** intent (with direction) per item-step distance of **horizontal** travel and a **highlight** intent (with direction) per step of **vertical** travel, honoring the launcher's direction-inversion settings; -- detect a **relative +1 finger** — a contact count that rises **above the current relaxed baseline** (not an absolute count of three) — and, on the subsequent resolving lift, emit an **open-with** resolution instead of a plain **open**; -- treat a fresh deliberate **four-finger horizontal swipe-away** as a **discard** resolution; -- treat the resolving lift (contact count dropping below two, with the standard below-target debounce) as a **one-shot** resolution (open / open-with / discard), so a stray re-lift after resolution is a **no-op**. - -The recognizer SHALL NOT itself implement directory navigation, preview, dwell, arm, or fire — it emits only intents; the controller and model interpret them (an up-step while already at the top of the list simply clamps — the navigator is pure-trackpad with no search to focus). - -#### Scenario: Drill-in bypasses the latch -- **WHEN** the navigator is open and a fresh contact lands -- **THEN** the recognizer routes to the drill tracker and does not open the switcher or a second launcher - -#### Scenario: Horizontal emits depth, vertical emits highlight -- **WHEN** in the drill sub-state the user travels horizontally or vertically past the step distance -- **THEN** the recognizer emits a depth intent (horizontal) or a highlight intent (vertical) in the travel direction, per the direction settings - -#### Scenario: Relative plus-one finger arms Open-With -- **WHEN** the user adds a finger above the current relaxed baseline and then lifts to resolve on a file -- **THEN** the recognizer emits an open-with resolution rather than a plain open - -#### Scenario: Plus-one is relative, not an absolute three -- **WHEN** the user has relaxed to three fingers and then adds a fourth (baseline three rising to four) -- **THEN** the added finger is detected as the Open-With morph (the trigger is "a finger was added", not "exactly three fingers") - -#### Scenario: Re-baseline on contact change emits no spurious step -- **WHEN** the contact count changes and the centroid shifts as fingers leave or land -- **THEN** the reference origin is re-baselined and no depth or highlight intent is emitted from the count change alone - -#### Scenario: Four-finger horizontal swipe discards -- **WHEN** in the drill sub-state the user makes a fresh deliberate four-finger horizontal swipe-away -- **THEN** the recognizer emits a discard resolution - -#### Scenario: Resolution is one-shot -- **WHEN** the selection has resolved (opened / open-with / discarded) and the user re-lifts -- **THEN** the recognizer emits nothing further (the re-lift is a no-op) - diff --git a/openspec/specs/launch-actions/spec.md b/openspec/specs/launch-actions/spec.md index 5fecfc5..fd650a4 100644 --- a/openspec/specs/launch-actions/spec.md +++ b/openspec/specs/launch-actions/spec.md @@ -215,14 +215,3 @@ The launcher editor SHALL let the user toggle "save to clipboard" on a Screensho - **WHEN** favorites saved before this option are loaded - **THEN** every `.action` item decodes successfully with the save-to-clipboard option off, and the favorites are not reset to defaults -### Requirement: Speak-last-response is a launcher system action -The system SHALL offer "Speak Last Response" as a one-shot launcher system action (System category): firing it SHALL resolve the frontmost window, read its text via accessibility, extract the final assistant reply (model-assisted when the model is resident; the last visible lines otherwise), and speak it aloud — identical behavior to the menu-bar trigger, reachable from any launcher band via the existing favorites editor. The action SHALL be dispatched through an injected seam (the launcher never references the AI coordinator directly), and a failure SHALL surface as one spoken line — bounded, never a modal. - -#### Scenario: Fired from a band -- **WHEN** the user fires a "Speak Last Response" item from a launcher band with a terminal frontmost -- **THEN** the terminal's last assistant reply is read aloud, exactly as the menu-bar item does - -#### Scenario: Present in the action picker -- **WHEN** the favorites editor lists system actions -- **THEN** "Speak Last Response" appears in the System category and can be added to any band - diff --git a/openspec/specs/launcher-overlay/spec.md b/openspec/specs/launcher-overlay/spec.md index 7f5e8c8..178f0e8 100644 --- a/openspec/specs/launcher-overlay/spec.md +++ b/openspec/specs/launcher-overlay/spec.md @@ -112,9 +112,6 @@ Landing the selection on an item and holding it (no further stepping) for at lea ### Requirement: Lift fires only when armed Lifting the fingers SHALL fire the currently armed item; if no item is armed, lifting SHALL dismiss the overlay without firing anything. A quick scrub-and-lift (no dwell) SHALL therefore never fire an item. The overlay SHALL be ordered out **before** the armed item is fired, so an action that switches Spaces (e.g. Next/Previous Space) does not carry the still-visible overlay onto the destination Space (the panel can join all Spaces, so firing first would leave it lingering there). -An **AI command item** is an exception to the order-out-before-fire rule: firing it does NOT dismiss the overlay. Instead, firing **begins the command and opens its streaming preview canvas**, leaving the overlay visible; the command then resolves through the AI command preview-and-commit behavior (a fresh four-finger **down** swipe commits, a fresh four-finger **horizontal** swipe discards) rather than completing on this first lift. Because the firing lift has already raised the fingers, the canvas is resolved by a *new* swipe, never by re-lifting; a stray lift while the canvas is open is a no-op. The order-out-before-fire rule continues to apply to items that complete on lift (launches, Space switches, paste-on-fire). - -A **screen-region (vision) command** is a further exception **within** the AI-command exception: because it must reveal the desktop so the user can designate a region, firing it **does** order the overlay out first (like a completing action), then presents the interactive region picker. The streaming preview canvas opens only **after** a region is captured, and the command then resolves through the normal preview-and-commit behavior. If the picker is **cancelled** (a click without a drag), no canvas opens, nothing is generated, and the captured front app is restored. #### Scenario: Armed lift fires - **WHEN** an item is armed and the fingers lift @@ -132,18 +129,6 @@ A **screen-region (vision) command** is a further exception **within** the AI-co - **WHEN** an armed Next/Previous Space item is fired on lift - **THEN** the overlay is dismissed before the Space switch, and it does not appear on the destination Space -#### Scenario: Armed AI command lift opens the preview canvas -- **WHEN** an armed AI command item is lifted -- **THEN** the command begins, the overlay stays visible, and its streaming preview canvas appears instead of the overlay dismissing - -#### Scenario: Armed screen-region command dismisses the overlay then picks -- **WHEN** an armed screen-region (vision) AI command item is lifted -- **THEN** the overlay is dismissed, the interactive region picker is presented over the revealed desktop, and the preview canvas opens only after a region is captured - -#### Scenario: Cancelled region pick opens no canvas -- **WHEN** the region picker is cancelled by a click without a drag -- **THEN** no preview canvas opens, nothing is generated, and the captured front app is restored - ### Requirement: Context-band visual encoding The launcher SHALL render the band strip as a **vertical list of band icons on the left** (icons only, never names), with the **active** band's icon drawn in its band color and the rest colorless. It SHALL render each item as an icon plus a short label tinted/accented by its context band color, and SHALL visually distinguish item kinds (e.g. a badge for presets, a marker for scripts). Each band's icon is user-configurable; the Clipboard band uses a dedicated preset icon. @@ -223,240 +208,3 @@ In the **Clipboard band**, horizontal auto-repeat SHALL be suppressed (there hor - **WHEN** the contact moves back off the edge or lifts - **THEN** auto-repeat stops -### Requirement: AI availability is resolved in the preview canvas, not by hiding items -AI-command items SHALL always appear and be fireable in the launcher regardless of whether AI is enabled or the model is downloaded. When an AI-command item is fired while AI is **disabled** or the selected model is **not yet available** (not downloaded or not ready), the overlay SHALL open the AI preview canvas in an **unavailable** state — a non-error presentation showing a clear message (a clean, bounded string — either an error headline routed through the single AI error→message translator, or a clear non-error guidance string; never raw error text), an **Enable** affordance that turns the AI opt-in on, a **Download** action that begins fetching the model, and a **model picker** to choose the desired model. This canvas SHALL be **dismissable with the normal swipe-to-resolve gesture** (a horizontal discard), and any download it starts SHALL continue **in the background** after dismissal. The unavailable state SHALL NOT be surfaced via an app-modal alert and SHALL NOT block; it is bounded and non-blocking per the AI error-handling convention. When AI is enabled and the model becomes ready, firing an AI-command item SHALL proceed to normal streaming. - -#### Scenario: Firing an AI item with AI off opens the enable/download canvas -- **WHEN** an AI-command item is fired while the AI opt-in is off -- **THEN** the preview canvas opens in the unavailable state offering Enable, Download, and a model picker, and nothing is generated yet - -#### Scenario: Firing with the model not downloaded offers download -- **WHEN** an AI-command item is fired while AI is enabled but the model is not downloaded -- **THEN** the canvas shows the unavailable state with a Download action and a model picker - -#### Scenario: Canvas is dismissable and the download continues in the background -- **WHEN** the user starts the model download from the unavailable canvas and then dismisses the canvas with a horizontal discard swipe -- **THEN** the canvas closes and the download continues in the background - -#### Scenario: Once available, firing streams normally -- **WHEN** AI is enabled and the selected model is ready and an AI-command item is fired -- **THEN** the command begins and its result streams into the preview canvas as usual (no unavailable state) - -### Requirement: AI command streaming preview canvas -When an AI command is fired, the overlay SHALL present a preview canvas (reusing the master-detail preview surface) into which the model's result is **streamed incrementally** as it is generated. The captured front app SHALL remain frontmost throughout (the overlay stays non-activating), and the canvas SHALL show a loading state while the model is loading or before the first tokens arrive. The canvas SHALL be the surface for **vision (screen-region) command results** as well — a vision command's text result streams into the same canvas exactly as a text command's does. - -#### Scenario: Result streams into the canvas -- **WHEN** an AI command is generating -- **THEN** the preview canvas fills with the result incrementally rather than only at completion - -#### Scenario: Loading is shown before tokens -- **WHEN** the model is loading or has not yet produced output -- **THEN** the canvas shows a loading state rather than appearing blank or frozen - -#### Scenario: Front app stays focused -- **WHEN** the preview canvas is visible -- **THEN** the previously focused app remains key and the overlay never becomes the key window - -#### Scenario: A vision command result streams into the same canvas -- **WHEN** a screen-region (vision) command is fired and the model produces a grounded answer -- **THEN** that answer streams into the same preview canvas as any text command's result - -### Requirement: Swipe-to-resolve (commit / discard) for AI commands -After an AI command's result is shown in the preview canvas, a fresh **two-finger down swipe SHALL commit** the result (routing it per the command's output target — paste/replace, or run the task; "bringing the result into the document") and a fresh **two-finger horizontal swipe (deliberate excursion) SHALL discard** it (cancelling any in-flight generation and writing nothing). Committing or discarding SHALL then dismiss the overlay. A down swipe before the result is committable (still loading or streaming) SHALL be ignored — the user waits — while a horizontal discard SHALL be honored at any time. An **up** swipe SHALL be ignored, so a stray upward motion never throws the result away. The resolution swipe SHALL be a **deliberate excursion past a threshold larger than incidental two-finger scrolling**, so reading/scrolling the canvas is not mistaken for a resolve; because the firing lift has already raised the fingers, resolution is always a new swipe, and a re-lift while the canvas is open commits nothing. - -This aligns the platform grammar: **four fingers open/dismiss the platform, two fingers act within it** — so the canvas (which is summoned by a two-finger trigger) is also resolved by two fingers, replacing the previous four-finger resolution. - -#### Scenario: Two-finger down swipe commits -- **WHEN** the result is committable and the user makes a deliberate two-finger down swipe -- **THEN** the result is routed to the command's output target and the overlay hides - -#### Scenario: Down swipe before ready is ignored -- **WHEN** the user swipes down while the model is still loading or streaming -- **THEN** nothing is committed and the canvas stays open until the result is ready - -#### Scenario: Two-finger horizontal swipe discards and cancels -- **WHEN** the result is streaming or shown and the user makes a deliberate two-finger horizontal swipe to discard -- **THEN** generation is cancelled, nothing is written, and the overlay hides - -#### Scenario: Up swipe is ignored -- **WHEN** the user swipes up while the canvas is open -- **THEN** nothing is committed or discarded and the canvas stays open - -#### Scenario: Scrolling the canvas is not mistaken for a resolve -- **WHEN** the user scrolls the canvas content with a small two-finger motion below the resolve excursion threshold -- **THEN** the canvas scrolls and neither commit nor discard is triggered - -### Requirement: Armed-confirmation state for side-effecting tasks (when enabled) -For an AI command whose output is a side-effecting task **with `confirmBeforeRun` enabled** (the default for side-effecting tasks), the preview canvas SHALL enter a distinct **armed-confirmation ("review the action") state** that displays the parsed action's concrete fields before it can be committed; the side effect SHALL fire only from this confirmed commit. When `confirmBeforeRun` is disabled for the command, the canvas SHALL NOT require this extra state and the task commits on the normal commit (down) swipe. A horizontal discard swipe SHALL always cancel with no side effect. - -#### Scenario: Task shows the parsed action before firing (review enabled) -- **WHEN** a side-effecting task whose command has confirmation enabled has produced its parsed action -- **THEN** the canvas enters the armed-confirmation state showing the action's fields, and nothing is applied yet - -#### Scenario: Confirm commit fires the side effect -- **WHEN** the user commits from the armed-confirmation state -- **THEN** the task executes its side effect and the overlay hides - -#### Scenario: Review disabled commits without the extra state -- **WHEN** a side-effecting task whose command has confirmation disabled is committed -- **THEN** the task executes on the normal commit (down) swipe without a separate armed-confirmation state - -#### Scenario: Discard cancels the side effect -- **WHEN** the user swipes horizontally to discard before the side effect fires -- **THEN** no side effect occurs and the overlay hides - -### Requirement: In-canvas runtime-parameter (language) selection re-runs the command -For an AI command that declares a runtime parameter (v1: a target language), the preview canvas SHALL present an **in-canvas control** (a language dropdown) reflecting the command's active value. Choosing a different value SHALL **re-run the command in place** — cancelling the in-flight generation (cancellation is not a failure) and starting a new generation with the re-resolved prompt (`{lang}` ⇒ the new value) streaming into the same canvas — **without reopening the launcher** or losing the captured front app. The newly chosen value SHALL be persisted (per command) so the next run defaults to it, and the control's initial selection SHALL reflect that persisted value. The dropdown SHALL offer a fixed list of languages (no free-form text entry, keyboardless). - -#### Scenario: Picking a language re-translates in place -- **WHEN** a translate result is shown and the user picks a different language from the in-canvas dropdown -- **THEN** the current generation is cancelled and the command re-runs to the new language, streaming into the same canvas - -#### Scenario: The dropdown opens on the remembered language -- **WHEN** the user previously translated to "Hebrew" with this command and fires it again -- **THEN** the canvas opens with the dropdown set to "Hebrew" and translates to Hebrew by default - -#### Scenario: Re-run keeps the captured app and output target -- **WHEN** the command re-runs after a language change -- **THEN** the captured front app remains frontmost and a subsequent commit still routes to the command's output target - -#### Scenario: A command with no runtime parameter shows no dropdown -- **WHEN** a command that declares no runtime parameter is fired -- **THEN** the canvas shows no language dropdown - -### Requirement: Collapsible live Thinking section and scrollable, input-capturing canvas -When a **reasoning** command streams, the preview canvas SHALL present the model's **thinking** in a **collapsible** section that is **collapsed by default** — showing a live activity indicator (an **elapsed-time** readout) so the user can see the model is actively working (not stuck or silently slow) without it sprawling across the screen — **expandable on tap** to watch the thinking stream live, and **scrollable** when long. The committed/inserted result SHALL remain the **response** only (thinking is never committed). The canvas's thinking and response panes SHALL be **scrollable**, and while the canvas is open it SHALL **capture 1–2-finger scroll** (routing it to the canvas content, not the front app) until the canvas is dismissed; the four-finger commit/discard swipe SHALL continue to resolve the canvas. - -> Note: the earlier animated "breathing" **pulse** on this indicator was **removed** — its `TimelineView` kept ticking after reasoning finished and while the canvas was hidden, pinning the main thread (see `docs/postmortem-idle-cpu-spin.md`). The elapsed timer (which freezes when reasoning ends) stays; any reintroduced pulse MUST pause when the canvas is not visible. - -#### Scenario: Thinking shows collapsed by default, expandable -- **WHEN** a reasoning command is generating -- **THEN** the canvas shows a collapsed Thinking section with a live elapsed timer; tapping it expands a scrollable live view of the thinking, and tapping again collapses it - -#### Scenario: Only the response is committed -- **WHEN** the user commits a reasoning command's result -- **THEN** only the response is inserted into the front app (or used by the task); the thinking is never committed - -#### Scenario: Scroll routes to the open canvas -- **WHEN** the canvas is open and the user does a 1–2-finger scroll -- **THEN** the canvas content scrolls (through the thinking or the response) rather than the front app, until the canvas is dismissed; a four-finger swipe still commits/discards - -### Requirement: Bidirectional (RTL/LTR) text rendering in the preview canvas -The preview canvas SHALL render text **bidirectionally**: each paragraph's **base direction SHALL be natural (first-strong)** — derived from its first strong directional character — so a right-to-left paragraph (e.g. Hebrew or Arabic) starts from the correct side and aligns correctly, while a left-to-right paragraph remains left-aligned. **Mixed** left-to-right and right-to-left runs within a paragraph SHALL resolve via the Unicode Bidi algorithm so combined text reads cleanly (e.g. a Latin word or URL inside a Hebrew sentence). This SHALL apply to the **streamed output**, the **input echo**, and the **task-review fields**. Because streaming may deliver the first strong character late, the base direction SHALL be **recomputed as content streams** rather than fixed at the first token. - -#### Scenario: A right-to-left result starts from the correct side -- **WHEN** the streamed result is Hebrew text -- **THEN** it renders right-aligned with a right-to-left base direction and correct punctuation placement - -#### Scenario: Mixed-direction text resolves cleanly -- **WHEN** a paragraph contains both Hebrew and an embedded Latin word or URL -- **THEN** the paragraph's base direction follows its first strong character and the embedded run is placed correctly by the Bidi algorithm - -#### Scenario: Base direction updates as tokens stream -- **WHEN** the first strong directional character arrives after some neutral characters have already streamed -- **THEN** the canvas updates the paragraph's base direction to match rather than locking to the environment direction - -#### Scenario: Left-to-right text is unaffected -- **WHEN** the streamed result is English text -- **THEN** it renders left-aligned with a left-to-right base direction as before - -### Requirement: Files band column-navigator layout - -The Files band SHALL render as a **column navigator** rather than the icon grid: a thin **icon rail of ancestor folders** on the left (one collapsed icon per ancestor, the deepest nearest the current column), a single **current-folder list** (the full vertical list of the current folder's entries, each with a type glyph), and a single **live preview** pane on the right. At any depth exactly one current-folder list and one preview are shown at full size; all ancestors are collapsed to the icon rail, so the visible width is **bounded regardless of navigation depth**. The overlay SHALL be sized to show several entries and a sizeable preview at once, and SHALL be sized to its **final frame** for a given depth (not re-stretched mid-animation). - -#### Scenario: Layout is rail + current list + preview -- **WHEN** the Files band is shown at any depth -- **THEN** ancestors appear as a left icon rail, the current folder is a full vertical list, and the highlighted entry has a live preview on the right - -#### Scenario: Visible width stays bounded with depth -- **WHEN** the user descends many levels -- **THEN** only one current list and one preview are shown at full size; deeper ancestors remain collapsed icons - -### Requirement: Files band depth and highlight navigation - -In the Files band, **horizontal** travel SHALL be the **depth** axis and **vertical** travel SHALL move the **highlight** within the current folder: horizontal in the descend direction enters the highlighted folder (collapsing the prior current list into its ancestor icon and budding the child list in), horizontal in the ascend direction returns one level (the deepest ancestor icon blooming back into the full current list), and vertical steps the highlight up/down the current list. These SHALL honor the **same direction-inversion settings** as the rest of the launcher. **Both axes SHALL use the launcher's odometer step distance (`launcherStepDistance`) — accumulated travel with carry — and SHALL auto-repeat at the trackpad edge** — depth has full parity with the launcher's item axis, so holding a contact at the horizontal edge auto-drills (descends/ascends) through the tree, just as holding at the vertical edge auto-scrolls the list. *(This supersedes the earlier rule that depth was a deliberate, non-auto-repeating step; the snappier, edge-accelerated depth was chosen deliberately.)* - -#### Scenario: Horizontal descends into the highlighted folder -- **WHEN** a folder is highlighted and the user travels horizontally in the descend direction past the step distance -- **THEN** the navigator descends: the prior current list collapses into an ancestor icon and the folder's contents bud in as the new current list - -#### Scenario: Horizontal ascends one level -- **WHEN** the user travels horizontally in the ascend direction -- **THEN** the deepest ancestor icon blooms back into the current list and the prior current list recedes - -#### Scenario: Vertical moves the highlight -- **WHEN** the user travels vertically in the current list -- **THEN** the highlight steps to the adjacent entry - -#### Scenario: Held horizontal edge auto-drills through the tree -- **WHEN** the user holds the controlling contact at the horizontal border in the Files band -- **THEN** depth auto-repeats with border acceleration (descending/ascending through the tree), exactly as holding at the vertical border auto-scrolls the list - -### Requirement: Files band live preview - -The Files band's preview pane SHALL show the **actual content of the highlighted entry**: a Quick Look-class content preview for a **file** (not merely its icon — with the file/app icon as a fallback when no preview is available), and a **peek of its contents** for a **folder**. The preview SHALL update as the highlight moves and SHALL load **without blocking navigation**. The preview pane SHALL NOT be a separately focusable/navigable pane (there is no horizontal crossing into it — horizontal is the depth axis). - -#### Scenario: A highlighted file previews its content -- **WHEN** the highlight rests on a file -- **THEN** the preview shows a content preview of that file (with the file/app icon as a fallback) - -#### Scenario: A highlighted folder previews a peek of its contents -- **WHEN** the highlight rests on a folder -- **THEN** the preview shows a peek of that folder's contents - -#### Scenario: The preview is not separately navigable -- **WHEN** the user travels horizontally with a file highlighted -- **THEN** horizontal is interpreted as depth (no crossing into the preview pane) - -### Requirement: Files band is pure-trackpad (no type-to-filter search) - -The Files band SHALL be navigated **entirely by trackpad** with **no keyboard input** of any kind: there SHALL be no type-to-filter search field, and the overlay panel SHALL never become key/main for the Files band. An **up** step at the top of the current list SHALL simply **clamp** (the highlight stays on the top row) — it SHALL NOT focus a search field or any other keyboard surface. This keeps the band consistent with the app's "pure trackpad, no keypresses" rule with no exception. - -#### Scenario: Up at the top of the list clamps -- **WHEN** the highlight is at the top entry and the user steps up again -- **THEN** the highlight stays on the top row and no search field or keyboard focus is engaged - -#### Scenario: The Files navigator takes no keyboard input -- **WHEN** the Files band is open at any depth -- **THEN** every interaction is a trackpad intent and the overlay panel does not become key/main - -### Requirement: Files band resolution — open, Open-With, discard - -Resolution of a Files-band selection SHALL be **lift-to-open with a defusable commit** (the navigator's reach-in-and-open intent — *not* the AI canvas's review-then-apply). **Lifting** on a highlighted entry SHALL **open** it — a file in its default application, a folder as a Finder window — on the current Space; the open SHALL be **defusable** for a brief window so a discard issued before it fires opens nothing. Adding a finger before the resolving lift — a **relative +1 finger** versus the current relaxed contact baseline — SHALL instead arm **Open-With**, so the lift presents the **relevant-apps picker** for the highlighted file, which the user navigates (vertical) and lifts to choose; choosing opens the file with that app. A fresh deliberate **four-finger horizontal swipe-away** SHALL **discard** — defusing any pending open and writing nothing — and SHALL **never terminate an already-running application**. Resolution SHALL be **one-shot**: once opened or discarded, a stray re-lift is a no-op. The captured front app SHALL remain frontmost throughout. - -#### Scenario: Lift opens the highlighted entry -- **WHEN** the user lifts on a highlighted file (or folder) -- **THEN** it opens in its default app (or as a Finder window) on the current Space - -#### Scenario: Four-finger swipe-away discards before it opens -- **WHEN** the user makes a four-finger horizontal swipe-away instead of lifting (or within the defuse window) -- **THEN** nothing opens, any pending open is defused, and no running application is terminated - -#### Scenario: Plus-one finger opens the Open-With picker -- **WHEN** the user adds a finger (one more than the current relaxed baseline) and lifts on a file -- **THEN** the relevant-apps picker opens, which the user navigates and lifts to choose the opening app - -#### Scenario: Resolution is one-shot -- **WHEN** the selection has already been opened or discarded -- **THEN** a subsequent stray re-lift does nothing - -### Requirement: Bubble-morph presentation in the Files band - -Every element that appears in the Files band — each column/list, each row's content, the preview, the ancestor icons, and the Open-With menu — SHALL **animate into presence** rather than appear abruptly: it SHALL scale up from a near-zero "droplet" while fading in, on a **soft spring**, and SHALL recede the same way on leave. Descending SHALL animate the current list **collapsing into its ancestor icon** while the child list **buds in**; ascending SHALL animate the ancestor icon **blooming back** into the full list. The single moving selection highlight SHALL remain a **continuous sliding element** (it SHALL NOT be re-created per row, to avoid scrub strobing); only entry **content** and **structural** changes use the morph. No element SHALL "pop" to full size instantly. This presentation SHALL introduce **no new haptics** beyond the existing arm tick, and SHALL respect the non-activating overlay (display-only animation; the overlay never becomes key). - -#### Scenario: Nothing pops in -- **WHEN** any column, row content, preview, ancestor icon, or menu appears -- **THEN** it scales-and-fades up from a droplet on a soft spring rather than appearing at full size instantly - -#### Scenario: Descend collapses the current list into its icon -- **WHEN** the user descends into a folder -- **THEN** the prior current list animates into its ancestor icon while the child list buds in - -#### Scenario: Ascend blooms the icon back -- **WHEN** the user ascends -- **THEN** the deepest ancestor icon animates back into a full current list while the prior list recedes - -#### Scenario: The moving highlight does not strobe -- **WHEN** the user scrubs quickly through the current list -- **THEN** the selection highlight slides continuously (it is not re-created per row) and does not flicker - diff --git a/openspec/specs/on-device-ai-runtime/spec.md b/openspec/specs/on-device-ai-runtime/spec.md deleted file mode 100644 index 8967a40..0000000 --- a/openspec/specs/on-device-ai-runtime/spec.md +++ /dev/null @@ -1,319 +0,0 @@ -# on-device-ai-runtime Specification - -## Purpose - -Define the swappable, on-device language-model runtime that backs the AI command feature: a single `LLMRuntime` abstraction over a v1 Gemma 4 (MLX-Swift) conformer, streaming and schema-validated structured output with repair/retry, vision input, model lifecycle (opt-in download, integrity-verified, lazy-loaded, resident, evictable), a capability-based model registry, cancellable generation, and a capable-hardware-only target. -## Requirements -### Requirement: Swappable model runtime abstraction -The system SHALL access all language-model functionality through a single `LLMRuntime` abstraction that exposes the runtime's capabilities (at least `text`, `vision`; later `audio`), a streaming text-generation call, and a structured-output call that returns a typed, schema-validated value. Feature code (the band, the executor, the tasks) SHALL depend only on this abstraction and SHALL NOT reference any concrete model or framework directly, so that an additional model (another Gemma 4 size, a future Gemma, Apple Foundation Models, or a cloud model) can be added later as one new conformer without changing feature code. - -#### Scenario: Feature code is model-agnostic -- **WHEN** the band executor needs a result -- **THEN** it calls the `LLMRuntime` abstraction and never a concrete model type - -#### Scenario: Adding a model is additive -- **WHEN** a new model conformer is introduced -- **THEN** it can be selected without modifying the band, executor, or task code - -### Requirement: On-device Gemma 4 via MLX-Swift -The system SHALL provide a v1 runtime conformer that runs **Gemma 4 in-process on Apple Silicon via MLX-Swift**, defaulting to the largest text+vision model the runtime is configured for (Gemma 4 31B). Generation SHALL run fully on-device (no network at inference time) and SHALL stream output tokens incrementally. - -#### Scenario: Inference is local and offline -- **WHEN** a command runs with the on-device runtime and the network is unavailable -- **THEN** generation still completes using the local model - -#### Scenario: Output streams incrementally -- **WHEN** a command is generating -- **THEN** partial tokens are delivered as they are produced, not only at completion - -### Requirement: Schema-targeted structured output with validation and repair -For structured-output calls, the runtime SHALL accept a JSON Schema, request output matching it, and **validate** the result against the schema, **repairing or retrying** within a bounded loop when the result does not conform. The runtime MAY use grammar-guided/constrained decoding as one technique but SHALL NOT depend on hard token-level caging as the sole guarantee, and SHALL preserve the model's ability to reason freely and to return a **declined / "not applicable"** result rather than being forced to emit a well-formed-but-fabricated value. The runtime SHALL NOT rely on brittle regex-only parsing of free-form text as the means of obtaining structure. - -#### Scenario: Structured result conforms to schema -- **WHEN** a structured-output call is made with a JSON Schema for a task -- **THEN** the returned value parses and validates against that schema - -#### Scenario: Non-conforming output is repaired or retried -- **WHEN** the model's first attempt does not satisfy the schema -- **THEN** the runtime repairs or retries within a bounded loop and returns a conforming value, or reports that it could not produce one - -#### Scenario: The model may decline rather than fabricate -- **WHEN** the input does not fit the requested structure (for example, it is not a meeting) -- **THEN** the runtime can return a "not applicable" / declined result instead of inventing values to satisfy the schema - -### Requirement: Vision input -The runtime conformer SHALL accept an image input (e.g. a captured screen region) alongside the text prompt for vision-capable commands, and SHALL produce a text or structured result describing/acting on that image. In v1 the **Gemma 4 (MLX-Swift) conformer SHALL actually process image input** rather than refusing it: when the selected model advertises the `vision` capability and a request carries an image, the conformer SHALL drive an image-aware generation path (consuming the captured PNG bytes) and generate a grounded result. The conformer SHALL NOT reject a vision request from a vision-capable model with `unsupportedModality(.vision)`. A text-only fast path SHALL be preserved for non-image requests so text commands do not pay the multimodal load/memory cost. - -#### Scenario: Screen region is interpreted -- **WHEN** a vision command supplies a captured screen region and a prompt such as "what is this?" -- **THEN** the runtime returns a text answer grounded in the image - -#### Scenario: The v1 Gemma conformer serves a vision request -- **WHEN** a vision-capable Gemma 4 model receives a request carrying an image -- **THEN** the conformer processes the image through its image-aware pipeline and streams a grounded result, rather than throwing `unsupportedModality(.vision)` - -#### Scenario: Text commands keep the text-only fast path -- **WHEN** a request carries no image -- **THEN** the conformer serves it on the text-only path without loading the multimodal graph - -### Requirement: Model lifecycle management -The system SHALL manage model weights via a lifecycle: weights are downloaded only after the user opts in (a multi-gigabyte, quantized QAT download), the download SHALL be resumable and **integrity-verified** before use, the model SHALL be **lazy-loaded** on first use, **kept resident** between calls to avoid repeated cold loads, **evicted** (from memory) on memory pressure or when the opt-in is turned off, and **deletable** (from disk) per model. Deleting a model SHALL remove its weights from the exact on-disk location the runtime loads from, so the model reads as not-downloaded afterwards and is not re-discovered as downloaded; a delete SHALL also evict the model if it is the one resident. The **displayed** lifecycle status SHALL reflect the **currently selected** model (each model's own on-disk/resident status), not a single global status carried over from a previously active model. While a model is loading, the system SHALL expose a loading state to the UI rather than blocking silently. - -**Loading SHALL be single-flight:** at most ONE heavy load runs at any moment. Concurrent requests for the model while a load is in flight SHALL **join** that load and receive its one result (the load cost — and its memory footprint — is paid exactly once; two full weight sets are NEVER resident from racing requests); a load failure SHALL propagate to every joined requester and clear the in-flight state so a later retry starts fresh. A request for a **different** model while a load is in flight SHALL wait for the in-flight load to settle before proceeding (never two concurrent heavy loads). - -**The runtime's GPU buffer cache SHALL be bounded:** the MLX-backed runtime SHALL set an explicit buffer-cache limit at composition so freed generation buffers return to the OS — the resident footprint tracks the model's size instead of growing without bound across turns. - -**Eviction SHALL be automatic, not only manual.** Memory-pressure eviction SHALL be genuinely wired (a real OS pressure observer, not an aspiration): on **critical** pressure the resident runtime SHALL be evicted whenever no generation turn and no load is in flight; on **warning** pressure it SHALL be evicted only when the AI system is **fully quiescent** (no turn in flight, no foreground-active session — an open chat *or voice* conversation — no parked work scheduled within the horizon). Additionally, an **idle-TTL backstop** SHALL evict the resident runtime after the AI system has been continuously quiescent for a user-configurable window (default generous; `0` SHALL disable the TTL trigger only — pressure triggers remain live). The CPU ternary lane's small resident footprint SHALL be exempt from both triggers. - -**The eviction decision SHALL be a pure policy** — evaluated with time, last-activity, pressure level, and a quiescence snapshot as explicit inputs — so every rule is deterministically testable without real weights or real OS pressure. All automatic triggers SHALL route through the same single eviction path as the manual control, and an eviction SHALL transition the lifecycle `loaded → ready` (weights remain on disk). - -**Eviction SHALL be invisible-correct:** the next request after an automatic eviction SHALL transparently lazy-load again through the single-flight load path — identical to first use after relaunch — surfacing the existing observable loading state and never a failure state, a lost turn, or a stranded scheduled advance. - -#### Scenario: No download until opt-in -- **WHEN** the AI commands opt-in is off -- **THEN** no model weights are downloaded - -#### Scenario: Deleting a model frees its weights and does not re-discover -- **WHEN** a downloaded model is deleted (per-model, or via the danger-zone clear) -- **THEN** its weights are removed from the directory the runtime loads from -- **AND** re-opening the AI surface or re-enabling the opt-in shows it as not-downloaded rather than "downloaded" - -#### Scenario: Status follows the selected model -- **WHEN** the user switches the selected model in the picker -- **THEN** the displayed status reflects that model's own on-disk/resident state, not the previously selected model's - -#### Scenario: Corrupt download is rejected -- **WHEN** a downloaded model fails its integrity check -- **THEN** it is not loaded and the user is told the download must be retried - -#### Scenario: Model stays resident between calls -- **WHEN** two commands are run in succession with the model already loaded -- **THEN** the second run does not pay a full cold-load cost - -#### Scenario: Concurrent requests share one load -- **WHEN** two AI requests race while the model is not yet resident (e.g. a background advance and a user turn during the multi-second load window) -- **THEN** the heavy load runs exactly once, both requests receive the same loaded runtime, and two full weight sets are never resident simultaneously - -#### Scenario: A failed shared load fails every joiner and permits retry -- **WHEN** the in-flight load fails while other requests have joined it -- **THEN** every joined request observes the failure (never a hang or a silent stub), and a subsequent request starts a fresh load - -#### Scenario: Loading state is observable -- **WHEN** the model is loading on first use -- **THEN** the preview surface shows a loading state - -#### Scenario: The GPU buffer cache does not grow without bound -- **WHEN** many generation turns run on the loaded model -- **THEN** the runtime's buffer cache stays within its configured limit and the app's resident footprint tracks the model size rather than creeping upward with use - -#### Scenario: Idle TTL evicts a quiescent system -- **WHEN** no turn is in flight, no session is foreground-active, no parked work is scheduled, and the state has held continuously for the configured TTL -- **THEN** the resident runtime is evicted via the single eviction path and the lifecycle reads `ready` (weights still on disk) - -#### Scenario: TTL of zero restores keep-forever -- **WHEN** the TTL setting is `0` and the system stays quiescent indefinitely -- **THEN** the TTL trigger never fires, while memory-pressure eviction remains armed - -#### Scenario: Warning pressure respects an open conversation -- **WHEN** warning-level memory pressure is reported while a session is foreground-active (but no turn is in flight) -- **THEN** the resident runtime is NOT evicted - -#### Scenario: Critical pressure evicts between turns -- **WHEN** critical memory pressure is reported with no turn and no load in flight (even if a session is foreground-active) -- **THEN** the resident runtime is evicted, and the session's next message transparently triggers a fresh single-flight load with the observable loading state - -#### Scenario: Never evict mid-turn or mid-load -- **WHEN** any automatic trigger fires while a generation turn or a load is in flight -- **THEN** the eviction is not executed, and the policy is re-evaluated on the next tick - -#### Scenario: Eviction never strands scheduled work -- **WHEN** a parked advance is scheduled within the quiescence horizon -- **THEN** the TTL and warning-pressure triggers treat the system as not quiescent, and an advance that lands after an eviction lazy-loads transparently rather than failing - -#### Scenario: The pure policy is testable without real pressure -- **WHEN** the eviction policy is evaluated in tests with faked time, activity, pressure level, and quiescence inputs -- **THEN** it returns deterministic keep/evict verdicts for every rule above, with no OS observer or resident model involved - -### Requirement: Model registry and capability-based selection -The system SHALL maintain a registry describing each known model (identifier, size, integrity hash, download source, and capability set), and SHALL select the model for a given command by its required capabilities — e.g. a vision command requires a vision-capable model. The registry SHALL make it possible to route a future audio command to an audio-capable Gemma 4 model without changing feature code. - -#### Scenario: Vision command selects a vision-capable model -- **WHEN** a command requires vision and the default model is vision-capable -- **THEN** that model is selected to serve it - -#### Scenario: Registry drives upgrades -- **WHEN** a newer model entry is added to the registry -- **THEN** it can become the selected model via configuration, without code changes in the feature - -### Requirement: Cancellable generation -Generation (both streaming and structured) SHALL be cancellable mid-flight so that discarding a result (a horizontal discard swipe in the preview) stops the model work promptly and frees its resources. - -#### Scenario: Discard cancels generation -- **WHEN** the user discards a streaming result before it completes -- **THEN** the underlying generation is cancelled and stops producing tokens - -### Requirement: Targets capable hardware only -The runtime SHALL target current, high-end Apple Silicon and use the best model the configuration specifies; it SHALL NOT provide a degraded small-model path for low-end hardware in this version. If the hardware or model cannot satisfy the feature, the system SHALL report the feature as unavailable rather than silently running a worse experience. The runtime SHALL NOT assume a **single GPU compute lane**: it MAY additionally use a **CPU compute lane** running a small ternary model **concurrently** with the GPU lane to serve short, frequent, structured work — and this CPU lane SHALL exist as a **bandwidth optimization for capable hardware, NOT as a degraded fallback for weak hardware**. The small ternary model on the CPU lane SHALL NOT be construed as the "degraded small-model path" this requirement forbids; it is an additional concurrent lane for light work on capable hardware, never a substitute for the GPU model on incapable hardware. - -#### Scenario: Unsupported configuration reports unavailable -- **WHEN** the required model cannot be run on the current machine -- **THEN** the feature reports itself unavailable instead of degrading silently - -#### Scenario: The CPU ternary lane is an optimization, not a low-end fallback -- **WHEN** the CPU compute lane is in use on capable hardware -- **THEN** it runs a small ternary model for short structured work **alongside** the GPU model, and it is never offered as a substitute for the GPU model on weak hardware - -### Requirement: Two compute lanes with a pure role-to-lane policy -The system SHALL model compute as **two physical lanes** — a **GPU lane** and a **CPU ternary lane** — and SHALL assign each unit of agent work to a lane via a **pure, total role-to-lane policy**. Heavy work SHALL route to the GPU lane: the **foreground generation** (the visible reply) and **media diffusion**. Short, frequent, structured work SHALL route to the CPU ternary lane: the **router structured turn**, **classification** decisions (such as should-park / needs-you / which-skill), **memory retrieval/index** work, and **parked-subagent** background advances. The policy SHALL be a deterministic function of the work role alone (the same role always maps to the same lane), so lane assignment is reproducible and cannot drift between call sites. The GPU lane SHALL remain the existing batched/continuous-batching runtime; this requirement adds the CPU lane beside it and does not change the GPU lane's batching behavior. - -#### Scenario: Heavy work routes to the GPU lane -- **WHEN** the work is the foreground reply or a media diffusion job -- **THEN** the role-to-lane policy assigns it to the GPU lane - -#### Scenario: Short structured work routes to the CPU ternary lane -- **WHEN** the work is a router structured turn, a classification, a memory retrieval, or a parked-subagent advance -- **THEN** the role-to-lane policy assigns it to the CPU ternary lane - -#### Scenario: The policy is deterministic and total -- **WHEN** the same work role is presented to the policy more than once -- **THEN** it always maps to the same lane, and every defined role has a lane - -### Requirement: The CPU ternary lane is a second runtime conformer, not a new protocol -The CPU ternary lane SHALL be served by **another conformer of the existing language-model runtime abstraction**, carrying a small ternary/BitNet-class model — NOT a new, separate runtime protocol. Feature code (the band, the executor, the router, the tasks) SHALL continue to depend only on the single runtime abstraction and SHALL select the CPU-lane runtime **by lane** (via the model registry's lane-tagged descriptor), never by referencing a concrete CPU runtime type. Adding the CPU lane SHALL therefore require no change to feature code beyond lane-aware selection. - -#### Scenario: The CPU lane is selected by lane, not by concrete type -- **WHEN** work is routed to the CPU ternary lane -- **THEN** the system resolves the lane's runtime through the single runtime abstraction tagged with that lane, and feature code never references a concrete CPU runtime type - -#### Scenario: Adding the CPU lane is additive to feature code -- **WHEN** the CPU ternary conformer is introduced -- **THEN** the band, executor, router, and tasks compile and run unchanged, depending only on the shared runtime abstraction - -### Requirement: The CPU ternary lane serves short bursts only, never the long reply -The CPU ternary lane SHALL serve only **short, frequent, structured** work. Because a CPU ternary decode is **slower per token** than a GPU batched decode, the **long foreground reply SHALL NEVER be routed to the CPU lane**; it SHALL always run on the GPU lane. The CPU lane's value SHALL come from running its short bursts **concurrently** with the GPU reply (not from being faster per token), justified by the measured hardware facts: GPU prefill is up to ~4× faster on the neural accelerators (heavy generation belongs on the GPU); token generation is bandwidth-bound on the ~153 GB/s unified-memory bus; and the ternary model's weights are roughly 32× smaller, so reading them per token consumes only a small fraction of that bandwidth — letting the CPU lane run without meaningfully contending with the GPU reply for the shared bus. - -#### Scenario: The long reply stays on the GPU lane -- **WHEN** the foreground reply is generated -- **THEN** it runs on the GPU lane and is never dispatched to the CPU ternary lane, even when the GPU lane is busy - -#### Scenario: Short structured bursts run concurrently on the CPU lane -- **WHEN** a router turn or a classification runs while a foreground reply streams on the GPU -- **THEN** the structured burst runs on the CPU ternary lane at the same time, rather than queuing behind the GPU decode loop - -### Requirement: Cross-lane concurrency with a residency budget that never starves the GPU reply -The system SHALL run the GPU lane and the CPU ternary lane **concurrently** under a **pure residency budget**. The budget SHALL account for the resident GPU chat weights (read once), the GPU key/value caches per stream, and the **small** ternary residency footprint (the ~32×-smaller ternary weights), and SHALL admit the ternary model to **co-reside** with the current GPU batch and key/value caches under the unified-memory budget. The arbiter SHALL bound CPU-lane concurrency on its **own** small cap and SHALL NOT borrow GPU batch slots. Two invariants SHALL hold: (1) a heavy GPU generation SHALL NEVER be made to wait on CPU-lane work; (2) CPU-lane bursts SHALL NEVER preempt or starve the foreground GPU reply. A CPU-lane unit that cannot be admitted under its own cap or the residency budget in a given step SHALL **wait** (remain runnable) rather than be treated as a failure. The residency budget and the arbiter SHALL be **pure** (free memory and time are inputs), so their decisions are deterministically testable without real GPU work. - -#### Scenario: The ternary model co-resides cheaply with the GPU batch -- **WHEN** the residency budget is computed with the GPU chat weights and key/value caches resident -- **THEN** the small ternary model is admitted to co-reside, fitting where a second full chat model would not - -#### Scenario: A heavy GPU generation is never blocked by CPU work -- **WHEN** CPU-lane bursts are active and a foreground GPU generation needs to advance -- **THEN** the GPU generation advances without waiting on the CPU-lane work - -#### Scenario: CPU bursts never starve the foreground reply -- **WHEN** many CPU-lane bursts are queued while the foreground GPU reply is streaming -- **THEN** the foreground reply continues unimpeded and the CPU bursts are bounded by their own cap - -#### Scenario: An unadmittable CPU burst waits, not fails -- **WHEN** a CPU-lane burst cannot be admitted under the CPU cap or residency budget in the current step -- **THEN** it waits for a later step and is not reported as a failure - -#### Scenario: The budget and arbiter are deterministic -- **WHEN** the residency budget and arbiter are evaluated with a given free-memory figure and timestamp -- **THEN** their decisions depend only on those inputs and the lane state, so they are reproducible in tests - -### Requirement: An additive lane-affinity hint dispatches a parked subagent to the CPU lane concurrently with a foreground GPU generation -The system SHALL carry a **lane-affinity hint** — the compute lane a runnable session prefers, derived from its work role via the role-to-lane policy — and SHALL consume it **additively**, without changing the pinned shapes of the parked-session scheduler's runnable-set request or the batched runtime's batch-step entry point. The dispatcher SHALL read each runnable session's lane affinity and route a **CPU-ternary-affined** session (such as a parked subagent) to the **CPU ternary lane** while the GPU batched runtime keeps serving the foreground and GPU-affined sessions. The net effect SHALL be that a **parked subagent advances on the CPU lane at the same time** as a foreground GPU generation, rather than waiting for a GPU batch slot. This hint SHALL be additive on the existing scheduler and batched-runtime seams; it SHALL NOT require those seams' methods to change signature. - -#### Scenario: A parked subagent runs on CPU while the foreground generates on GPU -- **WHEN** a parked subagent session is runnable and a foreground GPU generation is in flight -- **THEN** the subagent is dispatched to the CPU ternary lane and advances concurrently with the foreground GPU generation - -#### Scenario: The lane-affinity hint is additive to the existing seams -- **WHEN** the lane-affinity hint is introduced -- **THEN** the parked scheduler's runnable-set request and the batched runtime's batch-step entry point keep their existing shapes, and the hint is read alongside them rather than by changing their signatures - -#### Scenario: Affinity follows the work role -- **WHEN** a session's lane affinity is derived -- **THEN** it equals the lane the role-to-lane policy assigns to that session's work role (a parked-subagent advance maps to the CPU lane; a foreground generation maps to the GPU lane) - -### Requirement: The CPU lane is gated by the master toggle and off means one-lane behavior -The CPU ternary lane SHALL be gated by a sub-capability flag under the master full-potential toggle. When the flag is **off**, the system SHALL NOT install the CPU ternary runtime and SHALL route **all** work to the GPU lane, behaving exactly as a single-lane build — a one-lane (fleet-of-one) configuration SHALL remain valid. When the flag is on, the role-to-lane policy SHALL take effect. The cost of enabling the CPU lane — a small additional resident model and CPU heat under sustained structured bursts — SHALL be disclosed where the toggle is offered. - -#### Scenario: Off routes everything to the GPU lane -- **WHEN** the CPU-lane flag is off -- **THEN** no CPU ternary runtime is installed and every work role routes to the GPU lane, exactly as a single-lane build - -#### Scenario: On enables the two-lane policy -- **WHEN** the CPU-lane flag is on -- **THEN** the role-to-lane policy takes effect and short structured work routes to the CPU ternary lane - -#### Scenario: The CPU lane's cost is disclosed -- **WHEN** the CPU-lane toggle is offered to the user -- **THEN** its RAM and heat cost is stated alongside the capability, not hidden - -### Requirement: Reasoning (thinking) is toggle-gated and streamed as a separate channel -The runtime SHALL support an optional **reasoning** mode carried on the request, in which the model thinks before answering. When reasoning is enabled, the runtime SHALL make the model reason (for Gemma 4, via the `enable_thinking` chat-template flag — the model does NOT think by default) and SHALL stream output as **two distinct channels**, tagging each emitted token as **thinking** or **response**. Only the **response** channel SHALL be returned for in-place commit and for structured/task output; the **thinking** channel SHALL NEVER be committed to the front app nor parsed as a task action. When reasoning is **disabled** (the per-request default), the model SHALL produce response-only output and incur no thinking latency. - -#### Scenario: Reasoning streams thinking and response as separate channels -- **WHEN** a request with reasoning enabled is generated -- **THEN** the runtime streams thinking-channel and response-channel tokens distinctly, tagging each, and only the response is returned for commit - -#### Scenario: Thinking never reaches the document or a task -- **WHEN** reasoning is enabled and the result is committed in place or parsed as a task action -- **THEN** only the response text is written/parsed; the thinking is never inserted into the front app nor used as the task action - -#### Scenario: Reasoning off has no thinking latency -- **WHEN** a request has reasoning disabled -- **THEN** the model produces response-only output and does not generate a thinking block - -#### Scenario: A per-command override beats the global default -- **WHEN** a command carries an explicit reasoning override (on or off) that differs from the global default -- **THEN** the request's reasoning follows the command's override (and a command with no override follows the global default), for in-place and task commands alike - -### Requirement: Audio input is carried on the request seam and honestly refused until served -`LLMRequest` and `LLMChatRequest` SHALL carry an `audio` input (encoded audio byte payloads, defaulting to empty) alongside `images`, with a `requiresAudio` derivation, and capability selection SHALL treat a non-empty `audio` as requiring the `.audio` modality through the SAME `selectModel(requiring:)` path as vision. Until a conformer actually serves audio, every runtime — including the stub — SHALL REJECT a non-empty `audio` request with `unsupportedModality(.audio)`: the seam is statically typed and carried end-to-end, and its unimplemented half is an explicit, tested refusal, never a silently-ignored field. (This is the v4+ foundation for direct audio-in Gemma via the vendored audio tower; wiring the tower is a separate change.) - -#### Scenario: Audio requests select for the audio capability -- **WHEN** a request carries non-empty audio and model selection runs -- **THEN** only descriptors advertising `.audio` satisfy it, and `RuntimeError.unavailable` is reported when none does - -#### Scenario: A non-audio runtime refuses rather than ignores -- **WHEN** a request with non-empty audio reaches a runtime that does not serve audio (including the test stub) -- **THEN** it fails with `unsupportedModality(.audio)` — the audio bytes are never silently dropped - -#### Scenario: Empty audio changes nothing -- **WHEN** requests carry the default empty audio -- **THEN** behavior is byte-for-byte identical to before the field existed (text and vision paths unaffected) - -### Requirement: Automatic eviction never thrashes under chronic pressure -Warning-level eviction SHALL additionally require sustained idleness — no AI activity stamp for at -least the warning idle floor (5 minutes) — because a resident large model keeps the system at -sustained warning-level memory pressure as its NORMAL operating state. Critical-level eviction SHALL remain -immediate (guarded only by in-flight turn/load). The quiescence snapshot SHALL cover EVERY -conversational surface — the launcher canvas and the executor's in-flight/reviewing states, the notch -sessions, and a live voice conversation — so no surface's activity is invisible to the eviction -policy. A reload after eviction SHALL re-stamp activity, giving every reload an automatic grace -window. The net invariant: automatic eviction SHALL NOT produce an evict→reload cycle during an -active conversation on ANY surface. - -#### Scenario: Chronic warning pressure between canvas turns does not evict -- **WHEN** warning pressure is sustained, a canvas conversation is in use, and less than the idle - floor has passed since the last AI activity -- **THEN** the resident model is NOT evicted between turns - -#### Scenario: Warning pressure with genuine idleness still reclaims -- **WHEN** warning pressure is reported and no AI activity has been stamped for at least the idle - floor with every surface quiescent -- **THEN** the resident model is evicted - -#### Scenario: An executor turn is a turn in flight -- **WHEN** the launcher-canvas executor is loading or streaming a turn -- **THEN** the quiescence snapshot reports a turn in flight and no automatic trigger evicts - -#### Scenario: The gesture hot path pays one flag read -- **WHEN** touch frames stream during ordinary gestures with no agent act and no live voice phase -- **THEN** the agent-abort hook's per-frame cost is a stored-flag check — it never instantiates the - agent/voice stack and never reads settings per frame - diff --git a/openspec/specs/permissions-onboarding/spec.md b/openspec/specs/permissions-onboarding/spec.md index aad2255..71f3bdd 100644 --- a/openspec/specs/permissions-onboarding/spec.md +++ b/openspec/specs/permissions-onboarding/spec.md @@ -52,47 +52,3 @@ The system SHALL behave safely when permissions are missing: without Accessibili - **WHEN** Screen Recording is not granted - **THEN** the overlay shows app icon + title cards without thumbnails -### Requirement: Calendar permission for calendar tasks -The system SHALL detect Calendar (EventKit) authorization and SHALL request it **only when a calendar task first needs it**, never at launch and never on enabling the AI commands opt-in. If the permission is denied or restricted, the calendar task SHALL fail gracefully — surfacing that Calendar access is required and offering a deep-link to the relevant System Settings pane — and SHALL NOT block other AI commands. - -#### Scenario: Calendar permission requested lazily -- **WHEN** the user runs an "add to calendar" task for the first time -- **THEN** the Calendar permission is requested at that moment, not at launch or at opt-in - -#### Scenario: Denied calendar permission degrades gracefully -- **WHEN** Calendar permission is denied and a calendar task is confirmed -- **THEN** no event is created, the user is told access is required with a link to System Settings, and other AI commands continue to work - -### Requirement: AI command input reuses held permissions and degrades -AI command input SHALL reuse the already-granted Accessibility permission (to read/replace selected text) and Screen Recording permission (to capture a screen region for vision), requesting no new permission for these paths. When Accessibility is unavailable, selection read/replace SHALL fall back as specified by selection I/O; when Screen Recording is unavailable, screen-region (vision) commands SHALL be reported as unavailable rather than failing silently. - -#### Scenario: No new prompt for selection or vision input -- **WHEN** a selection or screen-region command runs with Accessibility and Screen Recording already granted -- **THEN** no additional permission prompt appears - -#### Scenario: Missing Screen Recording disables vision commands clearly -- **WHEN** Screen Recording is not granted and a screen-region command is fired -- **THEN** the command reports that screen capture is unavailable rather than running on no image - -### Requirement: Reminders permission for reminder tasks -The system SHALL detect Reminders (EventKit) authorization and SHALL request it **only when an add-to-reminders task first needs it**, never at launch and never on enabling the AI commands opt-in. If the permission is denied or restricted, the reminder task SHALL fail gracefully — surfacing that Reminders access is required and offering a deep-link to the relevant System Settings pane — and SHALL NOT block other AI commands. - -#### Scenario: Reminders permission requested lazily -- **WHEN** the user runs an "add to reminders" task for the first time -- **THEN** the Reminders permission is requested at that moment, not at launch or at opt-in - -#### Scenario: Denied reminders permission degrades gracefully -- **WHEN** Reminders permission is denied and a reminder task is confirmed -- **THEN** no reminder is created, the user is told access is required with a link to System Settings, and other AI commands continue to work - -### Requirement: Contacts permission for contact tasks -The system SHALL detect Contacts authorization and SHALL request it **only when a new-contact task first needs it**, never at launch and never on enabling the AI commands opt-in. If the permission is denied or restricted, the contact task SHALL fail gracefully — surfacing that Contacts access is required and offering a deep-link to the relevant System Settings pane — and SHALL NOT block other AI commands. - -#### Scenario: Contacts permission requested lazily -- **WHEN** the user runs a "new contact" task for the first time -- **THEN** the Contacts permission is requested at that moment, not at launch or at opt-in - -#### Scenario: Denied contacts permission degrades gracefully -- **WHEN** Contacts permission is denied and a contact task is confirmed -- **THEN** no contact is created, the user is told access is required with a link to System Settings, and other AI commands continue to work - diff --git a/openspec/specs/screen-region-picker/spec.md b/openspec/specs/screen-region-picker/spec.md deleted file mode 100644 index 4b54773..0000000 --- a/openspec/specs/screen-region-picker/spec.md +++ /dev/null @@ -1,50 +0,0 @@ -# screen-region-picker Specification - -## Purpose -TBD - created by archiving change add-region-capture-picker. Update Purpose after archive. -## Requirements -### Requirement: Interactive region picker surface -The system SHALL present an interactive **region-picker overlay** over the revealed desktop when a screen-region vision command is fired. The picker SHALL be cursor-driven (mouse-interactive), SHALL dim the screen and draw a live selection rectangle that follows the drag, SHALL preserve the captured front app's key/focus status (it is non-activating and SHALL never become key/main — no keyboard), and SHALL be torn down **synchronously** (ordered out and closed, never deferred behind an animation) to avoid the Space-switch ghost. It SHALL reuse the already-held Screen Recording permission and SHALL request no new permission. - -#### Scenario: Picker presents over the revealed desktop -- **WHEN** a screen-region command is fired and the launcher has been dismissed -- **THEN** a mouse-interactive picker overlay appears over the revealed desktop with a crosshair / live-rectangle selection affordance - -#### Scenario: Front app stays frontmost under the picker -- **WHEN** the picker is shown -- **THEN** the captured front app remains key/frontmost (the picker never activates or becomes key/main) - -#### Scenario: Picker tears down synchronously -- **WHEN** the picker is dismissed (by capture or by cancel) -- **THEN** it is ordered out synchronously and does not linger on a subsequent Space switch - -### Requirement: Drag selects a region; click-without-drag cancels -The picker SHALL capture the dragged rectangle as the designated region: on mouse-down it SHALL anchor the origin, on drag it SHALL track a live rectangle, and on mouse-up it SHALL commit that rectangle. A mouse-up whose rectangle is **below a small area / movement threshold** (a click without a meaningful drag) SHALL be treated as a **cancel** — defusing the picker, restoring the front app, opening no canvas, and generating nothing. There SHALL be **no keyboard cancel path** (the no-keypress rule); click-without-drag SHALL be the sole cancel gesture. The drag-versus-cancel decision SHALL be a pure, unit-testable verdict independent of AppKit. - -#### Scenario: Drag commits a region -- **WHEN** the user presses, drags out a rectangle, and releases -- **THEN** that rectangle is the designated region to capture - -#### Scenario: Click without dragging cancels -- **WHEN** the user clicks without dragging (a sub-threshold rectangle) and releases -- **THEN** the picker cancels, the front app is restored, and nothing is captured or generated - -#### Scenario: No keyboard cancel path -- **WHEN** the picker is active -- **THEN** cancellation is performed by click-without-drag and no key press is required or consumed - -### Requirement: Capture the designated region as a vision image -On commit, the system SHALL capture **only the designated rectangle** of the screen as a PNG image (via the held Screen Recording permission), excluding the app's own overlay windows and the cursor, and SHALL supply that PNG to the runtime as the command's image input. A cancelled pick SHALL yield **no image** (not a blank capture and not a full-screen fallback). When Screen Recording permission is missing at capture time, the system SHALL surface a **bounded, non-blocking** failure (clean headline, opt-in copyable details, retry) — never an app-modal `NSAlert`, never raw error text in a headline. - -#### Scenario: Designated region is captured as PNG -- **WHEN** a region is committed -- **THEN** exactly that rectangle is captured as PNG and passed to the runtime as the command's image input - -#### Scenario: The picker overlay is excluded from the capture -- **WHEN** the region is captured -- **THEN** the picker's own overlay windows are excluded so they do not appear in the captured image - -#### Scenario: Missing permission is surfaced, not crashed -- **WHEN** Screen Recording permission is absent at capture time -- **THEN** a bounded, non-blocking failure is shown (not an `NSAlert`, not raw error text) and no image is produced - diff --git a/openspec/specs/selection-io/spec.md b/openspec/specs/selection-io/spec.md deleted file mode 100644 index b192627..0000000 --- a/openspec/specs/selection-io/spec.md +++ /dev/null @@ -1,84 +0,0 @@ -# selection-io Specification - -## Purpose - -Define how AI commands read input from and write output into the captured front app using only already-held permissions: reading selected text via Accessibility without touching the clipboard, a clipboard-fallback copy that restores the prior pasteboard, screen-region capture for vision input, writing results back via Accessibility-or-paste, and graceful degradation when input cannot be obtained. -## Requirements -### Requirement: Read the front app's selected text via Accessibility -The system SHALL read the currently selected text of the app that was frontmost when the launcher opened, using Accessibility (`AXSelectedText` on the focused UI element), without modifying the clipboard. This SHALL reuse the Accessibility permission the app already holds and SHALL require no new permission. - -#### Scenario: Selected text is read without touching the clipboard -- **WHEN** text is selected in the front app and a selection-input command is fired -- **THEN** the selected text is obtained via Accessibility and the clipboard contents are unchanged - -#### Scenario: No selection yields no AX text -- **WHEN** nothing is selected in the front app -- **THEN** the Accessibility read returns no text (and the caller may fall back) - -### Requirement: Clipboard fallback with restore -When Accessibility does not expose the selection, the system SHALL fall back to synthesizing ⌘C against the captured front app, reading the resulting pasteboard text, and then **restoring the previous pasteboard contents** so the user's clipboard is left as it was. - -#### Scenario: Fallback reads via copy and restores the clipboard -- **WHEN** the AX selection read fails but text is selected, and the fallback is used -- **THEN** the text is obtained via a synthesized copy and the prior clipboard contents are restored afterward - -#### Scenario: Fallback does not clobber a password on the clipboard -- **WHEN** the clipboard held sensitive content before the fallback ran -- **THEN** that content is restored unchanged after the fallback read - -### Requirement: Screen-region capture for vision input -The system SHALL capture a **user-designated region** of the screen as an image for vision commands, using the interactive region picker, reusing the Screen Recording permission the app already holds. The captured image SHALL be the **designated rectangle only** (not the full display) and SHALL be supplied to the runtime as the command's input. A **cancelled** pick (a click without a drag) SHALL yield **no image**, so the command aborts rather than running on a blank or full-screen capture. The capture SHALL exclude the app's own overlay windows. - -#### Scenario: Designated region capture feeds the vision model -- **WHEN** a screen-region command is fired and the user drags out a region -- **THEN** that region (only) is captured as an image and passed to the runtime as input - -#### Scenario: Cancelled pick yields no image -- **WHEN** the user cancels the region pick (a click without a drag) -- **THEN** no image is produced and the command does not run on a fallback capture - -#### Scenario: Region capture reuses the held permission -- **WHEN** a region is captured -- **THEN** the already-held Screen Recording permission is used and no new permission is requested - -### Requirement: Write output back into the front app -The system SHALL deliver a command's committed result into the app that was frontmost when the launcher opened: for `replaceSelection`, by setting the selection's text via Accessibility when the element is settable, otherwise by pasting; for `pasteAtCursor`, by pasting at the insertion point. The paste path SHALL reuse the existing paste-on-fire mechanism (restore representations + synthesized ⌘V into the captured app). - -#### Scenario: Replace via Accessibility when settable -- **WHEN** the focused element exposes a settable selected-text attribute and a `replaceSelection` result is committed -- **THEN** the selection is replaced via Accessibility without a paste - -#### Scenario: Fall back to paste when not settable -- **WHEN** the element is not settable and a `replaceSelection` result is committed -- **THEN** the result is delivered by pasting into the captured front app - -### Requirement: Reuse held permissions and degrade gracefully -Selection I/O SHALL rely only on permissions the app already holds (Accessibility for read/replace, Screen Recording for region capture) and SHALL behave safely when input cannot be obtained: if neither Accessibility nor the clipboard fallback yields text for an input-requiring command, the system SHALL report a clear "no input" state and SHALL NOT run the model on empty input. - -#### Scenario: No new permission prompt for selection I/O -- **WHEN** a selection-input command runs -- **THEN** no new permission is requested (the held Accessibility permission is used) - -#### Scenario: Unobtainable input is reported, not guessed -- **WHEN** input cannot be obtained by any path for an input-requiring command -- **THEN** the system surfaces a "no input" state rather than invoking the model - -### Requirement: Read the current clipboard image as vision input -The system SHALL read the current clipboard **image** — from the **live system pasteboard** (preferring `public.png`, falling back to `public.tiff`), normalized to **PNG** bytes — and supply it to the runtime as a vision command's image input. This SHALL be symmetric with the existing live clipboard-**text** read: it reads the live pasteboard, **not** the stored clipboard history, and SHALL reuse already-held access (reading the pasteboard requires no new permission). When the pasteboard holds no image, or its image data cannot be decoded/normalized to PNG, the read SHALL yield **no image** so the caller surfaces a "no input" state rather than invoking the model on empty input. - -#### Scenario: Clipboard image is read as PNG for the vision model -- **WHEN** an image is on the clipboard and a `clipboardImage` command is fired -- **THEN** the image is read as PNG bytes and passed to the runtime as the command's image input - -#### Scenario: TIFF-only clipboard image is normalized to PNG -- **WHEN** the pasteboard holds only a `public.tiff` image and a `clipboardImage` command is fired -- **THEN** the image is normalized to PNG bytes before being supplied to the runtime - -#### Scenario: No clipboard image yields no input -- **WHEN** a `clipboardImage` command is fired and the clipboard holds no image (or undecodable image data) -- **THEN** no image is produced and the caller surfaces a "no input" state (the model is not invoked) - -#### Scenario: No new permission for the clipboard-image read -- **WHEN** a `clipboardImage` command runs -- **THEN** no new permission is requested (reading the pasteboard uses already-held access) - diff --git a/openspec/specs/tunable-settings/spec.md b/openspec/specs/tunable-settings/spec.md index 3ff0a90..801e6c8 100644 --- a/openspec/specs/tunable-settings/spec.md +++ b/openspec/specs/tunable-settings/spec.md @@ -163,40 +163,6 @@ The settings SHALL expose a "Keep clipboard history" opt-in that defaults to OFF - **WHEN** settings saved before this feature are loaded - **THEN** they decode successfully with the opt-in OFF and no clipboard history, and existing settings are not reset -### Requirement: AI commands opt-in -The settings SHALL expose an "AI commands" opt-in that defaults to OFF and gates both the AI command band and the on-device model (download and residency). Unlike the Space-row and launcher opt-ins, this opt-in SHALL NOT relocate any native gesture or require a re-login; unlike the clipboard opt-in, enabling it DOES initiate a one-time multi-gigabyte model download (and a calendar task will later request the Calendar permission at first use). Settings saved before this feature SHALL load unchanged with the opt-in OFF, no model downloaded, and no commands. - -#### Scenario: Opt-in defaults off and gates the feature -- **WHEN** the app loads with no prior AI settings -- **THEN** the AI commands opt-in is OFF, no model is downloaded, and no AI command band appears - -#### Scenario: Enabling needs no re-login or native-gesture change -- **WHEN** the user turns the opt-in on -- **THEN** the band and model become available without a re-login or any native-gesture relocation (a model download begins) - -#### Scenario: Older settings load with the feature off -- **WHEN** settings saved before this feature are loaded -- **THEN** they decode successfully with the opt-in OFF and no AI data, and existing settings are not reset - -### Requirement: AI model management settings -With the AI commands opt-in on, the settings SHALL let the user manage the on-device model: choose which Gemma 4 model is selected, see the **selected** model's download status and size, trigger or retry the download, evict the resident model from memory, and **delete the selected model's weights from disk**. The displayed status SHALL track the selected model (switching the picker SHALL refresh it). These controls SHALL persist their state across launches and apply immediately. - -#### Scenario: Download status is visible -- **WHEN** the user opens settings with the opt-in on and a model downloading -- **THEN** the settings show the model identity, size, and download progress/status - -#### Scenario: Status tracks the selected model -- **WHEN** the user switches the model picker to a different model -- **THEN** the status row refreshes to that model's own download/loaded state - -#### Scenario: Delete the selected model -- **WHEN** the user deletes the selected, downloaded model -- **THEN** its weights are removed from disk and the row shows it as not-downloaded (re-downloadable) - -#### Scenario: Evict frees memory immediately -- **WHEN** the user chooses to evict the resident model -- **THEN** the model is unloaded from memory and the next command reloads it on demand - ### Requirement: Device-link opt-in Settings SHALL expose an `enableDeviceLink` opt-in (default OFF) that gates the device-link receive/send service. Like the clipboard-history opt-in, it relocates no native gesture, needs no re-login, and has no `is…Effective` gate — it takes effect immediately when toggled. It SHALL persist across launches, and settings written before it existed SHALL load with it OFF. @@ -208,36 +174,6 @@ Settings SHALL expose an `enableDeviceLink` opt-in (default OFF) that gates the - **WHEN** settings written before this opt-in existed are loaded - **THEN** `enableDeviceLink` reads as false (no key present) -### Requirement: Persisted Files action-menu and lift settings - -The app SHALL persist the Files-band action configuration and SHALL default it to this change's grammar. The persisted settings SHALL include: - -- the **per-type action-menu item lists** (file and folder), each an ordered list drawn from the action catalog — defaulting to **file:** Copy as path · Copy · Paste · Open in ▸ and **folder:** Copy as path · Copy · Paste · ‹terminals› · Open in ▸; -- the **Files lift action** — defaulting to **deliver** (with the menu excursion defaulting to the `+1`-finger lift and discard to the four-finger horizontal), stored as part of the Files gesture-binding vocabulary; -- the **curated terminals/editors** allow-list — defaulting to the auto-detected installed set being enabled. - -These settings SHALL be included in the app's **reset-to-defaults** semantics and SHALL load to the defaults above when absent or unreadable. - -#### Scenario: Defaults reproduce the specified grammar - -- **WHEN** the user has never customized the Files action settings -- **THEN** the file and folder menus, the lift action (deliver), and the enabled terminals are exactly the defaults above - -#### Scenario: Customizations persist across launches - -- **WHEN** the user changes a menu list, the lift action, or the enabled terminals and relaunches -- **THEN** the changes are restored from persistence - -#### Scenario: Reset restores defaults - -- **WHEN** the user resets settings to defaults -- **THEN** the Files action menus, lift action, and terminal allow-list return to the specified defaults - -#### Scenario: Missing or unreadable settings fall back to defaults - -- **WHEN** the persisted Files action settings are absent or cannot be decoded -- **THEN** the app loads the specified defaults without error - ### Requirement: Include-minimized-windows opt-in The system SHALL expose an "include minimized windows in the switcher" opt-in, **off by default**, that makes minimized windows appear in the three-finger switcher and the ⌘-Tab reel — each flagged and badged as minimized — with selection **un-minimizing the window in place** and raising it. It SHALL be independent of the include-non-standard-windows setting (either may be on without the other). It SHALL persist across launches, take effect on the next gesture without a restart, and appear in the Settings UI. "Reset to defaults" SHALL restore it to off (it has no system side effect, permission, or download to preserve). diff --git a/openspec/specs/voice-conversation/spec.md b/openspec/specs/voice-conversation/spec.md deleted file mode 100644 index a2b7558..0000000 --- a/openspec/specs/voice-conversation/spec.md +++ /dev/null @@ -1,88 +0,0 @@ -# voice-conversation Specification - -## Purpose -TBD - created by archiving change add-voice-computer-use-agent. Update Purpose after archive. -## Requirements -### Requirement: Push-to-talk voice input, never always-listening -The system SHALL capture voice ONLY while the push-to-talk trigger is engaged (the configurable hold-key gesture, default Right Option, or the press-and-hold mic button). The microphone SHALL open on engagement and close on release — there SHALL be NO wake word, NO always-on capture, and NO voice-activity-triggered listening. The feature SHALL be a separate opt-in (default OFF) under the AI master gate, and microphone authorization SHALL be requested lazily on the FIRST actual engagement (never at enable time). A denied authorization SHALL surface as a bounded, non-blocking failure card with a System Settings link — never an app-modal alert, never a crash. - -**The hold-key trigger SHALL be double-tap-then-hold** (the macOS dictation idiom): a first click of the key (down and up, each within the tap window, ~0.3 s), a second press within the gap window (~0.3 s), HELD past the dwell (~0.15 s) — capture begins at dwell-elapsed and release sends. A double-tap WITHOUT the dwell SHALL be a complete no-op. A LONG SINGLE hold SHALL be a complete no-op (plain modifier use — special-character typing is untouched). Any OTHER key going down at any pre-capture stage SHALL cancel the gesture — a typing chord can never trigger voice — and the capture stack SHALL NOT be touched, nor the microphone opened, until dwell-elapsed. The observation SHALL be passive (no event consumed or delayed). Barge-in while the assistant thinks or speaks SHALL use the SAME gesture. - -#### Scenario: Double-tap-and-hold talks; release sends -- **WHEN** the user clicks the trigger key, presses it again within the gap, holds past the dwell, speaks, and releases -- **THEN** capture runs only from dwell-elapsed to release, the transcript is finalized on release, and the finalized text is sent as the agent turn - -#### Scenario: No capture outside the gesture -- **WHEN** the voice feature is enabled but the gesture has not completed its dwell -- **THEN** no audio session is active and no audio is read - -#### Scenario: A typing chord never triggers voice -- **WHEN** the user types any chord using the trigger modifier (⌥⌫, ⌥→, option-symbols) at any speed -- **THEN** the gesture cancels, the capture stack is never started, the mic never opens, and the chord reaches the target app unmodified - -#### Scenario: A long single hold is plain modifier use -- **WHEN** the user presses and holds the trigger key once (no second tap) -- **THEN** nothing happens — no capture, no timers left pending after release - -#### Scenario: A bare double-tap is a no-op -- **WHEN** the user double-taps the trigger key without holding the second press past the dwell -- **THEN** nothing happens - -#### Scenario: Denied mic permission is a clean, recoverable failure -- **WHEN** the user denies microphone authorization on first use -- **THEN** a bounded non-blocking card explains it with a Settings link, the feature stays off-path, and nothing crashes or blocks - -### Requirement: On-device transcription behind a seam, macOS 26-gated -Transcription SHALL run fully on-device via the injected `SpeechTranscribing` seam. The real conformer SHALL wrap Apple `SpeechAnalyzer`/`SpeechTranscriber` and SHALL be gated `@available(macOS 26)`; on older macOS the factory SHALL resolve to nil and the voice feature SHALL read as unavailable with a clean "requires macOS 26" reason (the platform floor does not rise). Core SHALL ship a scripted stub conformer so every voice behavior verifies under `swift test` with no Speech framework involvement. - -#### Scenario: Older macOS reports unavailable, never crashes -- **WHEN** the voice opt-in is viewed on macOS earlier than 26 -- **THEN** the toggle is disabled with a "requires macOS 26" disclosure and no Speech API is touched - -#### Scenario: Voice logic verifies with the stub -- **WHEN** the Core test suite runs -- **THEN** the full turn lifecycle (capture → transcript → turn → spoken reply → barge-in) executes against the stub transcriber/synthesizer deterministically - -### Requirement: Sentence-chunked spoken replies from the streaming turn -The spoken reply SHALL be produced by chunking the existing `.response`-channel token stream at sentence boundaries and feeding each closed chunk to the `SpeechSynthesizing` seam as it closes — the first sentence SHALL be speakable while the remainder is still generating. Thinking-channel tokens SHALL NEVER be spoken. Code blocks SHALL be summarized in speech (e.g. "code block, N lines"), not read symbol-by-symbol. The visible canvas/notch transcript SHALL remain the full, unmodified text. - -#### Scenario: First sentence speaks during generation -- **WHEN** a voice turn's reply streams -- **THEN** the first completed sentence begins speaking before the stream finishes, and remaining sentences queue in order - -#### Scenario: Thinking is never spoken -- **WHEN** a reply streams with reasoning enabled -- **THEN** only `.response`-channel text reaches the synthesizer - -### Requirement: Barge-in stops speech and cancels the turn as a discard -While the agent is thinking or speaking, a new push-to-talk press SHALL barge in: text-to-speech output stops, any in-flight generation is cancelled through the existing cancellable-generation path (a DISCARD, never a failure state), and listening begins immediately — a fluent correction, not an error. Tokens arriving after a barge-in SHALL be dropped, not spoken. A human trackpad touch during agent action SHALL abort identically (the computer-use arbitration requirement). - -#### Scenario: Barge-in mid-reply -- **WHEN** the user presses push-to-talk while the reply is being spoken -- **THEN** speech stops immediately, the generation is cancelled as a discard, late tokens are not spoken, and the mic is live - -#### Scenario: The voice turn lifecycle is pure and tested -- **WHEN** the voice-turn state machine is driven in tests with fake timestamps through idle→listening→transcribing→thinking→speaking and a barge-in at each interruptible state -- **THEN** every transition and emitted effect matches the specified lifecycle deterministically - -### Requirement: A voice session is a foreground conversational surface -An open voice conversation SHALL count as `foregroundSessionActive` in the model-eviction quiescence snapshot (the OR-shaped flag landed by `model-idle-ttl-and-memory-pressure`), so idle-TTL and warning-pressure eviction never unload the model between spoken turns of a live dialogue. - -#### Scenario: No eviction mid-dialogue -- **WHEN** a voice conversation is open and idle between spoken turns past the idle TTL -- **THEN** the TTL trigger does not evict the resident model - -### Requirement: Speak-last-response works without the microphone -A "speak the last response" command SHALL exist independently of the voice opt-in's mic capture: it resolves the target window through the switcher's own enumeration, reads the window text via the accessibility reader, extracts the relevant tail (e.g. the assistant's final reply in a terminal transcript), and speaks it through the synthesizer seam. It SHALL require no microphone, no new permission, and no open conversation. - -#### Scenario: Read Claude's last response aloud -- **WHEN** the user fires speak-last-response with a terminal window frontmost showing an assistant transcript -- **THEN** the window's text is read via AX, the final assistant reply is extracted, and it is spoken — with a bounded, non-blocking failure card if the window cannot be read - -### Requirement: Voice errors join the single error taxonomy -Voice failures SHALL be classified into a `VoiceError` taxonomy (`LocalizedError`, parallel to `FileActionError`) — mic denied, speech unavailable, OS too old, capture failure — mapped at the boundary where AVFoundation/Speech errors cross into app code, routed through the single `AIError.message(for:)` translator, and surfaced bounded + non-blocking. Raw vendor error text SHALL appear only in opt-in details, never in a headline. - -#### Scenario: A capture failure surfaces cleanly -- **WHEN** the audio engine fails to start mid-press -- **THEN** the user sees a clean headline card (details behind a disclosure), the turn is not sent, and the app remains responsive - diff --git a/scripts/build-app.sh b/scripts/build-app.sh index bbce2e3..848b0f7 100755 --- a/scripts/build-app.sh +++ b/scripts/build-app.sh @@ -34,17 +34,13 @@ else echo " Run ./scripts/make-dev-cert.sh once to fix this." fi -# The app transitively links MLX (Gemma 4 runtime), whose Metal shaders ONLY compile under -# xcodebuild — `swift build` cannot build the app target anymore. We still keep STABLE signing: -# xcodebuild builds WITHOUT signing (CODE_SIGNING_ALLOWED=NO), then the codesign block below applies -# the stable "ThreeFingerSwitcher Dev" identity, so TCC grants survive across rebuilds. +# STABLE signing: xcodebuild builds WITHOUT signing (CODE_SIGNING_ALLOWED=NO), then the codesign +# block below applies the stable "ThreeFingerSwitcher Dev" identity, so TCC grants survive across +# rebuilds. echo "▸ xcodebuild build -scheme $PRODUCT -configuration $CONFIG (no xcodebuild signing; stable codesign below)" # `-onlyUsePackageVersionsFromResolvedFile`: build EXACTLY the versions in the committed -# Package.resolved and never re-resolve. Some dependencies (gemma-4-swift-mlx, and its transitive -# mlx-swift-lm) track `branch: main`, whose upstream HEAD drifts and has shipped commits that fail to -# compile — without this flag a clean CI checkout re-resolves those branches to a broken newer HEAD -# and fails the release build even though the lockfile pins known-good commits. The lockfile is the -# source of truth; bump deliberately via `swift package update` + a verified `xcodebuild`. +# Package.resolved and never re-resolve, so a clean CI checkout can never drift onto a broken +# upstream HEAD. The lockfile is the source of truth; bump deliberately via `swift package update`. xcodebuild build \ -scheme "$PRODUCT" \ -destination 'platform=macOS' \ @@ -90,11 +86,10 @@ for dylib in "$BIN_PATH"/*.dylib; do cp "$dylib" "$APP/Contents/Frameworks/" done -# SwiftPM resource bundles (e.g. mlx-swift_Cmlx.bundle, which holds default.metallib — MLX's compiled -# Metal shaders). xcodebuild emits these into the products dir; they are NOT baked into the binary. -# Each package finds its bundle via `Bundle.module`, whose first candidate is `Bundle.main.resourceURL` -# = Contents/Resources. WITHOUT this copy the app is SIGKILLed with no crash report the instant MLX -# touches the GPU — i.e. "it disappears at 100%". Copy every produced *.bundle into Resources. +# SwiftPM resource bundles: xcodebuild emits these into the products dir; they are NOT baked into +# the binary. Each package finds its bundle via `Bundle.module`, whose first candidate is +# `Bundle.main.resourceURL` = Contents/Resources — so copy every produced *.bundle into Resources +# (a no-op when no dependency ships resources). for bundle in "$BIN_PATH"/*.bundle; do [ -e "$bundle" ] || continue echo "▸ bundling resource: $(basename "$bundle")" From 988937a850d23d699ce6f15e7c5e1c75b8266869 Mon Sep 17 00:00:00 2001 From: Amitay Keisar Date: Thu, 20 Aug 2026 01:06:37 +0300 Subject: [PATCH 2/5] =?UTF-8?q?feat!:=20remove=20the=20device=20link=20(iP?= =?UTF-8?q?hone=E2=86=94Mac=20pairing=20+=20clipboard/file=20bridge)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The great cleanup, part two. remove-local-ai kept the device link; on reflection it goes too — remote device syncing is the same drift from the app's intent (switcher · launcher · clipboard history) as the AI stack was, and its iOS companion never shipped. The v1 branch / v1.0.0 release preserve the full-featured app. Removed: - Sources/ThreeFingerSwitcher/DeviceLink/ (service, NW/sealing transports, handshake, ReceiveHUD, the QR-pairing suite) - The vendored DeviceLinkKit package + its Package.swift products - Clipboard Link{Inbound,Outbound}Adapter + the coordinator's receive/auto-paste/send-latest paths - The Hub Devices page (.devices destination, rail button, seams) - The enableDeviceLink setting; NSLocalNetworkUsageDescription + NSBonjourServices from Info.plist - ClipboardOrigin/.peer provenance + the band's ProvenanceChip (legacy indexes with an origin key still decode — keys ignored) - ClipboardMonitor.suppressSelfWrite (only caller was the receive path) - 9 spec capabilities (device-link-* + mac-qr-pairing); provenance requirements in clipboard-history replaced with legacy-compat swift build green; 777 tests pass (was 805 — deleted suites were device-link-only). Change folder: openspec/changes/remove-device-link. Story-origin: the second cut of the great cleanup — realizing the device link was the same scope drift the AI stack was, just wearing a hardware accessory. Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 2 +- DeviceLinkKit/Package.swift | 49 ---- .../Sources/DeviceLinkMirror/MovedItem.swift | 64 ----- .../DeviceLinkMirror/MovedItemStore.swift | 126 ---------- .../DeviceLinkPairing/LinkSession.swift | 211 ----------------- .../DeviceLinkPairing/LocalAddresses.swift | 66 ------ .../DeviceLinkPairing/PairingCode.swift | 19 -- .../DeviceLinkPairing/PairingExchange.swift | 101 -------- .../DeviceLinkPairing/PairingHandshake.swift | 44 ---- .../DeviceLinkPairing/PairingMessage.swift | 13 -- .../DeviceLinkPairing/PairingQRPayload.swift | 94 -------- .../DeviceLinkPairing/SealedRecord.swift | 118 ---------- .../Sources/DeviceLinkProtocol/Frame.swift | 89 ------- .../FrameStreamEncoder.swift | 51 ---- .../DeviceLinkProtocol/InboundAssembler.swift | 93 -------- .../DeviceLinkProtocol/LinkCodec.swift | 220 ------------------ .../Sources/DeviceLinkProtocol/LinkItem.swift | 41 ---- .../LinkProtocolError.swift | 44 ---- .../Sources/DeviceLinkProtocol/LinkPump.swift | 59 ----- .../Sources/DeviceLinkProtocol/LinkUTI.swift | 14 -- .../DeviceLinkProtocol/ProtocolVersion.swift | 46 ---- .../MovedItemStoreTests.swift | 93 -------- .../DeviceLinkPairingTests/PairingTests.swift | 58 ----- .../QRPairingTests.swift | 136 ----------- .../FrameStreamEncoderTests.swift | 74 ------ .../InboundAssemblerTests.swift | 125 ---------- .../LinkCodecTests.swift | 130 ----------- .../LinkPumpTests.swift | 82 ------- .../ProtocolVersionAndErrorTests.swift | 35 --- Package.swift | 19 +- README.md | 6 +- Resources/Info.plist | 10 - .../App/AppCoordinator.swift | 100 -------- .../Clipboard/ClipboardEntry.swift | 34 +-- .../Clipboard/ClipboardMonitor.swift | 30 +-- .../Clipboard/ClipboardStore.swift | 2 +- .../Clipboard/LinkInboundAdapter.swift | 127 ---------- .../Clipboard/LinkOutboundAdapter.swift | 61 ----- .../DeviceLink/DeviceLinkService.swift | 169 -------------- .../DeviceLink/LinkByteTransport.swift | 15 -- .../DeviceLink/LinkConnection.swift | 112 --------- .../DeviceLink/LinkHandshake.swift | 118 ---------- .../DeviceLink/NWByteTransport.swift | 53 ----- .../DeviceLink/Pairing/MacLocalIdentity.swift | 55 ----- .../Pairing/MacPairingChannel.swift | 135 ----------- .../Pairing/MacPairingCoordinator.swift | 98 -------- .../DeviceLink/Pairing/PairedDevice.swift | 17 -- .../Pairing/PairedDeviceStore.swift | 68 ------ .../DeviceLink/Pairing/QRImage.swift | 32 --- .../DeviceLink/ReceiveHUD.swift | 178 -------------- .../DeviceLink/SealingByteTransport.swift | 87 ------- .../Hub/HubDevicesPage.swift | 113 --------- Sources/ThreeFingerSwitcher/Hub/HubView.swift | 12 +- .../Launcher/LaunchService.swift | 3 +- .../Overlay/ClipboardBandView.swift | 27 --- .../Settings/AppSettings.swift | 9 - .../AppSettingsTests.swift | 11 - .../ClipboardMonitorTests.swift | 68 ------ .../LinkConnectionTests.swift | 94 -------- .../LinkInboundAdapterTests.swift | 144 ------------ .../LinkOutboundAdapterTests.swift | 96 -------- .../PairingHandshakeTests.swift | 28 --- .../QRImageTests.swift | 29 --- openspec/changes/remove-device-link/design.md | 19 ++ .../changes/remove-device-link/proposal.md | 28 +++ openspec/changes/remove-device-link/tasks.md | 17 ++ openspec/specs/clipboard-history/spec.md | 25 +- openspec/specs/device-link-hub/spec.md | 34 --- openspec/specs/device-link-inbound/spec.md | 57 ----- .../specs/device-link-mirror-store/spec.md | 38 --- openspec/specs/device-link-outbound/spec.md | 45 ---- openspec/specs/device-link-pairing/spec.md | 46 ---- openspec/specs/device-link-protocol/spec.md | 162 ------------- openspec/specs/device-link-qr-pairing/spec.md | 35 --- openspec/specs/device-link-transport/spec.md | 46 ---- openspec/specs/mac-qr-pairing/spec.md | 27 --- openspec/specs/tunable-settings/spec.md | 11 - 77 files changed, 84 insertions(+), 4963 deletions(-) delete mode 100644 DeviceLinkKit/Package.swift delete mode 100644 DeviceLinkKit/Sources/DeviceLinkMirror/MovedItem.swift delete mode 100644 DeviceLinkKit/Sources/DeviceLinkMirror/MovedItemStore.swift delete mode 100644 DeviceLinkKit/Sources/DeviceLinkPairing/LinkSession.swift delete mode 100644 DeviceLinkKit/Sources/DeviceLinkPairing/LocalAddresses.swift delete mode 100644 DeviceLinkKit/Sources/DeviceLinkPairing/PairingCode.swift delete mode 100644 DeviceLinkKit/Sources/DeviceLinkPairing/PairingExchange.swift delete mode 100644 DeviceLinkKit/Sources/DeviceLinkPairing/PairingHandshake.swift delete mode 100644 DeviceLinkKit/Sources/DeviceLinkPairing/PairingMessage.swift delete mode 100644 DeviceLinkKit/Sources/DeviceLinkPairing/PairingQRPayload.swift delete mode 100644 DeviceLinkKit/Sources/DeviceLinkPairing/SealedRecord.swift delete mode 100644 DeviceLinkKit/Sources/DeviceLinkProtocol/Frame.swift delete mode 100644 DeviceLinkKit/Sources/DeviceLinkProtocol/FrameStreamEncoder.swift delete mode 100644 DeviceLinkKit/Sources/DeviceLinkProtocol/InboundAssembler.swift delete mode 100644 DeviceLinkKit/Sources/DeviceLinkProtocol/LinkCodec.swift delete mode 100644 DeviceLinkKit/Sources/DeviceLinkProtocol/LinkItem.swift delete mode 100644 DeviceLinkKit/Sources/DeviceLinkProtocol/LinkProtocolError.swift delete mode 100644 DeviceLinkKit/Sources/DeviceLinkProtocol/LinkPump.swift delete mode 100644 DeviceLinkKit/Sources/DeviceLinkProtocol/LinkUTI.swift delete mode 100644 DeviceLinkKit/Sources/DeviceLinkProtocol/ProtocolVersion.swift delete mode 100644 DeviceLinkKit/Tests/DeviceLinkMirrorTests/MovedItemStoreTests.swift delete mode 100644 DeviceLinkKit/Tests/DeviceLinkPairingTests/PairingTests.swift delete mode 100644 DeviceLinkKit/Tests/DeviceLinkPairingTests/QRPairingTests.swift delete mode 100644 DeviceLinkKit/Tests/DeviceLinkProtocolTests/FrameStreamEncoderTests.swift delete mode 100644 DeviceLinkKit/Tests/DeviceLinkProtocolTests/InboundAssemblerTests.swift delete mode 100644 DeviceLinkKit/Tests/DeviceLinkProtocolTests/LinkCodecTests.swift delete mode 100644 DeviceLinkKit/Tests/DeviceLinkProtocolTests/LinkPumpTests.swift delete mode 100644 DeviceLinkKit/Tests/DeviceLinkProtocolTests/ProtocolVersionAndErrorTests.swift delete mode 100644 Sources/ThreeFingerSwitcher/Clipboard/LinkInboundAdapter.swift delete mode 100644 Sources/ThreeFingerSwitcher/Clipboard/LinkOutboundAdapter.swift delete mode 100644 Sources/ThreeFingerSwitcher/DeviceLink/DeviceLinkService.swift delete mode 100644 Sources/ThreeFingerSwitcher/DeviceLink/LinkByteTransport.swift delete mode 100644 Sources/ThreeFingerSwitcher/DeviceLink/LinkConnection.swift delete mode 100644 Sources/ThreeFingerSwitcher/DeviceLink/LinkHandshake.swift delete mode 100644 Sources/ThreeFingerSwitcher/DeviceLink/NWByteTransport.swift delete mode 100644 Sources/ThreeFingerSwitcher/DeviceLink/Pairing/MacLocalIdentity.swift delete mode 100644 Sources/ThreeFingerSwitcher/DeviceLink/Pairing/MacPairingChannel.swift delete mode 100644 Sources/ThreeFingerSwitcher/DeviceLink/Pairing/MacPairingCoordinator.swift delete mode 100644 Sources/ThreeFingerSwitcher/DeviceLink/Pairing/PairedDevice.swift delete mode 100644 Sources/ThreeFingerSwitcher/DeviceLink/Pairing/PairedDeviceStore.swift delete mode 100644 Sources/ThreeFingerSwitcher/DeviceLink/Pairing/QRImage.swift delete mode 100644 Sources/ThreeFingerSwitcher/DeviceLink/ReceiveHUD.swift delete mode 100644 Sources/ThreeFingerSwitcher/DeviceLink/SealingByteTransport.swift delete mode 100644 Sources/ThreeFingerSwitcher/Hub/HubDevicesPage.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/LinkConnectionTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/LinkInboundAdapterTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/LinkOutboundAdapterTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/PairingHandshakeTests.swift delete mode 100644 Tests/ThreeFingerSwitcherTests/QRImageTests.swift create mode 100644 openspec/changes/remove-device-link/design.md create mode 100644 openspec/changes/remove-device-link/proposal.md create mode 100644 openspec/changes/remove-device-link/tasks.md delete mode 100644 openspec/specs/device-link-hub/spec.md delete mode 100644 openspec/specs/device-link-inbound/spec.md delete mode 100644 openspec/specs/device-link-mirror-store/spec.md delete mode 100644 openspec/specs/device-link-outbound/spec.md delete mode 100644 openspec/specs/device-link-pairing/spec.md delete mode 100644 openspec/specs/device-link-protocol/spec.md delete mode 100644 openspec/specs/device-link-qr-pairing/spec.md delete mode 100644 openspec/specs/device-link-transport/spec.md delete mode 100644 openspec/specs/mac-qr-pairing/spec.md diff --git a/CLAUDE.md b/CLAUDE.md index a6209d0..41870aa 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -2,7 +2,7 @@ Orientation lives in **`README.md`** (it's written for an agent: Job A = install/run, Job B = work on the code). This file is the short list of things that are easy to get wrong. Read the **Building & signing** rule before you build anything. -> **The great cleanup (`remove-local-ai`):** the on-device AI stack (Gemma/MLX, the AI command band + canvas, background agents, the notch timeline, voice/computer-use) and the Files band were **removed** — the app is refocused on the **switcher, the launcher, and clipboard history**. The `v1` branch / `v1.0.0` release preserve the full-featured app; don't reintroduce those features casually. +> **The great cleanup (`remove-local-ai`, `remove-device-link`):** the on-device AI stack (Gemma/MLX, the AI command band + canvas, background agents, the notch timeline, voice/computer-use), the Files band, and the iPhone↔Mac device link (QR pairing, the local-network clipboard/file bridge, the vendored `DeviceLinkKit` package, the Hub Devices page) were **removed** — the app is refocused on the **switcher, the launcher, and clipboard history**. The `v1` branch / `v1.0.0` release preserve the full-featured app; don't reintroduce those features casually. ## Building & signing — read this first diff --git a/DeviceLinkKit/Package.swift b/DeviceLinkKit/Package.swift deleted file mode 100644 index d54eddf..0000000 --- a/DeviceLinkKit/Package.swift +++ /dev/null @@ -1,49 +0,0 @@ -// swift-tools-version: 6.2 -import PackageDescription - -// The shared, cross-platform device-link packages: the wire contract, the iOS "moved items" store, and -// the pairing crypto. ZERO external dependencies (no MLX/AppKit/UIKit), declared for BOTH macOS and iOS -// so the macOS app and the iOS companion app can each consume the products. Verified under `swift test`. -let package = Package( - name: "DeviceLinkKit", - platforms: [ - .macOS(.v13), - .iOS(.v15) - ], - products: [ - .library(name: "DeviceLinkProtocol", targets: ["DeviceLinkProtocol"]), - .library(name: "DeviceLinkMirror", targets: ["DeviceLinkMirror"]), - .library(name: "DeviceLinkPairing", targets: ["DeviceLinkPairing"]) - ], - targets: [ - .target( - name: "DeviceLinkProtocol", - swiftSettings: [.swiftLanguageMode(.v6)] - ), - .target( - name: "DeviceLinkMirror", - dependencies: ["DeviceLinkProtocol"], - swiftSettings: [.swiftLanguageMode(.v6)] - ), - .target( - name: "DeviceLinkPairing", - dependencies: ["DeviceLinkProtocol"], // for DeviceIdentity (QR payload + pairing exchange) - swiftSettings: [.swiftLanguageMode(.v6)] - ), - .testTarget( - name: "DeviceLinkProtocolTests", - dependencies: ["DeviceLinkProtocol"], - swiftSettings: [.swiftLanguageMode(.v5)] - ), - .testTarget( - name: "DeviceLinkMirrorTests", - dependencies: ["DeviceLinkMirror"], - swiftSettings: [.swiftLanguageMode(.v5)] - ), - .testTarget( - name: "DeviceLinkPairingTests", - dependencies: ["DeviceLinkPairing", "DeviceLinkProtocol"], - swiftSettings: [.swiftLanguageMode(.v5)] - ) - ] -) diff --git a/DeviceLinkKit/Sources/DeviceLinkMirror/MovedItem.swift b/DeviceLinkKit/Sources/DeviceLinkMirror/MovedItem.swift deleted file mode 100644 index 8915c9b..0000000 --- a/DeviceLinkKit/Sources/DeviceLinkMirror/MovedItem.swift +++ /dev/null @@ -1,64 +0,0 @@ -import Foundation -import DeviceLinkProtocol - -/// Which way a thing moved between the iPhone and the Mac. -public enum MoveDirection: String, Codable, Sendable, Equatable { - case sent // the phone sent it to the Mac - case received // the phone received it from the Mac -} - -/// One record in the iPhone app's "what moved" list. Carries enough to show the row and to re-share a -/// received item (its representation bytes). Pure value type; no UIKit. -public struct MovedItem: Codable, Equatable, Identifiable, Sendable { - public var id: UUID - public var direction: MoveDirection - public var kind: LinkItemKind - /// A single-line label for the row. - public var title: String - /// The other device's name, when known. - public var peerName: String? - public var movedAt: Date - /// Materialized representation bytes keyed by UTI (loaded from blobs by the store). - public var representations: [String: Data] - - public init(id: UUID, direction: MoveDirection, kind: LinkItemKind, title: String, - peerName: String?, movedAt: Date, representations: [String: Data]) { - self.id = id - self.direction = direction - self.kind = kind - self.title = title - self.peerName = peerName - self.movedAt = movedAt - self.representations = representations - } - - /// Build a moved-item record from a wire `LinkItem`. - public static func from(_ item: LinkItem, direction: MoveDirection, at date: Date) -> MovedItem { - MovedItem(id: item.messageID, direction: direction, kind: item.kind, - title: title(for: item), peerName: item.origin?.name, movedAt: date, - representations: item.representations) - } - - /// A single-line title: first line of text/url, the file's suggested name, or a fixed label. - static func title(for item: LinkItem) -> String { - switch item.kind { - case .text, .url, .richText: - let data = item.representations[LinkUTI.plainText] - ?? item.representations[LinkUTI.url] - ?? item.representations.values.first - ?? Data() - return firstLine(String(decoding: data, as: UTF8.self)) - case .image: return "Image" - case .color: return "Color" - case .file: return item.suggestedName ?? "File" - } - } - - static func firstLine(_ text: String, max: Int = 80) -> String { - let line = text.split(whereSeparator: \.isNewline) - .first { !$0.trimmingCharacters(in: .whitespaces).isEmpty } - .map(String.init) ?? text - let trimmed = line.trimmingCharacters(in: .whitespacesAndNewlines) - return trimmed.count <= max ? trimmed : String(trimmed.prefix(max)) + "…" - } -} diff --git a/DeviceLinkKit/Sources/DeviceLinkMirror/MovedItemStore.swift b/DeviceLinkKit/Sources/DeviceLinkMirror/MovedItemStore.swift deleted file mode 100644 index 4c844e8..0000000 --- a/DeviceLinkKit/Sources/DeviceLinkMirror/MovedItemStore.swift +++ /dev/null @@ -1,126 +0,0 @@ -import Foundation -import DeviceLinkProtocol - -/// Persists the iPhone app's moved-items list. Every representation's bytes go to a **blob file**; the -/// JSON index holds only metadata + blob filenames, so the index stays small and the store never holds -/// all payloads in memory (it materializes to `MovedItem` only on `list()`). Newest-first, replace-by-id, -/// with a count cap that evicts the oldest and deletes their blobs. Injectable directory for tests. -public final class MovedItemStore { - private let directory: URL - private var blobsDir: URL { directory.appendingPathComponent("blobs", isDirectory: true) } - private var indexURL: URL { directory.appendingPathComponent("index.json") } - - public var maxCount: Int - - private var stored: [StoredItem] = [] - - public init(directory: URL, maxCount: Int = 300) { - self.directory = directory - self.maxCount = maxCount - load() - } - - public var count: Int { stored.count } - - /// Items newest-first, with representation bytes materialized from blobs. - public func list() -> [MovedItem] { - stored.sorted { $0.movedAt > $1.movedAt }.compactMap(materialize) - } - - /// Insert (or replace a same-id record), evicting the oldest beyond the cap. - public func insert(_ item: MovedItem) { - removeBlobs(forID: item.id) - stored.removeAll { $0.id == item.id } - stored.append(writeBlobs(for: item)) - evict() - save() - } - - public func remove(id: UUID) { - removeBlobs(forID: id) - stored.removeAll { $0.id == id } - save() - } - - public func clear() { - for item in stored { removeBlobs(forID: item.id) } - stored.removeAll() - save() - } - - // MARK: - On-disk model - - private struct StoredItem: Codable { - var id: UUID - var direction: MoveDirection - var kind: LinkItemKind - var title: String - var peerName: String? - var movedAt: Date - var repFiles: [String: String] // uti -> blob filename - } - - // MARK: - Blobs - - private func writeBlobs(for item: MovedItem) -> StoredItem { - try? FileManager.default.createDirectory(at: blobsDir, withIntermediateDirectories: true) - var repFiles: [String: String] = [:] - for (uti, data) in item.representations { - let name = "\(item.id.uuidString)-\(stableName(uti)).bin" - let url = blobsDir.appendingPathComponent(name) - try? data.write(to: url, options: .atomic) - repFiles[uti] = name - } - return StoredItem(id: item.id, direction: item.direction, kind: item.kind, - title: item.title, peerName: item.peerName, movedAt: item.movedAt, repFiles: repFiles) - } - - private func materialize(_ s: StoredItem) -> MovedItem? { - var reps: [String: Data] = [:] - for (uti, name) in s.repFiles { - if let data = try? Data(contentsOf: blobsDir.appendingPathComponent(name)) { reps[uti] = data } - } - return MovedItem(id: s.id, direction: s.direction, kind: s.kind, title: s.title, - peerName: s.peerName, movedAt: s.movedAt, representations: reps) - } - - private func removeBlobs(forID id: UUID) { - guard let item = stored.first(where: { $0.id == id }) else { return } - for name in item.repFiles.values { - try? FileManager.default.removeItem(at: blobsDir.appendingPathComponent(name)) - } - } - - private func evict() { - guard stored.count > maxCount else { return } - let sorted = stored.sorted { $0.movedAt > $1.movedAt } - let keep = Array(sorted.prefix(maxCount)) - let drop = sorted.dropFirst(maxCount) - for item in drop { - for name in item.repFiles.values { - try? FileManager.default.removeItem(at: blobsDir.appendingPathComponent(name)) - } - } - stored = keep - } - - // MARK: - Persistence - - private func load() { - guard let data = try? Data(contentsOf: indexURL), - let decoded = try? JSONDecoder().decode([StoredItem].self, from: data) else { return } - stored = decoded - } - - private func save() { - try? FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true) - guard let data = try? JSONEncoder().encode(stored) else { return } - try? data.write(to: indexURL, options: .atomic) - } - - private func stableName(_ uti: String) -> String { - var hash: UInt64 = 0xcbf29ce484222325 - for byte in uti.utf8 { hash ^= UInt64(byte); hash = hash &* 0x100000001b3 } - return String(hash, radix: 16) - } -} diff --git a/DeviceLinkKit/Sources/DeviceLinkPairing/LinkSession.swift b/DeviceLinkKit/Sources/DeviceLinkPairing/LinkSession.swift deleted file mode 100644 index 1b03971..0000000 --- a/DeviceLinkKit/Sources/DeviceLinkPairing/LinkSession.swift +++ /dev/null @@ -1,211 +0,0 @@ -import Foundation -import CryptoKit -import DeviceLinkProtocol - -/// The authenticated link handshake (a Noise-KK-shaped exchange), reusing the proven `DeviceLinkPairing` -/// primitives: X25519 ECDH, role-independent HKDF-SHA256 derivation (sorted public keys + info), and -/// constant-time HMAC-SHA256 confirmations. -/// -/// Both ends already pinned the other's long-lived X25519 public key during pairing (by -/// `SHA256(rawRepresentation)`). Each side generates a fresh per-connection **ephemeral** key and sends -/// `authHello(staticPub, ephemeralPub, identity)`. On the peer's `authHello` the receiver: -/// (a) **rejects (fail closed)** if `SHA256(peer staticPub)` is not in the supplied pinned fingerprint set, -/// (b) derives a **role-independent** session `SymmetricKey` mixing the static–static term `ss` (which -/// authenticates: only the pinned-key holders can compute it) with the ephemeral term `ee` and the -/// two cross terms `se`/`es` (per-session freshness / forward secrecy), -/// (c) produces / constant-time-verifies an `authConfirm` HMAC under that key. -/// -/// A peer that presents a pinned public key but does **not** hold its private key cannot compute `ss`, -/// derives a different key, and fails the confirmation → the caller drops the connection. -/// -/// No I/O: a transport ferries the `Frame.authHello` / `Frame.authConfirm` control frames. -public struct LinkSession { - /// Why a handshake failed (rejected before / at confirmation). All are fail-closed: no session key. - public enum Failure: Error, Equatable { - /// The peer's presented long-lived key is malformed, or its ephemeral key is malformed. - case badKey - /// `SHA256(peer staticPub)` is not in the pinned fingerprint set. - case unpinned - /// The peer's `authConfirm` did not verify under the derived key (forged identity / wrong key). - case confirmationFailed - } - - private static let infoPrefix = "device-link-session-v1" - - public let identity: DeviceIdentity - public let staticKey: Curve25519.KeyAgreement.PrivateKey - public let ephemeralKey: Curve25519.KeyAgreement.PrivateKey - - /// - Parameters: - /// - identity: this device's identity, carried in `authHello`. - /// - staticKey: the local long-lived (pinned) X25519 key agreement private key. - /// - ephemeralKey: a fresh per-connection ephemeral; defaults to a new random key. - public init(identity: DeviceIdentity, - staticKey: Curve25519.KeyAgreement.PrivateKey, - ephemeralKey: Curve25519.KeyAgreement.PrivateKey = Curve25519.KeyAgreement.PrivateKey()) { - self.identity = identity - self.staticKey = staticKey - self.ephemeralKey = ephemeralKey - } - - /// This side's opening handshake frame: its pinned static public key, fresh ephemeral, and identity. - public func hello() -> Frame { - .authHello(staticPub: staticKey.publicKey.rawRepresentation, - ephemeralPub: ephemeralKey.publicKey.rawRepresentation, - identity: identity) - } - - /// `true` iff `SHA256(peerStaticRaw)` is a pinned fingerprint (fail closed when absent). - public static func isPinned(peerStaticRaw: Data, pinnedFingerprints: Set) -> Bool { - pinnedFingerprints.contains(Data(SHA256.hash(data: peerStaticRaw))) - } - - /// Consume the peer's `authHello`. Verifies the peer's static key is pinned (else `.unpinned`), - /// then derives the role-independent session key. On success returns the established session. - /// - /// - Throws: `Failure.badKey` (malformed peer key), `Failure.unpinned` (not in `pinnedFingerprints`). - public func accept(peerHello frame: Frame, - pinnedFingerprints: Set) throws -> Established { - guard case let .authHello(peerStaticRaw, peerEphemeralRaw, peerIdentity) = frame else { - throw Failure.badKey - } - // Reject a peer presenting OUR own static key (a reflection): it would collapse the role-label - // tie-break to an acceptable self-confirm. Unreachable in practice (we never pin our own key), - // but a cheap, decisive guard. - guard peerStaticRaw != staticKey.publicKey.rawRepresentation else { - throw Failure.badKey - } - guard Self.isPinned(peerStaticRaw: peerStaticRaw, pinnedFingerprints: pinnedFingerprints) else { - throw Failure.unpinned - } - guard let peerStatic = try? Curve25519.KeyAgreement.PublicKey(rawRepresentation: peerStaticRaw), - let peerEphemeral = try? Curve25519.KeyAgreement.PublicKey(rawRepresentation: peerEphemeralRaw) else { - throw Failure.badKey - } - let key = try deriveSessionKey(peerStatic: peerStatic, peerEphemeral: peerEphemeral) - // Role-independent confirm labels: the side whose static key sorts lower sends the "low" label - // and verifies the peer's "high" label (and vice versa). Both sides agree on the assignment from - // the sorted statics, so each can produce its own confirm and verify the peer's distinct one. - let localIsLow = staticKey.publicKey.rawRepresentation.lexicographicallyPrecedes(peerStaticRaw) - return Established(sessionKey: key, - peerIdentity: peerIdentity, - peerStaticFingerprint: Data(SHA256.hash(data: peerStaticRaw)), - localIsLow: localIsLow) - } - - // MARK: - Key derivation - - /// Role-independent session key: `HKDF-SHA256` over `ss ‖ ee ‖ se‖es` (cross terms ordered - /// role-independently) with `info = "device-link-session-v1" ‖ sorted(statics) ‖ sorted(ephemerals)`. - /// Mirrors `PairingHandshake.confirmationKey`'s sorted-public-key construction. - func deriveSessionKey(peerStatic: Curve25519.KeyAgreement.PublicKey, - peerEphemeral: Curve25519.KeyAgreement.PublicKey) throws -> SymmetricKey { - let localStaticRaw = staticKey.publicKey.rawRepresentation - let localEphemeralRaw = ephemeralKey.publicKey.rawRepresentation - let peerStaticRaw = peerStatic.rawRepresentation - let peerEphemeralRaw = peerEphemeral.rawRepresentation - - let ss = try staticKey.sharedSecretFromKeyAgreement(with: peerStatic) - let ee = try ephemeralKey.sharedSecretFromKeyAgreement(with: peerEphemeral) - // Cross terms: ECDH(localStatic, peerEphemeral) and ECDH(localEphemeral, peerStatic). The two ends - // compute the SAME two physical DH values but with the roles of "local"/"peer" swapped, so each - // cross term must be ordered by a key that is symmetric in its two participating public keys — - // `sorted(staticRaw, ephemeralRaw)` of that DH pair — which both ends see identically. - let crossA = try staticKey.sharedSecretFromKeyAgreement(with: peerEphemeral) // localStatic · peerEph - let crossB = try ephemeralKey.sharedSecretFromKeyAgreement(with: peerStatic) // localEph · peerStatic - let crossAKey = Self.symmetricKeyBytes(localStaticRaw, peerEphemeralRaw) - let crossBKey = Self.symmetricKeyBytes(localEphemeralRaw, peerStaticRaw) - let (firstCross, secondCross) = crossAKey.lexicographicallyPrecedes(crossBKey) - ? (crossA, crossB) : (crossB, crossA) - - var ikm = Data() - ikm.append(rawBytes(ss)) - ikm.append(rawBytes(ee)) - ikm.append(rawBytes(firstCross)) - ikm.append(rawBytes(secondCross)) - - let (lowStatic, highStatic) = Self.ordered(localStaticRaw, peerStaticRaw) - let (lowEph, highEph) = Self.ordered(localEphemeralRaw, peerEphemeralRaw) - var info = Data(Self.infoPrefix.utf8) - info.append(lowStatic) - info.append(highStatic) - info.append(lowEph) - info.append(highEph) - - return HKDF.deriveKey(inputKeyMaterial: SymmetricKey(data: ikm), - info: info, - outputByteCount: 32) - } - - private func rawBytes(_ secret: SharedSecret) -> Data { - secret.withUnsafeBytes { Data($0) } - } - - private static func ordered(_ a: Data, _ b: Data) -> (Data, Data) { - a.lexicographicallyPrecedes(b) ? (a, b) : (b, a) - } - - /// A role-independent ordering key for a DH pair: `sorted(a, b)` concatenated. Both ends produce the - /// identical bytes regardless of which key they call "local". - private static func symmetricKeyBytes(_ a: Data, _ b: Data) -> Data { - let (low, high) = ordered(a, b) - return low + high - } - - /// An authenticated session: the derived key plus the verified peer identity. The confirm helpers - /// reuse the constant-time HMAC-SHA256 confirmations from `PairingHandshake`, with distinct role - /// labels so a confirmation can't be reflected back. - public struct Established: Sendable { - public let sessionKey: SymmetricKey - public let peerIdentity: DeviceIdentity - /// `SHA256(peer staticPub)` — the pinned fingerprint the peer authenticated as. - public let peerStaticFingerprint: Data - /// Whether this side's static key sorts before the peer's — selects the confirm role labels. - let localIsLow: Bool - - private static let labelLow = "device-link-confirm-low" - private static let labelHigh = "device-link-confirm-high" - - /// The label this side sends; the peer verifies it with the same string. - private var sendLabel: String { localIsLow ? Self.labelLow : Self.labelHigh } - /// The label the peer sends; this side verifies the peer's confirm with it. - private var peerLabel: String { localIsLow ? Self.labelHigh : Self.labelLow } - - // MARK: Directional record keys - // - // The session key is role-INDEPENDENT (both ends derive the identical key), so sealing BOTH stream - // directions under it would reuse `(key, counter-nonce)` between the two `Sealer`s — a catastrophic - // AEAD nonce reuse. Instead each direction gets its OWN key, derived from the session key by an - // HKDF label that names the DIRECTION (low→high vs high→low), not the local role. Both ends agree on - // the assignment from the sorted statics, so this side's `sealKey` equals the peer's `openKey`. - private static let labelLowToHigh = "device-link-record-low-to-high" - private static let labelHighToLow = "device-link-record-high-to-low" - - private static func directionKey(_ session: SymmetricKey, _ label: String) -> SymmetricKey { - HKDF.deriveKey(inputKeyMaterial: session, info: Data(label.utf8), outputByteCount: 32) - } - - /// The key for records THIS side SEALS (its transmit direction). - public var sealKey: SymmetricKey { - Self.directionKey(sessionKey, localIsLow ? Self.labelLowToHigh : Self.labelHighToLow) - } - /// The key for records THIS side OPENS (its receive direction) — equals the peer's `sealKey`. - public var openKey: SymmetricKey { - Self.directionKey(sessionKey, localIsLow ? Self.labelHighToLow : Self.labelLowToHigh) - } - - /// The `authConfirm` frame this side sends — HMAC over its role label under the session key. - public func confirm() -> Frame { - .authConfirm(mac: Data(HMAC.authenticationCode(for: Data(sendLabel.utf8), using: sessionKey))) - } - - /// Constant-time verify the peer's received `authConfirm`. A peer that derived a different key - /// (e.g. it doesn't hold the pinned private key) produces a non-matching MAC → `false`. - public func verify(peerConfirm frame: Frame) -> Bool { - guard case let .authConfirm(mac) = frame else { return false } - return HMAC.isValidAuthenticationCode(mac, - authenticating: Data(peerLabel.utf8), - using: sessionKey) - } - } -} diff --git a/DeviceLinkKit/Sources/DeviceLinkPairing/LocalAddresses.swift b/DeviceLinkKit/Sources/DeviceLinkPairing/LocalAddresses.swift deleted file mode 100644 index bce56eb..0000000 --- a/DeviceLinkKit/Sources/DeviceLinkPairing/LocalAddresses.swift +++ /dev/null @@ -1,66 +0,0 @@ -import Foundation - -/// Enumerates this device's reachable unicast IP addresses to embed in a pairing QR, so a scanner can -/// dial directly without mDNS/Bonjour discovery. Returns Wi-Fi/Ethernet (`en*`) first and IPv4 before -/// IPv6, excluding loopback, link-local (`169.254.*` / `fe80::`), and tunnel/VM interfaces (AWDL, utun, -/// bridges, …). Pure + dependency-free (`getifaddrs`), so it's unit-testable and shared by both ends. -public enum LocalAddresses { - - /// Current routable unicast addresses, most-likely-reachable first, capped to `limit`. - public static func current(limit: Int = 4) -> [String] { - var ifaddr: UnsafeMutablePointer? - guard getifaddrs(&ifaddr) == 0 else { return [] } - defer { freeifaddrs(ifaddr) } - - struct Candidate { let address: String; let isIPv4: Bool; let isEthernet: Bool } - var candidates: [Candidate] = [] - - var ptr = ifaddr - while let p = ptr { - defer { ptr = p.pointee.ifa_next } - - let flags = Int32(bitPattern: p.pointee.ifa_flags) - guard (flags & IFF_UP) == IFF_UP, (flags & IFF_LOOPBACK) == 0 else { continue } - guard let sa = p.pointee.ifa_addr else { continue } - - let family = sa.pointee.sa_family - let isIPv4 = family == sa_family_t(AF_INET) - let isIPv6 = family == sa_family_t(AF_INET6) - guard isIPv4 || isIPv6 else { continue } - - let name = String(cString: p.pointee.ifa_name) - guard !isExcludedInterface(name) else { continue } - - var host = [CChar](repeating: 0, count: Int(NI_MAXHOST)) - guard getnameinfo(sa, socklen_t(sa.pointee.sa_len), - &host, socklen_t(host.count), nil, 0, NI_NUMERICHOST) == 0 else { continue } - - var address = host.withUnsafeBufferPointer { String(cString: $0.baseAddress!) } - if let pct = address.firstIndex(of: "%") { address = String(address[.. Bool { - ["lo", "awdl", "llw", "utun", "ipsec", "ppp", "bridge", "vmnet", "tap", "tun", "gif", "stf"] - .contains { name.hasPrefix($0) } - } - - private static func isLinkLocalOrUnspecified(_ address: String, isIPv4: Bool) -> Bool { - if address.isEmpty { return true } - if isIPv4 { return address.hasPrefix("169.254.") || address == "0.0.0.0" } - let lower = address.lowercased() - return lower.hasPrefix("fe80:") || lower == "::" || lower == "::1" - } -} diff --git a/DeviceLinkKit/Sources/DeviceLinkPairing/PairingCode.swift b/DeviceLinkKit/Sources/DeviceLinkPairing/PairingCode.swift deleted file mode 100644 index 40a9b57..0000000 --- a/DeviceLinkKit/Sources/DeviceLinkPairing/PairingCode.swift +++ /dev/null @@ -1,19 +0,0 @@ -import Foundation - -/// A short, high-entropy pairing code shown on the Mac and entered on the iPhone. It is a low-entropy -/// secret used only to *authenticate* a strong key agreement — never sent on the wire, never used -/// directly as a key. Generated from the system CSPRNG. Shared by both ends. -public enum PairingCode { - public static let defaultDigits = 8 - - /// A fresh code of `digits` decimal digits (default 8 → ~27 bits). - public static func generate(digits: Int = defaultDigits) -> String { - var rng = SystemRandomNumberGenerator() - return (0.. Bool { - code.count == digits && code.allSatisfy { $0.isASCII && $0.isNumber } - } -} diff --git a/DeviceLinkKit/Sources/DeviceLinkPairing/PairingExchange.swift b/DeviceLinkKit/Sources/DeviceLinkPairing/PairingExchange.swift deleted file mode 100644 index 9720be9..0000000 --- a/DeviceLinkKit/Sources/DeviceLinkPairing/PairingExchange.swift +++ /dev/null @@ -1,101 +0,0 @@ -import Foundation -import CryptoKit -import DeviceLinkProtocol - -/// The pure, authenticated QR pairing exchange. The **joiner** (scanned the QR) and the **host** (showed -/// it) exchange ephemeral X25519 keys and HMAC confirmations keyed by `HKDF(ECDH, salt: the QR secret)`. -/// Both end pinned to the other's long-lived fingerprint + identity, and only if they used the same -/// secret — a man-in-the-middle who couldn't read the QR derives a different key and fails confirmation. -/// No I/O: a transport ferries the `PairingMessage`s. -public struct PairingExchange { - public enum Role: Sendable { case host, joiner } - - public enum Result: Equatable, Sendable { - /// Pinned the peer: its identity + long-lived SPKI fingerprint. - case pinned(DeviceIdentity, Data) - case failed - } - - public let role: Role - - private let secret: Data - private let identity: DeviceIdentity - private let spki: Data - private let ephemeral: Curve25519.KeyAgreement.PrivateKey - - // Host-side state carried between its two `consume` calls. - private var sharedKey: SymmetricKey? - private var peerIdentity: DeviceIdentity? - private var peerSPKI: Data? - - public init(role: Role, secret: Data, identity: DeviceIdentity, spkiFingerprint: Data, - ephemeral: Curve25519.KeyAgreement.PrivateKey = Curve25519.KeyAgreement.PrivateKey()) { - self.role = role - self.secret = secret - self.identity = identity - self.spki = spkiFingerprint - self.ephemeral = ephemeral - } - - /// Joiner only: the opening message. - public func start() -> PairingMessage? { - guard role == .joiner else { return nil } - return .joinerHello(ephemeral: ephemeral.publicKey.rawRepresentation, identity: identity, spki: spki) - } - - /// Consume a message; return the reply to send (if any) and a terminal result (if reached). - public mutating func consume(_ message: PairingMessage) throws -> (reply: PairingMessage?, result: Result?) { - switch (role, message) { - case let (.host, .joinerHello(ephData, joinerID, joinerSPKI)): - let key = try deriveKey(peerEphemeral: ephData) - sharedKey = key - peerIdentity = joinerID - peerSPKI = joinerSPKI - let confirm = mac(key, label: "host") - return (.hostHello(ephemeral: ephemeral.publicKey.rawRepresentation, identity: identity, spki: spki, confirm: confirm), nil) - - case let (.joiner, .hostHello(ephData, hostID, hostSPKI, hostConfirm)): - let key = try deriveKey(peerEphemeral: ephData) - guard verify(hostConfirm, key: key, label: "host") else { return (nil, .failed) } - return (.joinerConfirm(confirm: mac(key, label: "joiner")), .pinned(hostID, hostSPKI)) - - case let (.host, .joinerConfirm(joinerConfirm)): - guard let key = sharedKey, let pid = peerIdentity, let psp = peerSPKI, - verify(joinerConfirm, key: key, label: "joiner") else { return (nil, .failed) } - return (nil, .pinned(pid, psp)) - - default: - return (nil, .failed) - } - } - - // MARK: - Crypto - - private func deriveKey(peerEphemeral: Data) throws -> SymmetricKey { - guard let peerPub = try? Curve25519.KeyAgreement.PublicKey(rawRepresentation: peerEphemeral) else { - throw PairingExchangeError.badKey - } - let shared = try ephemeral.sharedSecretFromKeyAgreement(with: peerPub) - let (low, high) = ordered(ephemeral.publicKey.rawRepresentation, peerEphemeral) - var info = Data("device-link-qr-v1".utf8) - info.append(low) - info.append(high) - return shared.hkdfDerivedSymmetricKey(using: SHA256.self, salt: secret, sharedInfo: info, outputByteCount: 32) - } - - private func mac(_ key: SymmetricKey, label: String) -> Data { - Data(HMAC.authenticationCode(for: Data(label.utf8), using: key)) - } - - private func verify(_ tag: Data, key: SymmetricKey, label: String) -> Bool { - HMAC.isValidAuthenticationCode(tag, authenticating: Data(label.utf8), using: key) - } - - private func ordered(_ a: Data, _ b: Data) -> (Data, Data) { - a.lexicographicallyPrecedes(b) ? (a, b) : (b, a) - } -} - -public enum PairingExchangeError: Error, Equatable { - case badKey -} diff --git a/DeviceLinkKit/Sources/DeviceLinkPairing/PairingHandshake.swift b/DeviceLinkKit/Sources/DeviceLinkPairing/PairingHandshake.swift deleted file mode 100644 index 701e833..0000000 --- a/DeviceLinkKit/Sources/DeviceLinkPairing/PairingHandshake.swift +++ /dev/null @@ -1,44 +0,0 @@ -import Foundation -import CryptoKit - -/// The cryptographic core of serverless, CA-free pairing, shared by the Mac and the iPhone. Each side -/// holds an ephemeral X25519 key pair. Given the peer's public key and the shared code, both derive the -/// SAME confirmation key — and only if they used the same code — by HKDF over the ECDH shared secret, -/// salted by the code and bound to both public keys. An active man-in-the-middle who substitutes keys but -/// does not know the code derives a different key and cannot forge a matching HMAC confirmation. -public struct PairingHandshake { - public let privateKey: Curve25519.KeyAgreement.PrivateKey - public var publicKey: Curve25519.KeyAgreement.PublicKey { privateKey.publicKey } - - public init(privateKey: Curve25519.KeyAgreement.PrivateKey = Curve25519.KeyAgreement.PrivateKey()) { - self.privateKey = privateKey - } - - /// Derive the shared confirmation key. Role-independent: both sides sort the two public keys, so - /// initiator and responder compute the identical key. - public func confirmationKey(peerPublicKey: Curve25519.KeyAgreement.PublicKey, code: String) throws -> SymmetricKey { - let shared = try privateKey.sharedSecretFromKeyAgreement(with: peerPublicKey) - let (low, high) = Self.ordered(publicKey.rawRepresentation, peerPublicKey.rawRepresentation) - var info = Data("device-link-pairing-v1".utf8) - info.append(low) - info.append(high) - return shared.hkdfDerivedSymmetricKey(using: SHA256.self, - salt: Data(code.utf8), - sharedInfo: info, - outputByteCount: 32) - } - - /// The confirmation MAC a side sends to prove it derived the same key. - public func confirmation(_ key: SymmetricKey, label: String) -> Data { - Data(HMAC.authenticationCode(for: Data(label.utf8), using: key)) - } - - /// Constant-time verification of a received confirmation MAC. - public func verify(_ mac: Data, key: SymmetricKey, label: String) -> Bool { - HMAC.isValidAuthenticationCode(mac, authenticating: Data(label.utf8), using: key) - } - - private static func ordered(_ a: Data, _ b: Data) -> (Data, Data) { - a.lexicographicallyPrecedes(b) ? (a, b) : (b, a) - } -} diff --git a/DeviceLinkKit/Sources/DeviceLinkPairing/PairingMessage.swift b/DeviceLinkKit/Sources/DeviceLinkPairing/PairingMessage.swift deleted file mode 100644 index edbc2ce..0000000 --- a/DeviceLinkKit/Sources/DeviceLinkPairing/PairingMessage.swift +++ /dev/null @@ -1,13 +0,0 @@ -import Foundation -import DeviceLinkProtocol - -/// The three messages of the QR pairing exchange. X25519 public keys are carried as their raw -/// representation; confirmations are HMAC tags. `Codable` so a transport can ferry them. -public enum PairingMessage: Codable, Equatable, Sendable { - /// Joiner (scanned the QR) opens with its ephemeral public key, identity, and long-lived fingerprint. - case joinerHello(ephemeral: Data, identity: DeviceIdentity, spki: Data) - /// Host (showed the QR) replies with its own + a confirmation it knew the secret. - case hostHello(ephemeral: Data, identity: DeviceIdentity, spki: Data, confirm: Data) - /// Joiner confirms it, too, knew the secret. - case joinerConfirm(confirm: Data) -} diff --git a/DeviceLinkKit/Sources/DeviceLinkPairing/PairingQRPayload.swift b/DeviceLinkKit/Sources/DeviceLinkPairing/PairingQRPayload.swift deleted file mode 100644 index cac4619..0000000 --- a/DeviceLinkKit/Sources/DeviceLinkPairing/PairingQRPayload.swift +++ /dev/null @@ -1,94 +0,0 @@ -import Foundation -import DeviceLinkProtocol - -/// What a pairing QR encodes: the showing device's identity, a fresh high-entropy secret (the -/// out-of-band authenticator — far stronger than an 8-digit code), and the device's long-lived -/// public-key (SPKI) fingerprint so the scanner can pin its TLS identity. v2 additionally carries the -/// shower's reachable network address(es) + its listener port so the scanner can dial it **directly** -/// (unicast) without relying on mDNS/Bonjour discovery, which routers commonly filter between clients. -/// Encoded as a scheme-tagged, versioned base64url string. -public struct PairingQRPayload: Equatable, Sendable { - /// Bumped to 2 for the optional `addresses` + `port` endpoint. v1 (no endpoint) still decodes. - public static let currentVersion = 2 - public static let scheme = "tfslink:" - - public var version: Int - public var device: DeviceIdentity - public var secret: Data - public var spkiFingerprint: Data - /// The shower's reachable unicast address(es), most-likely-reachable first (Wi-Fi/Ethernet, IPv4 first). - /// Empty for a v1 payload or when none could be enumerated → the scanner falls back to discovery. - public var addresses: [String] - /// The bound TCP port of the shower's pairing listener. `nil` for a v1 payload. - public var port: UInt16? - - public init(device: DeviceIdentity, - secret: Data, - spkiFingerprint: Data, - addresses: [String] = [], - port: UInt16? = nil, - version: Int = PairingQRPayload.currentVersion) { - self.version = version - self.device = device - self.secret = secret - self.spkiFingerprint = spkiFingerprint - self.addresses = addresses - self.port = port - } - - /// True when the payload carries a directly-dialable endpoint (at least one address + a port). - public var hasEndpoint: Bool { !addresses.isEmpty && port != nil } - - /// 32 cryptographically-random bytes (the global RNG is a CSPRNG). - public static func makeSecret() -> Data { - Data((0..<32).map { _ in UInt8.random(in: UInt8.min...UInt8.max) }) - } - - public func encodedString() -> String { - let wire = Wire(v: version, id: device.id, name: device.name, secret: secret, fp: spkiFingerprint, - addrs: addresses.isEmpty ? nil : addresses, port: port) - let json = (try? JSONEncoder().encode(wire)) ?? Data() - return Self.scheme + Self.base64url(json) - } - - public init(string: String) throws { - guard string.hasPrefix(Self.scheme) else { throw PairingQRError.badScheme } - let body = String(string.dropFirst(Self.scheme.count)) - guard let json = Self.base64urlDecode(body) else { throw PairingQRError.malformed } - guard let wire = try? JSONDecoder().decode(Wire.self, from: json) else { throw PairingQRError.malformed } - // Accept v1 (no endpoint) and the current version; reject anything else. - guard wire.v == 1 || wire.v == Self.currentVersion else { throw PairingQRError.unsupportedVersion } - self.init(device: DeviceIdentity(id: wire.id, name: wire.name), - secret: wire.secret, spkiFingerprint: wire.fp, - addresses: wire.addrs ?? [], port: wire.port, version: wire.v) - } - - private struct Wire: Codable { - var v: Int - var id: String - var name: String - var secret: Data - var fp: Data - var addrs: [String]? // v2+, optional — omitted on the wire when empty/v1 - var port: UInt16? // v2+, optional - } - - static func base64url(_ data: Data) -> String { - data.base64EncodedString() - .replacingOccurrences(of: "+", with: "-") - .replacingOccurrences(of: "/", with: "_") - .replacingOccurrences(of: "=", with: "") - } - - static func base64urlDecode(_ s: String) -> Data? { - var b = s.replacingOccurrences(of: "-", with: "+").replacingOccurrences(of: "_", with: "/") - while b.count % 4 != 0 { b += "=" } - return Data(base64Encoded: b) - } -} - -public enum PairingQRError: Error, Equatable { - case badScheme - case unsupportedVersion - case malformed -} diff --git a/DeviceLinkKit/Sources/DeviceLinkPairing/SealedRecord.swift b/DeviceLinkKit/Sources/DeviceLinkPairing/SealedRecord.swift deleted file mode 100644 index 80d438f..0000000 --- a/DeviceLinkKit/Sources/DeviceLinkPairing/SealedRecord.swift +++ /dev/null @@ -1,118 +0,0 @@ -import Foundation -import CryptoKit - -/// The confidential-channel layer for an authenticated link: every outbound buffer is wrapped as a -/// length-prefixed `ChaChaPoly` sealed record under the session key, and opened on receive — transparently, -/// *below* the framing pump so the tested `LinkPump` / wire `LinkItem` is unchanged. -/// -/// **Nonce discipline (security-critical):** the 96-bit nonce is a strictly-monotonic per-direction -/// counter, NOT carried on the wire. Each direction owns one `Sealer` and the peer owns the matching -/// `Opener`; both start at 0 and advance in lockstep. A fresh session key resets the counters. Because the -/// counter is implicit, records must be opened in the exact order they were sealed — a dropped, reordered, -/// duplicated, or bit-flipped record fails AEAD authentication and `open` throws (the caller closes the -/// connection; no partial item is surfaced). -/// -/// Wire record: `length(UInt32 BE) ‖ ChaChaPoly(ciphertext ‖ tag)`. The 16-byte tag is included in the -/// length. Only the long-lived key holders share the session key, so an on-path tap can neither read nor -/// forge records. -public enum SealedRecord { - /// AEAD overhead per record: the 4-byte length prefix plus ChaChaPoly's 16-byte authentication tag. - public static let tagSize = 16 - public static let lengthPrefixSize = 4 - - public enum Error: Swift.Error, Equatable { - /// A record was shorter than the length prefix, or its declared length exceeded the buffer. - case truncated - /// AEAD authentication failed: a tampered/reordered/duplicated record, or a nonce-counter skew. - case authenticationFailed - /// The per-direction counter would overflow 2^64 records (never reached in practice). - case counterExhausted - } - - /// Seals outbound buffers for one direction under the session key with a monotonic counter nonce. - /// One instance per connection-direction; not thread-safe (the transport owns its serial context). - public struct Sealer { - private let key: SymmetricKey - private var counter: UInt64 = 0 - - public init(key: SymmetricKey) { self.key = key } - - /// The current (next-to-use) counter value — for tests / diagnostics. - public var nextCounter: UInt64 { counter } - - /// Seal one buffer into a length-prefixed record and advance the counter. Throws only if the - /// counter is exhausted (2^64 records). - public mutating func seal(_ plaintext: Data) throws -> Data { - guard counter < UInt64.max else { throw Error.counterExhausted } - let nonce = try ChaChaPoly.Nonce(data: SealedRecord.nonceData(counter)) - let box = try ChaChaPoly.seal(plaintext, using: key, nonce: nonce) - // `box.combined` is nonce(12) ‖ ciphertext ‖ tag(16). The nonce is implicit (the counter), so - // we transmit only ciphertext ‖ tag — strictly enforcing in-order opening. - let payload = box.ciphertext + box.tag - counter &+= 1 - var out = Data() - SealedRecord.appendU32BE(UInt32(payload.count), to: &out) - out.append(payload) - return out - } - } - - /// Opens inbound records for one direction. Mirrors a peer `Sealer`: same key, counter starting at 0. - public struct Opener { - private let key: SymmetricKey - private var counter: UInt64 = 0 - - public init(key: SymmetricKey) { self.key = key } - - /// The current (next-expected) counter value — for tests / diagnostics. - public var nextCounter: UInt64 { counter } - - /// Open exactly one length-prefixed record from the front of `record`, returning the plaintext and - /// the number of bytes consumed. Throws `.truncated` if the buffer is short, `.authenticationFailed` - /// on any AEAD failure (tamper / reorder / wrong key / counter skew). - public mutating func open(_ record: Data) throws -> (plaintext: Data, consumed: Int) { - guard record.count >= SealedRecord.lengthPrefixSize else { throw Error.truncated } - let s = record.startIndex - let length = Int(SealedRecord.u32(record, 0)) - guard length >= SealedRecord.tagSize else { throw Error.authenticationFailed } - let total = SealedRecord.lengthPrefixSize + length - guard record.count >= total else { throw Error.truncated } - let payload = record[(s + SealedRecord.lengthPrefixSize)..<(s + total)] - let cipherEnd = payload.endIndex - SealedRecord.tagSize - let ciphertext = payload[payload.startIndex.. Data { - var d = Data(repeating: 0, count: 4) - var be = counter.bigEndian - withUnsafeBytes(of: &be) { d.append(contentsOf: $0) } - return d - } - - static func appendU32BE(_ v: UInt32, to data: inout Data) { - data.append(UInt8((v >> 24) & 0xff)) - data.append(UInt8((v >> 16) & 0xff)) - data.append(UInt8((v >> 8) & 0xff)) - data.append(UInt8(v & 0xff)) - } - - static func u32(_ d: Data, _ offset: Int) -> UInt32 { - let s = d.startIndex + offset - return (UInt32(d[s]) << 24) | (UInt32(d[s + 1]) << 16) | (UInt32(d[s + 2]) << 8) | UInt32(d[s + 3]) - } -} diff --git a/DeviceLinkKit/Sources/DeviceLinkProtocol/Frame.swift b/DeviceLinkKit/Sources/DeviceLinkProtocol/Frame.swift deleted file mode 100644 index 9676858..0000000 --- a/DeviceLinkKit/Sources/DeviceLinkProtocol/Frame.swift +++ /dev/null @@ -1,89 +0,0 @@ -import Foundation - -/// The header that opens an item: its kind, the per-representation byte manifest (UTI → total bytes, -/// so the receiver knows the full size before any bytes arrive), and optional metadata. `Codable` — -/// encoded as a control body by the codec. -public struct ItemHeader: Equatable, Sendable, Codable { - public var messageID: UUID - public var kind: LinkItemKind - public var manifest: [String: UInt32] - public var suggestedName: String? - public var capturedAt: Date? - public var origin: DeviceIdentity? - - public init(messageID: UUID, - kind: LinkItemKind, - manifest: [String: UInt32], - suggestedName: String? = nil, - capturedAt: Date? = nil, - origin: DeviceIdentity? = nil) { - self.messageID = messageID - self.kind = kind - self.manifest = manifest - self.suggestedName = suggestedName - self.capturedAt = capturedAt - self.origin = origin - } -} - -/// One bounded slice of one representation's bytes. Hand-encoded as raw bytes (never JSON/base64) so a -/// large file streams without inflation. `seq` is the 0-based, per-representation chunk index. -public struct ChunkFrame: Equatable, Sendable { - public var messageID: UUID - public var uti: String - public var seq: UInt32 - public var bytes: Data - - public init(messageID: UUID, uti: String, seq: UInt32, bytes: Data) { - self.messageID = messageID - self.uti = uti - self.seq = seq - self.bytes = bytes - } -} - -/// The closed set of wire frames. Item-bearing frames carry their `messageID` so frames for different -/// items can be interleaved on a single stream. -public enum Frame: Equatable, Sendable { - case hello(DeviceIdentity, ProtocolVersion) - case ack(UUID) - case error(LinkProtocolError.Code) - case itemBegin(ItemHeader) - case chunk(ChunkFrame) - case itemEnd(UUID) - case cancel(UUID) - /// First control frame of the authenticated link handshake: the sender's pinned long-lived X25519 - /// public key (raw), a fresh per-connection ephemeral X25519 public key (raw), and its identity. - /// Carried in the clear (only public keys + identity); the receiver pin-verifies the static key. - case authHello(staticPub: Data, ephemeralPub: Data, identity: DeviceIdentity) - /// Second handshake frame: an HMAC over the session key proving the sender derived the same key - /// (and therefore holds the private key behind its pinned fingerprint). Constant-time verified. - case authConfirm(mac: Data) - - /// The message id this frame belongs to, when it is item-scoped (nil for `hello`/`error`/auth frames). - public var messageID: UUID? { - switch self { - case .hello, .error, .authHello, .authConfirm: - return nil - case let .ack(id), let .itemEnd(id), let .cancel(id): - return id - case let .itemBegin(header): - return header.messageID - case let .chunk(chunk): - return chunk.messageID - } - } -} - -/// The 1-byte wire tag for each frame type (written in the envelope). Internal to the codec. -enum FrameType: UInt8 { - case hello = 1 - case ack = 2 - case error = 3 - case itemBegin = 4 - case chunk = 5 - case itemEnd = 6 - case cancel = 7 - case authHello = 8 - case authConfirm = 9 -} diff --git a/DeviceLinkKit/Sources/DeviceLinkProtocol/FrameStreamEncoder.swift b/DeviceLinkKit/Sources/DeviceLinkProtocol/FrameStreamEncoder.swift deleted file mode 100644 index 40728ed..0000000 --- a/DeviceLinkKit/Sources/DeviceLinkProtocol/FrameStreamEncoder.swift +++ /dev/null @@ -1,51 +0,0 @@ -import Foundation - -/// The send-side counterpart to `InboundAssembler`: splits a `LinkItem` into the ordered frame sequence -/// `itemBegin → chunk… → itemEnd`, honoring a chunk byte bound so large representations stream. Pure and -/// deterministic (representations in sorted-UTI order, 0-based per-representation sequence numbers), so -/// `encode(item) → InboundAssembler` reconstructs an equal item. -public struct FrameStreamEncoder { - /// Max bytes per `chunk` frame's representation slice. - public var chunkByteBound: Int - - public init(chunkByteBound: Int = LinkProtocol.defaultChunkByteBound) { - self.chunkByteBound = max(1, chunkByteBound) - } - - /// The ordered frames for an item: one header, then bounded chunks per representation, then a terminator. - public func frames(for item: LinkItem) -> [Frame] { - var frames: [Frame] = [] - - let manifest = item.representations.mapValues { UInt32($0.count) } - let header = ItemHeader(messageID: item.messageID, - kind: item.kind, - manifest: manifest, - suggestedName: item.suggestedName, - capturedAt: item.capturedAt, - origin: item.origin) - frames.append(.itemBegin(header)) - - // Deterministic representation order so output is reproducible and byte-stable. - for uti in item.representations.keys.sorted() { - let data = item.representations[uti] ?? Data() - if data.isEmpty { - // Emit one empty chunk so the assembler records the (empty) representation and the - // round-trip preserves it. - frames.append(.chunk(ChunkFrame(messageID: item.messageID, uti: uti, seq: 0, bytes: Data()))) - continue - } - var seq: UInt32 = 0 - var offset = 0 - while offset < data.count { - let end = min(offset + chunkByteBound, data.count) - let slice = data.subdata(in: (data.startIndex + offset)..<(data.startIndex + end)) - frames.append(.chunk(ChunkFrame(messageID: item.messageID, uti: uti, seq: seq, bytes: slice))) - seq += 1 - offset = end - } - } - - frames.append(.itemEnd(item.messageID)) - return frames - } -} diff --git a/DeviceLinkKit/Sources/DeviceLinkProtocol/InboundAssembler.swift b/DeviceLinkKit/Sources/DeviceLinkProtocol/InboundAssembler.swift deleted file mode 100644 index f5271c6..0000000 --- a/DeviceLinkKit/Sources/DeviceLinkProtocol/InboundAssembler.swift +++ /dev/null @@ -1,93 +0,0 @@ -import Foundation - -/// Reassembles streamed frames into complete `LinkItem`s. Pure (no I/O); holds only the bytes of items -/// currently in flight. Feed it decoded `Frame`s in arrival order; it emits an item on `itemEnd`, passes -/// control frames through, and throws a typed `LinkProtocolError` on a protocol violation. -/// -/// For very large files a transport MAY bypass this and stream chunks straight to disk (see the design's -/// D4 disk-streaming seam); this assembler is the simple, correct in-memory path used for control and -/// small/medium items. -public struct InboundAssembler { - /// The result of consuming one frame. - public enum Output: Equatable, Sendable { - case item(LinkItem) // a complete item was reassembled - case control(Frame) // a hello/ack/error passed through for the transport to handle - case none // progress was made; nothing to surface yet - } - - private struct InFlight { - var header: ItemHeader - var buffers: [String: Data] // uti -> accumulated bytes - var nextSeq: [String: UInt32] // uti -> expected next chunk index - } - - private var inFlight: [UUID: InFlight] = [:] - - public init() {} - - /// Items currently being reassembled (diagnostics / tests). - public var inFlightCount: Int { inFlight.count } - - public mutating func consume(_ frame: Frame) throws -> Output { - switch frame { - case .hello, .ack, .error, .authHello, .authConfirm: - return .control(frame) - - case let .itemBegin(header): - guard inFlight[header.messageID] == nil else { - throw LinkProtocolError(.duplicateMessage) - } - inFlight[header.messageID] = InFlight(header: header, buffers: [:], nextSeq: [:]) - return .none - - case let .chunk(chunk): - guard var flight = inFlight[chunk.messageID] else { - throw LinkProtocolError(.unknownMessage) - } - guard let total = flight.header.manifest[chunk.uti] else { - inFlight[chunk.messageID] = nil - throw LinkProtocolError(.manifestMismatch) - } - let expected = flight.nextSeq[chunk.uti] ?? 0 - guard chunk.seq == expected else { - inFlight[chunk.messageID] = nil - throw LinkProtocolError(.badSequence) - } - var accumulated = flight.buffers[chunk.uti] ?? Data() - accumulated.append(chunk.bytes) - guard accumulated.count <= Int(total) else { - inFlight[chunk.messageID] = nil - throw LinkProtocolError(.manifestMismatch) - } - flight.buffers[chunk.uti] = accumulated - flight.nextSeq[chunk.uti] = expected + 1 - inFlight[chunk.messageID] = flight - return .none - - case let .itemEnd(id): - guard let flight = inFlight[id] else { - throw LinkProtocolError(.unknownMessage) - } - // Every declared representation must be exactly complete. - for (uti, total) in flight.header.manifest { - let have = flight.buffers[uti]?.count ?? 0 - guard have == Int(total) else { - inFlight[id] = nil - throw LinkProtocolError(.manifestMismatch) - } - } - inFlight[id] = nil - let item = LinkItem(messageID: id, - kind: flight.header.kind, - representations: flight.buffers, - suggestedName: flight.header.suggestedName, - capturedAt: flight.header.capturedAt, - origin: flight.header.origin) - return .item(item) - - case let .cancel(id): - inFlight[id] = nil // discard partial state; not an error - return .none - } - } -} diff --git a/DeviceLinkKit/Sources/DeviceLinkProtocol/LinkCodec.swift b/DeviceLinkKit/Sources/DeviceLinkProtocol/LinkCodec.swift deleted file mode 100644 index c55235d..0000000 --- a/DeviceLinkKit/Sources/DeviceLinkProtocol/LinkCodec.swift +++ /dev/null @@ -1,220 +0,0 @@ -import Foundation - -/// The length-prefixed binary codec. Frame envelope on the wire: -/// -/// magic(4) | wireFormatVersion(1) | frameType(1) | length(UInt32 BE) | payload(length) -/// -/// Control/header frame bodies (`hello`/`ack`/`error`/`itemBegin`/`itemEnd`/`cancel`) are encoded with -/// a deterministic JSON encoder (small, structural, forward-evolvable). `chunk` bodies are raw bytes -/// with a fixed sub-header — never JSON/base64 — so large transfers stream without inflation. -public enum LinkCodec { - static let magic: [UInt8] = [0x54, 0x46, 0x53, 0x4C] // "TFSL" - static let wireFormatVersion: UInt8 = 1 - /// Envelope size: magic(4) + version(1) + type(1) + length(4). - static let envelopePrefix = 10 - - // MARK: Encode - - public static func encode(_ frame: Frame) throws -> Data { - let (type, payload) = try encodePayload(frame) - var out = Data() - out.append(contentsOf: magic) - out.append(wireFormatVersion) - out.append(type.rawValue) - appendU32BE(UInt32(payload.count), to: &out) - out.append(payload) - return out - } - - static func encodePayload(_ frame: Frame) throws -> (FrameType, Data) { - switch frame { - case let .hello(identity, version): - return (.hello, try json.encode(HelloBody(identity: identity, version: version))) - case let .ack(id): - return (.ack, try json.encode(IDBody(messageID: id))) - case let .error(code): - return (.error, try json.encode(ErrorBody(code: code))) - case let .itemBegin(header): - return (.itemBegin, try json.encode(header)) - case let .chunk(chunk): - return (.chunk, encodeChunk(chunk)) - case let .itemEnd(id): - return (.itemEnd, try json.encode(IDBody(messageID: id))) - case let .cancel(id): - return (.cancel, try json.encode(IDBody(messageID: id))) - case let .authHello(staticPub, ephemeralPub, identity): - return (.authHello, try json.encode(AuthHelloBody(staticPub: staticPub, ephemeralPub: ephemeralPub, identity: identity))) - case let .authConfirm(mac): - return (.authConfirm, try json.encode(AuthConfirmBody(mac: mac))) - } - } - - static func encodeChunk(_ chunk: ChunkFrame) -> Data { - var out = Data() - out.append(uuidBytes(chunk.messageID)) // 16 - let utiBytes = Array(chunk.uti.utf8) - appendU16BE(UInt16(utiBytes.count), to: &out) // 2 - out.append(contentsOf: utiBytes) // utiLen - appendU32BE(chunk.seq, to: &out) // 4 - out.append(chunk.bytes) // rest - return out - } - - // MARK: Decode - - static func decodePayload(type: FrameType, payload: Data) throws -> Frame { - do { - switch type { - case .hello: - let body = try json.decode(HelloBody.self, from: payload) - return .hello(body.identity, body.version) - case .ack: - return .ack(try json.decode(IDBody.self, from: payload).messageID) - case .error: - return .error(try json.decode(ErrorBody.self, from: payload).code) - case .itemBegin: - return .itemBegin(try json.decode(ItemHeader.self, from: payload)) - case .chunk: - return .chunk(try decodeChunk(payload)) - case .itemEnd: - return .itemEnd(try json.decode(IDBody.self, from: payload).messageID) - case .cancel: - return .cancel(try json.decode(IDBody.self, from: payload).messageID) - case .authHello: - let body = try json.decode(AuthHelloBody.self, from: payload) - return .authHello(staticPub: body.staticPub, ephemeralPub: body.ephemeralPub, identity: body.identity) - case .authConfirm: - return .authConfirm(mac: try json.decode(AuthConfirmBody.self, from: payload).mac) - } - } catch let error as LinkProtocolError { - throw error - } catch { - throw LinkProtocolError(.malformedPayload) - } - } - - static func decodeChunk(_ payload: Data) throws -> ChunkFrame { - guard payload.count >= 16 + 2 else { throw LinkProtocolError(.malformedPayload) } - let messageID = uuid(from: payload, offset: 0) - let utiLen = Int(u16(payload, 16)) - let utiStart = 18 - guard payload.count >= utiStart + utiLen + 4 else { throw LinkProtocolError(.malformedPayload) } - let s = payload.startIndex - let uti = String(decoding: payload[(s + utiStart)..<(s + utiStart + utiLen)], as: UTF8.self) - let seq = u32(payload, utiStart + utiLen) - let bytesStart = utiStart + utiLen + 4 - let bytes = Data(payload[(s + bytesStart)...]) - return ChunkFrame(messageID: messageID, uti: uti, seq: seq, bytes: bytes) - } - - // MARK: Codable bodies (control/header frames) - - static let json: JSONCoder = JSONCoder() - - struct HelloBody: Codable { var identity: DeviceIdentity; var version: ProtocolVersion } - struct IDBody: Codable { var messageID: UUID } - struct ErrorBody: Codable { var code: LinkProtocolError.Code } - struct AuthHelloBody: Codable { var staticPub: Data; var ephemeralPub: Data; var identity: DeviceIdentity } - struct AuthConfirmBody: Codable { var mac: Data } - - // MARK: Byte helpers - - static func appendU16BE(_ v: UInt16, to data: inout Data) { - data.append(UInt8((v >> 8) & 0xff)) - data.append(UInt8(v & 0xff)) - } - - static func appendU32BE(_ v: UInt32, to data: inout Data) { - data.append(UInt8((v >> 24) & 0xff)) - data.append(UInt8((v >> 16) & 0xff)) - data.append(UInt8((v >> 8) & 0xff)) - data.append(UInt8(v & 0xff)) - } - - static func u16(_ d: Data, _ offset: Int) -> UInt16 { - let s = d.startIndex + offset - return (UInt16(d[s]) << 8) | UInt16(d[s + 1]) - } - - static func u32(_ d: Data, _ offset: Int) -> UInt32 { - let s = d.startIndex + offset - return (UInt32(d[s]) << 24) | (UInt32(d[s + 1]) << 16) | (UInt32(d[s + 2]) << 8) | UInt32(d[s + 3]) - } - - static func uuidBytes(_ uuid: UUID) -> Data { - var u = uuid.uuid - return withUnsafeBytes(of: &u) { Data($0) } - } - - static func uuid(from d: Data, offset: Int) -> UUID { - var bytes = [UInt8](repeating: 0, count: 16) - let s = d.startIndex + offset - for i in 0..<16 { bytes[i] = d[s + i] } - return bytes.withUnsafeBytes { UUID(uuid: $0.load(as: uuid_t.self)) } - } -} - -/// A small deterministic JSON encode/decode pair (sorted keys → reproducible bytes). Wrapped so the -/// codec holds one instance and tests can rely on byte-stable output. -struct JSONCoder { - private let encoder: JSONEncoder - private let decoder: JSONDecoder - - init() { - let e = JSONEncoder() - e.outputFormatting = [.sortedKeys] - self.encoder = e - self.decoder = JSONDecoder() - } - - func encode(_ value: T) throws -> Data { try encoder.encode(value) } - func decode(_ type: T.Type, from data: Data) throws -> T { try decoder.decode(type, from: data) } -} - -/// A streaming frame splitter. Push bytes as they arrive; pull complete `Frame`s until `next()` returns -/// nil (needs more bytes). Reassembles a frame across multiple reads, enforces a max-frame cap, and -/// throws a typed `LinkProtocolError` on a malformed/oversize stream. -public struct FrameDecoder { - public var maxFrameLength: Int - private var buffer = Data() - - public init(maxFrameLength: Int = LinkProtocol.defaultMaxFrameLength) { - self.maxFrameLength = maxFrameLength - } - - /// Append newly-received bytes. - public mutating func push(_ data: Data) { - buffer.append(data) - } - - /// The next complete frame, or nil if more bytes are needed. Throws on a malformed/oversize stream. - public mutating func next() throws -> Frame? { - guard buffer.count >= LinkCodec.envelopePrefix else { return nil } - let s = buffer.startIndex - for i in 0..<4 where buffer[s + i] != LinkCodec.magic[i] { - throw LinkProtocolError(.badMagic) - } - guard buffer[s + 4] == LinkCodec.wireFormatVersion else { - throw LinkProtocolError(.unsupportedVersion) - } - guard let type = FrameType(rawValue: buffer[s + 5]) else { - throw LinkProtocolError(.unknownFrameType) - } - let length = Int(LinkCodec.u32(buffer, 6)) - guard length <= maxFrameLength else { throw LinkProtocolError(.oversizeLength) } - let total = LinkCodec.envelopePrefix + length - guard buffer.count >= total else { return nil } // need more bytes - let payload = Data(buffer[(s + LinkCodec.envelopePrefix)..<(s + total)]) - buffer = Data(buffer[(s + total)...]) // advance + reset indices - return try LinkCodec.decodePayload(type: type, payload: payload) - } - - /// Number of buffered bytes not yet consumed (a partial frame). - public var bufferedByteCount: Int { buffer.count } - - /// Call when the underlying stream has closed AFTER draining all frames via `next()`. Throws - /// `.truncatedFrame` if a partial frame remains. - public func close() throws { - if !buffer.isEmpty { throw LinkProtocolError(.truncatedFrame) } - } -} diff --git a/DeviceLinkKit/Sources/DeviceLinkProtocol/LinkItem.swift b/DeviceLinkKit/Sources/DeviceLinkProtocol/LinkItem.swift deleted file mode 100644 index 85103d2..0000000 --- a/DeviceLinkKit/Sources/DeviceLinkProtocol/LinkItem.swift +++ /dev/null @@ -1,41 +0,0 @@ -import Foundation - -/// What kind of content a moved item holds — mirrors the Mac clipboard's `ClipboardKind` shape so the -/// inbound adapter can map 1:1, but defined here independently (the wire never couples to storage). -public enum LinkItemKind: String, Codable, Equatable, Sendable, CaseIterable { - case text - case richText - case image - case color - case url - case file -} - -/// The transport DTO for one moved item: a kind plus its representations keyed by UTI string, with -/// optional metadata. Pure value type, `Sendable`, no AppKit/UIKit. The Mac side maps `LinkItem ⇄ -/// ClipboardEntry` at its boundary; the iOS side stores it directly. `representations` holds the -/// **materialized** bytes of an assembled item (the assembler bounds this to items in flight). -public struct LinkItem: Equatable, Sendable { - public var messageID: UUID - public var kind: LinkItemKind - /// UTI string → representation bytes. The same UTIs that appear in the item's `ItemHeader.manifest`. - public var representations: [String: Data] - public var suggestedName: String? - public var capturedAt: Date? - /// The device that originated the item (set by the sender; used for a "from iPhone/Mac" chip). - public var origin: DeviceIdentity? - - public init(messageID: UUID, - kind: LinkItemKind, - representations: [String: Data], - suggestedName: String? = nil, - capturedAt: Date? = nil, - origin: DeviceIdentity? = nil) { - self.messageID = messageID - self.kind = kind - self.representations = representations - self.suggestedName = suggestedName - self.capturedAt = capturedAt - self.origin = origin - } -} diff --git a/DeviceLinkKit/Sources/DeviceLinkProtocol/LinkProtocolError.swift b/DeviceLinkKit/Sources/DeviceLinkProtocol/LinkProtocolError.swift deleted file mode 100644 index 73ad3af..0000000 --- a/DeviceLinkKit/Sources/DeviceLinkProtocol/LinkProtocolError.swift +++ /dev/null @@ -1,44 +0,0 @@ -import Foundation - -/// The single error taxonomy for the protocol. Every decode/reassembly failure maps to one of these -/// cases — the package never surfaces a raw Foundation/decoding error to callers. Transports map these -/// at their boundary into their own presented errors. Mirrors the app's one-taxonomy convention. -public struct LinkProtocolError: Error, Equatable, Sendable { - public enum Code: String, Codable, Equatable, Sendable { - case badMagic // frame did not begin with the protocol magic - case unsupportedVersion // wire-format version byte (or negotiated major) not understood - case unknownFrameType // frame type tag not recognized - case oversizeLength // declared frame length exceeds the configured cap - case truncatedFrame // stream ended mid-frame - case malformedPayload // a frame body failed to decode - case manifestMismatch // accumulated bytes do not match the header manifest - case unknownMessage // a chunk/end for a message with no live header - case duplicateMessage // a second itemBegin for a live message id - case badSequence // a chunk out of sequence - case cancelled // the message was cancelled mid-flight - } - - public var code: Code - - public init(_ code: Code) { - self.code = code - } -} - -extension LinkProtocolError: LocalizedError { - public var errorDescription: String? { - switch code { - case .badMagic: return "Not a device-link stream." - case .unsupportedVersion: return "The other device speaks an incompatible link version." - case .unknownFrameType: return "Received an unrecognized message." - case .oversizeLength: return "A message exceeded the allowed size." - case .truncatedFrame: return "The connection ended mid-transfer." - case .malformedPayload: return "A message was malformed." - case .manifestMismatch: return "A transfer did not match its declared size." - case .unknownMessage: return "Received data for an unknown transfer." - case .duplicateMessage: return "Received a duplicate transfer." - case .badSequence: return "A transfer arrived out of order." - case .cancelled: return "The transfer was cancelled." - } - } -} diff --git a/DeviceLinkKit/Sources/DeviceLinkProtocol/LinkPump.swift b/DeviceLinkKit/Sources/DeviceLinkProtocol/LinkPump.swift deleted file mode 100644 index cc5ff89..0000000 --- a/DeviceLinkKit/Sources/DeviceLinkProtocol/LinkPump.swift +++ /dev/null @@ -1,59 +0,0 @@ -import Foundation - -/// The single synchronous bridge between `LinkItem`s and the raw bytes on a channel. Composes the -/// frame encoder, codec, decoder, and inbound assembler so a transport never re-wires them: call -/// `outbound(_:)` to get the byte buffers to write, and `ingest(_:)` on each received buffer to get -/// back completed items / control frames. Pure (no I/O); the transport owns the async channel and its -/// own serial context, so the pump is a plain `mutating struct`, one per connection. -public struct LinkPump { - private let encoder: FrameStreamEncoder - private var decoder: FrameDecoder - private var assembler: InboundAssembler - - /// A completed inbound result from `ingest`. - public enum Inbound: Equatable, Sendable { - case item(LinkItem) - case control(Frame) // hello / ack / error — for the transport's handshake/ack logic - } - - public init(chunkByteBound: Int = LinkProtocol.defaultChunkByteBound, - maxFrameLength: Int = LinkProtocol.defaultMaxFrameLength) { - self.encoder = FrameStreamEncoder(chunkByteBound: chunkByteBound) - self.decoder = FrameDecoder(maxFrameLength: maxFrameLength) - self.assembler = InboundAssembler() - } - - // MARK: Outbound - - /// The ordered encoded byte buffers for an item (one complete encoded frame each). - public func outbound(_ item: LinkItem) throws -> [Data] { - try encoder.frames(for: item).map { try LinkCodec.encode($0) } - } - - /// Encode a single control frame (hello/ack/error) to bytes. - public func outbound(control frame: Frame) throws -> Data { - try LinkCodec.encode(frame) - } - - // MARK: Inbound - - /// Push received bytes; return any completed inbound results. Throws a typed `LinkProtocolError` - /// on a malformed/violating stream. - public mutating func ingest(_ data: Data) throws -> [Inbound] { - decoder.push(data) - var out: [Inbound] = [] - while let frame = try decoder.next() { - switch try assembler.consume(frame) { - case .item(let item): out.append(.item(item)) - case .control(let f): out.append(.control(f)) - case .none: break - } - } - return out - } - - /// Assert the stream ended cleanly (no partial frame buffered). Throws `.truncatedFrame` otherwise. - public func finish() throws { - try decoder.close() - } -} diff --git a/DeviceLinkKit/Sources/DeviceLinkProtocol/LinkUTI.swift b/DeviceLinkKit/Sources/DeviceLinkProtocol/LinkUTI.swift deleted file mode 100644 index 24f140f..0000000 --- a/DeviceLinkKit/Sources/DeviceLinkProtocol/LinkUTI.swift +++ /dev/null @@ -1,14 +0,0 @@ -import Foundation - -/// Shared UTI string constants so both ends label representations identically (the manifest keys in an -/// `ItemHeader` and the `uti` in a `ChunkFrame` must match across devices). Deliberately mirrors the -/// Mac's `ClipboardUTI` values; kept as plain strings so this package stays AppKit/UIKit-free. -public enum LinkUTI { - public static let plainText = "public.utf8-plain-text" - public static let rtf = "public.rtf" - public static let png = "public.png" - public static let tiff = "public.tiff" - public static let fileURL = "public.file-url" - public static let url = "public.url" - public static let color = "com.apple.cocoa.pasteboard.color" -} diff --git a/DeviceLinkKit/Sources/DeviceLinkProtocol/ProtocolVersion.swift b/DeviceLinkKit/Sources/DeviceLinkProtocol/ProtocolVersion.swift deleted file mode 100644 index e4164f4..0000000 --- a/DeviceLinkKit/Sources/DeviceLinkProtocol/ProtocolVersion.swift +++ /dev/null @@ -1,46 +0,0 @@ -import Foundation - -/// The negotiated protocol version (carried in `hello`). Major bumps are breaking — a peer with a -/// different major is refused at the handshake rather than mis-parsed. A newer minor is accepted -/// (additive, optional fields default). -public struct ProtocolVersion: Equatable, Sendable, Codable { - public var major: UInt16 - public var minor: UInt16 - - public init(major: UInt16, minor: UInt16) { - self.major = major - self.minor = minor - } - - /// Compatible iff the major versions match. The receiver tolerates a peer on any minor. - public func isCompatible(with other: ProtocolVersion) -> Bool { - major == other.major - } -} - -/// A device's identity on the link: a stable id plus a human-readable name (shown in pairing UI). -public struct DeviceIdentity: Equatable, Sendable, Codable { - public var id: String - public var name: String - - public init(id: String, name: String) { - self.id = id - self.name = name - } -} - -/// Protocol-wide constants. Distinct from `ProtocolVersion` (the negotiated semantic version): these -/// are the wire-format/codec knobs. -public enum LinkProtocol { - /// The semantic protocol version this build speaks. - public static let version = ProtocolVersion(major: 1, minor: 0) - - /// Default upper bound on a single `chunk` frame's representation bytes. Senders SHOULD split a - /// representation larger than this into multiple chunks. (Tunable; not part of the wire contract.) - public static let defaultChunkByteBound = 256 * 1024 - - /// Default hard cap on a single decoded frame's declared length — a guard against an oversize-length - /// stream consuming unbounded memory. Large representations are *many* chunks, so no single frame is - /// huge. (Tunable; not part of the wire contract.) - public static let defaultMaxFrameLength = 8 * 1024 * 1024 -} diff --git a/DeviceLinkKit/Tests/DeviceLinkMirrorTests/MovedItemStoreTests.swift b/DeviceLinkKit/Tests/DeviceLinkMirrorTests/MovedItemStoreTests.swift deleted file mode 100644 index 9a8a28e..0000000 --- a/DeviceLinkKit/Tests/DeviceLinkMirrorTests/MovedItemStoreTests.swift +++ /dev/null @@ -1,93 +0,0 @@ -import XCTest -import DeviceLinkProtocol -@testable import DeviceLinkMirror - -final class MovedItemStoreTests: XCTestCase { - - private var dir: URL! - override func setUpWithError() throws { - dir = FileManager.default.temporaryDirectory.appendingPathComponent("tfs-mirror-\(UUID().uuidString)") - } - override func tearDownWithError() throws { try? FileManager.default.removeItem(at: dir) } - - private func blobCount() -> Int { - (try? FileManager.default.contentsOfDirectory(at: dir.appendingPathComponent("blobs"), includingPropertiesForKeys: nil))?.count ?? 0 - } - - private func textItem(_ s: String, at t: TimeInterval, direction: MoveDirection = .received) -> MovedItem { - let link = LinkItem(messageID: UUID(), kind: .text, representations: [LinkUTI.plainText: Data(s.utf8)], - origin: DeviceIdentity(id: "mac", name: "Mac")) - return MovedItem.from(link, direction: direction, at: Date(timeIntervalSince1970: t)) - } - - // MARK: Mapping - - func testMappingTextTitleAndReps() { - let item = textItem("hello\nworld", at: 1) - XCTAssertEqual(item.kind, .text) - XCTAssertEqual(item.title, "hello") - XCTAssertEqual(item.peerName, "Mac") - XCTAssertEqual(item.representations[LinkUTI.plainText], Data("hello\nworld".utf8)) - } - - func testMappingFileTitle() { - let link = LinkItem(messageID: UUID(), kind: .file, representations: ["public.data": Data([1, 2, 3])], - suggestedName: "report.pdf") - let item = MovedItem.from(link, direction: .sent, at: Date(timeIntervalSince1970: 1)) - XCTAssertEqual(item.title, "report.pdf") - XCTAssertEqual(item.direction, .sent) - } - - // MARK: Store - - func testInsertListNewestFirst() { - let store = MovedItemStore(directory: dir) - store.insert(textItem("old", at: 100)) - store.insert(textItem("new", at: 200)) - XCTAssertEqual(store.list().map(\.title), ["new", "old"]) - } - - func testBytesSurviveReload() { - let bytes = Data((0..<5000).map { UInt8($0 & 0xff) }) - let link = LinkItem(messageID: UUID(), kind: .image, representations: [LinkUTI.png: bytes]) - let store = MovedItemStore(directory: dir) - store.insert(MovedItem.from(link, direction: .received, at: Date(timeIntervalSince1970: 1))) - - let reloaded = MovedItemStore(directory: dir) - XCTAssertEqual(reloaded.list().first?.representations[LinkUTI.png], bytes) - } - - func testReplaceBySameID() { - let store = MovedItemStore(directory: dir) - var item = textItem("first", at: 1) - store.insert(item) - item.title = "second" - store.insert(item) // same id - XCTAssertEqual(store.count, 1) - XCTAssertEqual(store.list().first?.title, "second") - } - - func testRemoveAndClearDeleteBlobs() { - let store = MovedItemStore(directory: dir) - let a = textItem("a", at: 1) - store.insert(a) - store.insert(textItem("b", at: 2)) - XCTAssertEqual(blobCount(), 2) - store.remove(id: a.id) - XCTAssertEqual(store.count, 1) - XCTAssertEqual(blobCount(), 1) - store.clear() - XCTAssertEqual(store.count, 0) - XCTAssertEqual(blobCount(), 0) - } - - func testCountCapEvictsOldestAndDeletesBlobs() { - let store = MovedItemStore(directory: dir, maxCount: 2) - store.insert(textItem("a", at: 100)) - store.insert(textItem("b", at: 200)) - store.insert(textItem("c", at: 300)) // evicts "a" - XCTAssertEqual(store.list().map(\.title), ["c", "b"]) - XCTAssertEqual(store.count, 2) - XCTAssertEqual(blobCount(), 2, "evicted item's blob is deleted") - } -} diff --git a/DeviceLinkKit/Tests/DeviceLinkPairingTests/PairingTests.swift b/DeviceLinkKit/Tests/DeviceLinkPairingTests/PairingTests.swift deleted file mode 100644 index 58ba6d8..0000000 --- a/DeviceLinkKit/Tests/DeviceLinkPairingTests/PairingTests.swift +++ /dev/null @@ -1,58 +0,0 @@ -import XCTest -import CryptoKit -@testable import DeviceLinkPairing - -/// The shared pairing crypto: code format, and the code-authenticated X25519 confirmation including the -/// MITM-resistance property and role independence. -final class PairingTests: XCTestCase { - - // MARK: Code - - func testCodeGenerationFormat() { - let code = PairingCode.generate() - XCTAssertEqual(code.count, 8) - XCTAssertTrue(code.allSatisfy { $0.isNumber }) - XCTAssertTrue(PairingCode.isValid(code)) - } - - func testCodeValidation() { - XCTAssertTrue(PairingCode.isValid("12345678")) - XCTAssertFalse(PairingCode.isValid("1234567")) - XCTAssertFalse(PairingCode.isValid("1234567a")) - XCTAssertFalse(PairingCode.isValid("123456789")) - } - - // MARK: Handshake - - func testSameCodeBothSidesAgree() throws { - let mac = PairingHandshake() - let phone = PairingHandshake() - let code = "12345678" - let kMac = try mac.confirmationKey(peerPublicKey: phone.publicKey, code: code) - let kPhone = try phone.confirmationKey(peerPublicKey: mac.publicKey, code: code) - - let macConfirm = mac.confirmation(kMac, label: "mac→phone") - XCTAssertTrue(phone.verify(macConfirm, key: kPhone, label: "mac→phone")) - let phoneConfirm = phone.confirmation(kPhone, label: "phone→mac") - XCTAssertTrue(mac.verify(phoneConfirm, key: kMac, label: "phone→mac")) - } - - func testDifferentCodeDefeatsMITM() throws { - let mac = PairingHandshake() - let phone = PairingHandshake() - let kMac = try mac.confirmationKey(peerPublicKey: phone.publicKey, code: "12345678") - let kPhoneWrong = try phone.confirmationKey(peerPublicKey: mac.publicKey, code: "87654321") - let macConfirm = mac.confirmation(kMac, label: "mac→phone") - XCTAssertFalse(phone.verify(macConfirm, key: kPhoneWrong, label: "mac→phone"), - "a different code must fail confirmation — this is the MITM defense") - } - - func testRoleIndependentDerivation() throws { - let mac = PairingHandshake() - let phone = PairingHandshake() - let code = "55554444" - let kMac = try mac.confirmationKey(peerPublicKey: phone.publicKey, code: code) - let kPhone = try phone.confirmationKey(peerPublicKey: mac.publicKey, code: code) - XCTAssertEqual(mac.confirmation(kMac, label: "x"), phone.confirmation(kPhone, label: "x")) - } -} diff --git a/DeviceLinkKit/Tests/DeviceLinkPairingTests/QRPairingTests.swift b/DeviceLinkKit/Tests/DeviceLinkPairingTests/QRPairingTests.swift deleted file mode 100644 index 40133d2..0000000 --- a/DeviceLinkKit/Tests/DeviceLinkPairingTests/QRPairingTests.swift +++ /dev/null @@ -1,136 +0,0 @@ -import XCTest -import DeviceLinkProtocol -@testable import DeviceLinkPairing - -/// QR pairing: the payload string codec, and the authenticated exchange ending in mutual pinning -/// (including MITM and tamper resistance). -final class QRPairingTests: XCTestCase { - - private let host = DeviceIdentity(id: "mac-1", name: "Mac") - private let joiner = DeviceIdentity(id: "phone-1", name: "iPhone") - private let hostSPKI = Data((0..<32).map { UInt8($0) }) - private let joinerSPKI = Data((0..<32).map { UInt8(255 - $0) }) - - // MARK: Payload - - func testPayloadRoundTrips() throws { - let secret = PairingQRPayload.makeSecret() - let payload = PairingQRPayload(device: host, secret: secret, spkiFingerprint: hostSPKI) - let decoded = try PairingQRPayload(string: payload.encodedString()) - XCTAssertEqual(decoded, payload) - XCTAssertEqual(decoded.device, host) - XCTAssertEqual(decoded.secret, secret) - XCTAssertEqual(decoded.spkiFingerprint, hostSPKI) - } - - func testSecretIs32Bytes() { - XCTAssertEqual(PairingQRPayload.makeSecret().count, 32) - // Two secrets should (overwhelmingly) differ. - XCTAssertNotEqual(PairingQRPayload.makeSecret(), PairingQRPayload.makeSecret()) - } - - func testBadSchemeRejected() { - XCTAssertThrowsError(try PairingQRPayload(string: "https://example.com/x")) { - XCTAssertEqual($0 as? PairingQRError, .badScheme) - } - } - - func testBadVersionRejected() throws { - var payload = PairingQRPayload(device: host, secret: PairingQRPayload.makeSecret(), spkiFingerprint: hostSPKI) - payload.version = 99 - XCTAssertThrowsError(try PairingQRPayload(string: payload.encodedString())) { - XCTAssertEqual($0 as? PairingQRError, .unsupportedVersion) - } - } - - func testV2EndpointRoundTrips() throws { - let secret = PairingQRPayload.makeSecret() - let payload = PairingQRPayload(device: host, secret: secret, spkiFingerprint: hostSPKI, - addresses: ["10.0.0.21", "2a06:c701::1"], port: 52344) - let decoded = try PairingQRPayload(string: payload.encodedString()) - XCTAssertEqual(decoded, payload) - XCTAssertEqual(decoded.version, PairingQRPayload.currentVersion) - XCTAssertEqual(decoded.addresses, ["10.0.0.21", "2a06:c701::1"]) - XCTAssertEqual(decoded.port, 52344) - XCTAssertTrue(decoded.hasEndpoint) - } - - func testV1BackCompatDecodesWithNoEndpoint() throws { - // A v1 payload (no addresses/port) must still decode — the scanner then falls back to discovery. - let secret = PairingQRPayload.makeSecret() - let v1 = PairingQRPayload(device: host, secret: secret, spkiFingerprint: hostSPKI, version: 1) - let decoded = try PairingQRPayload(string: v1.encodedString()) - XCTAssertEqual(decoded.version, 1) - XCTAssertTrue(decoded.addresses.isEmpty) - XCTAssertNil(decoded.port) - XCTAssertFalse(decoded.hasEndpoint) - XCTAssertEqual(decoded, v1) - } - - func testV2WithEmptyAddressesDecodesWithoutEndpoint() throws { - let payload = PairingQRPayload(device: host, secret: PairingQRPayload.makeSecret(), spkiFingerprint: hostSPKI) - XCTAssertEqual(payload.version, PairingQRPayload.currentVersion) - let decoded = try PairingQRPayload(string: payload.encodedString()) - XCTAssertTrue(decoded.addresses.isEmpty) - XCTAssertNil(decoded.port) - XCTAssertFalse(decoded.hasEndpoint) - XCTAssertEqual(decoded, payload) - } - - func testLocalAddressesAreRoutableAndCapped() { - let addrs = LocalAddresses.current(limit: 4) - XCTAssertLessThanOrEqual(addrs.count, 4) - for a in addrs { - XCTAssertFalse(a.hasPrefix("169.254."), "link-local IPv4 must be excluded") - XCTAssertFalse(a.lowercased().hasPrefix("fe80:"), "link-local IPv6 must be excluded") - XCTAssertNotEqual(a, "127.0.0.1") - XCTAssertNotEqual(a, "::1") - } - } - - // MARK: Exchange - - /// Drive a full host↔joiner exchange and return the two results. - private func runExchange(hostSecret: Data, joinerSecret: Data) throws -> (joiner: PairingExchange.Result?, host: PairingExchange.Result?) { - var j = PairingExchange(role: .joiner, secret: joinerSecret, identity: joiner, spkiFingerprint: joinerSPKI) - var h = PairingExchange(role: .host, secret: hostSecret, identity: host, spkiFingerprint: hostSPKI) - let m1 = j.start()! - let (m2, r2) = try h.consume(m1) - XCTAssertNil(r2) - let (m3, rJoiner) = try j.consume(m2!) - let (_, rHost) = m3 != nil ? try h.consume(m3!) : (nil, nil) - return (rJoiner, rHost) - } - - func testMatchingSecretMutuallyPins() throws { - let secret = PairingQRPayload.makeSecret() - let (rJoiner, rHost) = try runExchange(hostSecret: secret, joinerSecret: secret) - - guard case let .pinned(pinnedHostID, pinnedHostSPKI) = rJoiner else { return XCTFail("joiner not pinned") } - XCTAssertEqual(pinnedHostID, host) - XCTAssertEqual(pinnedHostSPKI, hostSPKI) - - guard case let .pinned(pinnedJoinerID, pinnedJoinerSPKI) = rHost else { return XCTFail("host not pinned") } - XCTAssertEqual(pinnedJoinerID, joiner) - XCTAssertEqual(pinnedJoinerSPKI, joinerSPKI) - } - - func testDifferentSecretDefeatsMITM() throws { - let (rJoiner, _) = try runExchange(hostSecret: PairingQRPayload.makeSecret(), - joinerSecret: PairingQRPayload.makeSecret()) - XCTAssertEqual(rJoiner, .failed, "a different secret must fail confirmation") - } - - func testTamperedConfirmationFails() throws { - let secret = PairingQRPayload.makeSecret() - var j = PairingExchange(role: .joiner, secret: secret, identity: joiner, spkiFingerprint: joinerSPKI) - var h = PairingExchange(role: .host, secret: secret, identity: host, spkiFingerprint: hostSPKI) - let m1 = j.start()! - let (m2, _) = try h.consume(m1) - guard case let .hostHello(e, i, s, c) = m2! else { return XCTFail("expected hostHello") } - var bad = c; bad[0] ^= 0xFF - let tampered = PairingMessage.hostHello(ephemeral: e, identity: i, spki: s, confirm: bad) - let (_, result) = try j.consume(tampered) - XCTAssertEqual(result, .failed, "a tampered confirmation must fail") - } -} diff --git a/DeviceLinkKit/Tests/DeviceLinkProtocolTests/FrameStreamEncoderTests.swift b/DeviceLinkKit/Tests/DeviceLinkProtocolTests/FrameStreamEncoderTests.swift deleted file mode 100644 index 497adc6..0000000 --- a/DeviceLinkKit/Tests/DeviceLinkProtocolTests/FrameStreamEncoderTests.swift +++ /dev/null @@ -1,74 +0,0 @@ -import XCTest -@testable import DeviceLinkProtocol - -/// The send-side encoder + the closed encode→decode round-trip against InboundAssembler. -final class FrameStreamEncoderTests: XCTestCase { - - private let id = UUID(uuidString: "DDDDDDDD-0000-0000-0000-000000000001")! - - /// Drive every encoded frame through an assembler and return the reassembled item. - private func reassemble(_ frames: [Frame]) throws -> LinkItem? { - var assembler = InboundAssembler() - var out: LinkItem? - for f in frames { - if case let .item(item) = try assembler.consume(f) { out = item } - } - return out - } - - func testSmallSingleRepEncoding() { - let item = LinkItem(messageID: id, kind: .text, representations: [LinkUTI.plainText: Data("hi".utf8)]) - let frames = FrameStreamEncoder().frames(for: item) - XCTAssertEqual(frames.count, 3) - guard case let .itemBegin(header) = frames[0] else { return XCTFail("expected itemBegin") } - XCTAssertEqual(header.manifest[LinkUTI.plainText], 2) - guard case let .chunk(c) = frames[1] else { return XCTFail("expected chunk") } - XCTAssertEqual(c.seq, 0) - XCTAssertEqual(c.bytes, Data("hi".utf8)) - guard case .itemEnd = frames[2] else { return XCTFail("expected itemEnd") } - } - - func testLargeRepIsBoundedAndConsecutive() { - let big = Data((0..<1000).map { UInt8($0 & 0xff) }) - let item = LinkItem(messageID: id, kind: .file, representations: [LinkUTI.fileURL: big], suggestedName: "x.bin") - let bound = 256 - let frames = FrameStreamEncoder(chunkByteBound: bound).frames(for: item) - let chunks: [ChunkFrame] = frames.compactMap { if case let .chunk(c) = $0 { return c }; return nil } - XCTAssertEqual(chunks.count, (1000 + bound - 1) / bound) // ceil - XCTAssertEqual(chunks.map(\.seq), Array(0.. bound → multi-chunk - LinkUTI.plainText: Data("caption".utf8), // small - ], - suggestedName: "pic.png", - capturedAt: nil, - origin: DeviceIdentity(id: "dev", name: "iPhone")) - let frames = FrameStreamEncoder(chunkByteBound: 128).frames(for: item) - let back = try reassemble(frames) - XCTAssertEqual(back, item) - } - - func testRoundTripEmptyRepresentation() throws { - let item = LinkItem(messageID: id, kind: .text, - representations: [LinkUTI.plainText: Data(), LinkUTI.url: Data("u".utf8)]) - let frames = FrameStreamEncoder().frames(for: item) - let back = try reassemble(frames) - XCTAssertEqual(back, item, "empty representation must survive the round-trip") - XCTAssertEqual(back?.representations[LinkUTI.plainText], Data()) - } -} diff --git a/DeviceLinkKit/Tests/DeviceLinkProtocolTests/InboundAssemblerTests.swift b/DeviceLinkKit/Tests/DeviceLinkProtocolTests/InboundAssemblerTests.swift deleted file mode 100644 index abfe86f..0000000 --- a/DeviceLinkKit/Tests/DeviceLinkProtocolTests/InboundAssemblerTests.swift +++ /dev/null @@ -1,125 +0,0 @@ -import XCTest -@testable import DeviceLinkProtocol - -/// Reassembly contract: emit complete items, interleave by message id, and reject every protocol -/// violation (mismatch, unknown message, duplicate, bad sequence), discarding state on failure/cancel. -final class InboundAssemblerTests: XCTestCase { - - private let a = UUID(uuidString: "AAAAAAAA-0000-0000-0000-000000000001")! - private let b = UUID(uuidString: "BBBBBBBB-0000-0000-0000-000000000002")! - - /// One-chunk text item frames. - private func textFrames(_ id: UUID, _ text: String) -> [Frame] { - let data = Data(text.utf8) - let header = ItemHeader(messageID: id, kind: .text, manifest: [LinkUTI.plainText: UInt32(data.count)]) - return [ - .itemBegin(header), - .chunk(ChunkFrame(messageID: id, uti: LinkUTI.plainText, seq: 0, bytes: data)), - .itemEnd(id), - ] - } - - func testCompleteItemEmitted() throws { - var assembler = InboundAssembler() - var emitted: LinkItem? - for frame in textFrames(a, "hello world") { - if case let .item(item) = try assembler.consume(frame) { emitted = item } - } - XCTAssertEqual(emitted?.messageID, a) - XCTAssertEqual(emitted?.kind, .text) - XCTAssertEqual(emitted?.representations[LinkUTI.plainText], Data("hello world".utf8)) - XCTAssertEqual(assembler.inFlightCount, 0) - } - - func testInterleavedSmallAheadOfLarge() throws { - // B is a 2-chunk "file"; A is a small text item that arrives and completes mid-B. - let big = Data((0..<5000).map { UInt8($0 & 0xff) }) - let half = big.count / 2 - let bHeader = ItemHeader(messageID: b, kind: .file, manifest: [LinkUTI.fileURL: UInt32(big.count)]) - let aData = Data("ping".utf8) - let aHeader = ItemHeader(messageID: a, kind: .text, manifest: [LinkUTI.plainText: UInt32(aData.count)]) - - let sequence: [Frame] = [ - .itemBegin(bHeader), - .itemBegin(aHeader), - .chunk(ChunkFrame(messageID: a, uti: LinkUTI.plainText, seq: 0, bytes: aData)), - .chunk(ChunkFrame(messageID: b, uti: LinkUTI.fileURL, seq: 0, bytes: big.prefix(half))), - .itemEnd(a), // A completes while B is still mid-flight - .chunk(ChunkFrame(messageID: b, uti: LinkUTI.fileURL, seq: 1, bytes: big.suffix(from: big.startIndex + half))), - .itemEnd(b), - ] - - var assembler = InboundAssembler() - var emitted: [LinkItem] = [] - for frame in sequence { - if case let .item(item) = try assembler.consume(frame) { emitted.append(item) } - } - XCTAssertEqual(emitted.map(\.messageID), [a, b], "A should emit before B") - XCTAssertEqual(emitted.first?.representations[LinkUTI.plainText], aData) - XCTAssertEqual(emitted.last?.representations[LinkUTI.fileURL], big) - XCTAssertEqual(assembler.inFlightCount, 0) - } - - func testByteCountMismatchRejectedAndStateDiscarded() { - var assembler = InboundAssembler() - // Manifest says 10 bytes; we send 3, then end. - let header = ItemHeader(messageID: a, kind: .text, manifest: [LinkUTI.plainText: 10]) - XCTAssertNoThrow(try assembler.consume(.itemBegin(header))) - XCTAssertNoThrow(try assembler.consume(.chunk(ChunkFrame(messageID: a, uti: LinkUTI.plainText, seq: 0, bytes: Data("abc".utf8))))) - XCTAssertThrowsError(try assembler.consume(.itemEnd(a))) { - XCTAssertEqual(($0 as? LinkProtocolError)?.code, .manifestMismatch) - } - XCTAssertEqual(assembler.inFlightCount, 0, "failed message must be discarded") - } - - func testOverflowBeyondManifestRejected() { - var assembler = InboundAssembler() - let header = ItemHeader(messageID: a, kind: .text, manifest: [LinkUTI.plainText: 2]) - XCTAssertNoThrow(try assembler.consume(.itemBegin(header))) - XCTAssertThrowsError(try assembler.consume(.chunk(ChunkFrame(messageID: a, uti: LinkUTI.plainText, seq: 0, bytes: Data("abcdef".utf8))))) { - XCTAssertEqual(($0 as? LinkProtocolError)?.code, .manifestMismatch) - } - XCTAssertEqual(assembler.inFlightCount, 0) - } - - func testCancelDiscardsPartialState() throws { - var assembler = InboundAssembler() - let header = ItemHeader(messageID: a, kind: .text, manifest: [LinkUTI.plainText: 10]) - _ = try assembler.consume(.itemBegin(header)) - _ = try assembler.consume(.chunk(ChunkFrame(messageID: a, uti: LinkUTI.plainText, seq: 0, bytes: Data("abc".utf8)))) - let out = try assembler.consume(.cancel(a)) - XCTAssertEqual(out, .none) - XCTAssertEqual(assembler.inFlightCount, 0) - } - - func testChunkForUnknownMessageRejected() { - var assembler = InboundAssembler() - XCTAssertThrowsError(try assembler.consume(.chunk(ChunkFrame(messageID: a, uti: LinkUTI.plainText, seq: 0, bytes: Data("x".utf8))))) { - XCTAssertEqual(($0 as? LinkProtocolError)?.code, .unknownMessage) - } - } - - func testDuplicateItemBeginRejected() throws { - var assembler = InboundAssembler() - let header = ItemHeader(messageID: a, kind: .text, manifest: [LinkUTI.plainText: 1]) - _ = try assembler.consume(.itemBegin(header)) - XCTAssertThrowsError(try assembler.consume(.itemBegin(header))) { - XCTAssertEqual(($0 as? LinkProtocolError)?.code, .duplicateMessage) - } - } - - func testBadSequenceRejected() throws { - var assembler = InboundAssembler() - let header = ItemHeader(messageID: a, kind: .image, manifest: [LinkUTI.png: 100]) - _ = try assembler.consume(.itemBegin(header)) - XCTAssertThrowsError(try assembler.consume(.chunk(ChunkFrame(messageID: a, uti: LinkUTI.png, seq: 1, bytes: Data(repeating: 0, count: 10))))) { - XCTAssertEqual(($0 as? LinkProtocolError)?.code, .badSequence) - } - } - - func testControlFramesPassThrough() throws { - var assembler = InboundAssembler() - let hello = Frame.hello(DeviceIdentity(id: "x", name: "y"), ProtocolVersion(major: 1, minor: 0)) - XCTAssertEqual(try assembler.consume(hello), .control(hello)) - } -} diff --git a/DeviceLinkKit/Tests/DeviceLinkProtocolTests/LinkCodecTests.swift b/DeviceLinkKit/Tests/DeviceLinkProtocolTests/LinkCodecTests.swift deleted file mode 100644 index 63cb20a..0000000 --- a/DeviceLinkKit/Tests/DeviceLinkProtocolTests/LinkCodecTests.swift +++ /dev/null @@ -1,130 +0,0 @@ -import XCTest -@testable import DeviceLinkProtocol - -/// Codec contract: round-trip every frame, reassemble across reads, preserve trailing bytes, and reject -/// bad-magic / unknown-tag / oversize / truncated streams with the right typed error. -final class LinkCodecTests: XCTestCase { - - private let id = UUID(uuidString: "11111111-2222-3333-4444-555555555555")! - - private func allFrames() -> [Frame] { - [ - .hello(DeviceIdentity(id: "device-1", name: "Amit's Mac"), ProtocolVersion(major: 1, minor: 0)), - .ack(id), - .error(.manifestMismatch), - .itemBegin(ItemHeader(messageID: id, kind: .text, - manifest: [LinkUTI.plainText: 5], - suggestedName: "note.txt", - capturedAt: nil, - origin: DeviceIdentity(id: "device-2", name: "iPhone"))), - .chunk(ChunkFrame(messageID: id, uti: LinkUTI.plainText, seq: 0, bytes: Data("hello".utf8))), - .itemEnd(id), - .cancel(id), - ] - } - - /// Build a raw envelope by hand, for the adversarial (malformed) cases. - private func rawEnvelope(type: UInt8, - declaredLength: UInt32, - payload: Data = Data(), - magic: [UInt8] = LinkCodec.magic, - version: UInt8 = LinkCodec.wireFormatVersion) -> Data { - var d = Data(magic) - d.append(version) - d.append(type) - d.append(UInt8((declaredLength >> 24) & 0xff)) - d.append(UInt8((declaredLength >> 16) & 0xff)) - d.append(UInt8((declaredLength >> 8) & 0xff)) - d.append(UInt8(declaredLength & 0xff)) - d.append(payload) - return d - } - - func testRoundTripEveryFrame() throws { - for frame in allFrames() { - let bytes = try LinkCodec.encode(frame) - var decoder = FrameDecoder() - decoder.push(bytes) - let decoded = try decoder.next() - XCTAssertEqual(decoded, frame, "round-trip mismatch for \(frame)") - XCTAssertNil(try decoder.next(), "decoder should be drained after one frame") - try decoder.close() - } - } - - func testPartialBufferReassembles() throws { - let frame = Frame.chunk(ChunkFrame(messageID: id, uti: LinkUTI.png, seq: 0, bytes: Data(repeating: 7, count: 5000))) - let bytes = try LinkCodec.encode(frame) - var decoder = FrameDecoder() - let split = bytes.count / 3 - decoder.push(bytes.prefix(split)) - XCTAssertNil(try decoder.next(), "should need more bytes") - decoder.push(bytes.suffix(from: bytes.startIndex + split)) - XCTAssertEqual(try decoder.next(), frame) - XCTAssertNil(try decoder.next()) - } - - func testTrailingBytesPreservedAcrossFrames() throws { - let a = Frame.ack(id) - let b = Frame.itemEnd(id) - var stream = Data() - stream.append(try LinkCodec.encode(a)) - stream.append(try LinkCodec.encode(b)) - var decoder = FrameDecoder() - decoder.push(stream) - XCTAssertEqual(try decoder.next(), a) - XCTAssertEqual(try decoder.next(), b) - XCTAssertNil(try decoder.next()) - } - - func testBadMagicRejected() { - let bytes = rawEnvelope(type: FrameType.ack.rawValue, declaredLength: 0, magic: [0x00, 0x00, 0x00, 0x00]) - var decoder = FrameDecoder() - decoder.push(bytes) - XCTAssertThrowsError(try decoder.next()) { - XCTAssertEqual(($0 as? LinkProtocolError)?.code, .badMagic) - } - } - - func testUnknownFrameTypeRejected() { - let bytes = rawEnvelope(type: 99, declaredLength: 0) - var decoder = FrameDecoder() - decoder.push(bytes) - XCTAssertThrowsError(try decoder.next()) { - XCTAssertEqual(($0 as? LinkProtocolError)?.code, .unknownFrameType) - } - } - - func testOversizeLengthRejected() { - let bytes = rawEnvelope(type: FrameType.ack.rawValue, declaredLength: 100) - var decoder = FrameDecoder(maxFrameLength: 16) - decoder.push(bytes) - XCTAssertThrowsError(try decoder.next()) { - XCTAssertEqual(($0 as? LinkProtocolError)?.code, .oversizeLength) - } - } - - func testTruncatedFrameRejectedAtClose() throws { - let frame = Frame.chunk(ChunkFrame(messageID: id, uti: LinkUTI.plainText, seq: 0, bytes: Data("abcdef".utf8))) - let bytes = try LinkCodec.encode(frame) - var decoder = FrameDecoder() - decoder.push(bytes.dropLast(2)) // stream ends mid-frame - XCTAssertNil(try decoder.next(), "incomplete frame should not be emitted") - XCTAssertThrowsError(try decoder.close()) { - XCTAssertEqual(($0 as? LinkProtocolError)?.code, .truncatedFrame) - } - } - - func testChunkBytesSurviveExactly() throws { - let payload = Data((0..<1024).map { UInt8($0 & 0xff) }) - let frame = Frame.chunk(ChunkFrame(messageID: id, uti: LinkUTI.fileURL, seq: 42, bytes: payload)) - let bytes = try LinkCodec.encode(frame) - var decoder = FrameDecoder() - decoder.push(bytes) - guard case let .chunk(decoded)? = try decoder.next() else { return XCTFail("expected chunk") } - XCTAssertEqual(decoded.bytes, payload) - XCTAssertEqual(decoded.seq, 42) - XCTAssertEqual(decoded.uti, LinkUTI.fileURL) - XCTAssertEqual(decoded.messageID, id) - } -} diff --git a/DeviceLinkKit/Tests/DeviceLinkProtocolTests/LinkPumpTests.swift b/DeviceLinkKit/Tests/DeviceLinkProtocolTests/LinkPumpTests.swift deleted file mode 100644 index ce3ef95..0000000 --- a/DeviceLinkKit/Tests/DeviceLinkProtocolTests/LinkPumpTests.swift +++ /dev/null @@ -1,82 +0,0 @@ -import XCTest -@testable import DeviceLinkProtocol - -/// The pump's loopback fidelity under arbitrary buffer fragmentation — the end-to-end proof of the -/// encode→codec→decode→assemble stack. -final class LinkPumpTests: XCTestCase { - - private let id = UUID(uuidString: "EEEEEEEE-0000-0000-0000-000000000001")! - - private func sampleItem() -> LinkItem { - LinkItem(messageID: id, kind: .image, - representations: [ - LinkUTI.png: Data((0..<900).map { UInt8($0 & 0xff) }), // multi-chunk at small bound - LinkUTI.plainText: Data("caption".utf8), - ], - suggestedName: "p.png", - origin: DeviceIdentity(id: "d", name: "iPhone")) - } - - func testOutboundIsFramePerBufferAndIngestsToItem() throws { - var sender = LinkPump(chunkByteBound: 64) - var receiver = LinkPump() - let item = sampleItem() - let buffers = try sender.outbound(item) - XCTAssertGreaterThan(buffers.count, 3, "expected itemBegin + several chunks + itemEnd") - - var got: [LinkPump.Inbound] = [] - for b in buffers { got += try receiver.ingest(b) } - XCTAssertEqual(got, [.item(item)]) - } - - func testLoopbackConcatenated() throws { - var sender = LinkPump(chunkByteBound: 64) - var receiver = LinkPump() - let item = sampleItem() - let all = try sender.outbound(item).reduce(Data(), +) - let got = try receiver.ingest(all) - XCTAssertEqual(got, [.item(item)]) - } - - func testLoopbackReSplitAtArbitraryBoundaries() throws { - var sender = LinkPump(chunkByteBound: 64) - var receiver = LinkPump() - let item = sampleItem() - let all = try sender.outbound(item).reduce(Data(), +) - - // Re-slice into 37-byte pieces that don't align with frame boundaries. - var got: [LinkPump.Inbound] = [] - var offset = all.startIndex - while offset < all.endIndex { - let end = min(offset + 37, all.endIndex) - got += try receiver.ingest(all[offset../spec.md` (**35 capabilities** — the switcher core: `gesture-recognition`, `switcher-overlay`, `window-enumeration-and-raising`, `window-groups`, `command-tab-switcher`, `touch-input`, `native-gesture-config`, `spaces-rearrange-config`, `tunable-settings`, `menubar-app-shell`, `permissions-onboarding`, `first-run-onboarding`; the opt-in features: `runtime-gesture-ownership`, `launcher-overlay`, `launch-items`, `launch-actions`, `open-claude-here`, `favorites-editor`, `clipboard-history`, `dock-hover-detection`, `dock-preview-overlay` (the mouse-hover Dock previews); the device link: the `device-link-*` capabilities + `mac-qr-pairing`; keyboard language: `per-app-keyboard-language`, `per-site-keyboard-language`; and the shell/pipeline: `configuration-hub`, `app-branding`, `release-pipeline`). Every feature was a `change/` (proposal → design → spec delta → tasks), now in `openspec/changes/archive/`. **Before changing behavior, read the relevant spec; after changing behavior, update it.** The archived changes are an excellent design history: +This project was built spec-first with **OpenSpec**. The **canonical behavior** lives in `openspec/specs//spec.md` (**26 capabilities** — the switcher core: `gesture-recognition`, `switcher-overlay`, `window-enumeration-and-raising`, `window-groups`, `command-tab-switcher`, `touch-input`, `native-gesture-config`, `spaces-rearrange-config`, `tunable-settings`, `menubar-app-shell`, `permissions-onboarding`, `first-run-onboarding`; the opt-in features: `runtime-gesture-ownership`, `launcher-overlay`, `launch-items`, `launch-actions`, `open-claude-here`, `favorites-editor`, `clipboard-history`, `dock-hover-detection`, `dock-preview-overlay` (the mouse-hover Dock previews); keyboard language: `per-app-keyboard-language`, `per-site-keyboard-language`; and the shell/pipeline: `configuration-hub`, `app-branding`, `release-pipeline`). Every feature was a `change/` (proposal → design → spec delta → tasks), now in `openspec/changes/archive/`. **Before changing behavior, read the relevant spec; after changing behavior, update it.** The archived changes are an excellent design history: - **Switcher / window-raising internals:** `cross-space-windows`, `fix-focus-vacuum-on-raise`, `space-grid-navigation`, `fix-off-space-listing-and-focus` — read their `design.md` for the hard-won private-API details. - **Optional features:** `optional-space-row-gesture` (runtime gesture ownership — the scroll tap + Mission Control synthesis substrate), `four-finger-launcher` (the launcher, favorites model, launch strategies, dwell-to-arm), `launcher-two-finger-nav` (drop-to-two-finger navigation), `launcher-vertical-band-list` (the band strip as a vertical icon list on the left; vertical switches bands, horizontal crosses into the grid), and `clipboard-history` (the launcher's Clipboard band: a polled-`changeCount` recorder + on-disk store, paste-into-front, pin/edge-accel — its `design.md` covers the band-not-keyboard-picker decision). - **Shell & later features:** `configuration-hub` (folds the Settings/Favorites/Setup windows into one **Hub**) and `hub-in-switcher` (a Hub item in the switcher so it's reachable in accessory mode); `per-app-keyboard-language` / `per-site-keyboard-language` (auto-remember the input source per app, and per website inside browsers). @@ -142,7 +142,7 @@ Sources/ThreeFingerSwitcher/ ── ThreeFingerSwitcherCore library (AL FirstTouchWizardModel/View + WizardActs (the acts: live-touch demo, permission upgrades, lanes consent, playground, curtain), WizardContext (closure wiring), LanesLiveToast (post-re-login acknowledgment) Settings/ AppSettings (tunables + opt-ins, persisted), AppDataReset (the Danger-zone selective reset) - Hub/ the single configuration window (configuration-hub): HubView + sidebar pages (Overview / Bands / Switcher·Launcher·Clipboard·Keyboard-Language·Devices / Setup·General), + Hub/ the single configuration window (configuration-hub): HubView + sidebar pages (Overview / Bands / Switcher·Launcher·Clipboard·Keyboard-Language / Setup·General), BandsCanvas (the favorites "small IDE": sources → bands → items, AppearanceEditor icon/tint picker), HubControls / HubStyle (shared Liquid-Glass controls) KeyboardLanguage/ per-app & per-site keyboard language (opt-in): KeyboardLanguageStore (context-key → input-source map), KeyboardLanguagePolicy (pure activate/learn), KeyboardLanguageService (learn-on-deactivation / apply-on-context-change engine), InputSourceController + CarbonInputSourceController (Carbon TIS seam), ContextResolver + BrowserRegistry + ContextKey + HostNormalizer (per-site host context = bundleID|host), AXHostProvider / AppleEventsHostProvider (host readers, AX default + Apple Events opt-in), BrowserContextMonitor (within-browser host-change poll) — auto-remembers the input source per app, and per website (host root) inside browsers diff --git a/Resources/Info.plist b/Resources/Info.plist index f88caf3..2e225fb 100644 --- a/Resources/Info.plist +++ b/Resources/Info.plist @@ -28,15 +28,5 @@ NSInputMonitoringUsageDescription Reads raw trackpad touches to detect the three-finger swipe. - - NSLocalNetworkUsageDescription - Connects directly to your paired iPhone on your local network to move clipboard items and files between your devices. - - NSBonjourServices - - _tfslink._tcp - _tfspair._tcp - diff --git a/Sources/ThreeFingerSwitcher/App/AppCoordinator.swift b/Sources/ThreeFingerSwitcher/App/AppCoordinator.swift index f4fe214..4e2e6ab 100644 --- a/Sources/ThreeFingerSwitcher/App/AppCoordinator.swift +++ b/Sources/ThreeFingerSwitcher/App/AppCoordinator.swift @@ -2,7 +2,6 @@ import AppKit import Combine import ServiceManagement import SwiftUI -import DeviceLinkProtocol import os private let windowGroupsLog = Logger(subsystem: "ThreeFingerSwitcher", category: "WindowGroups") @@ -22,8 +21,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate private lazy var windowService = WindowService(mru: mru, focus: focus, settings: settings) private let thumbnails = ThumbnailService() private let overlay = OverlayController() - /// The transient on-receive notch HUD (success/failure feedback for inbound device-link items). - private lazy var receiveHUD = ReceiveHUDController() private let touchEngine = TouchEngine() private lazy var recognizer = GestureRecognizer(settings: settings) let spacesRearrange = SpacesRearrangeConfig() @@ -114,28 +111,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate private let clipboardStore = ClipboardStore.shared private lazy var clipboardMonitor = ClipboardMonitor(store: clipboardStore) - // Device link (opt-in; the iPhone↔Mac local-network bridge). Received items are adapted into the - // clipboard store (so they appear in the Clipboard band); the service is started/stopped by the - // `enableDeviceLink` toggle, like the clipboard recorder. Security (pinned TLS) is a pairing follow-up. - private var deviceLinkService: DeviceLinkService? - private let pairedDeviceStore = PairedDeviceStore(directory: PairedDeviceStore.defaultDirectory()) - /// Host coordinator for QR pairing (show a code on the Hub, accept a scanner, pin it). - private lazy var macPairingCoordinator = MacPairingCoordinator(store: pairedDeviceStore, identity: localDeviceIdentity) - /// Received-item adapter: files land in a `received/` inbox beside the clipboard store. - private lazy var linkInboundAdapter = LinkInboundAdapter( - inboxDirectory: ClipboardStore.defaultDirectory().appendingPathComponent("inbox", isDirectory: true)) - private let linkOutboundAdapter = LinkOutboundAdapter() - /// This Mac's stable link identity (id persisted so a peer's pin survives relaunches; name = host name). - private var localDeviceIdentity: DeviceIdentity { - let key = "deviceLinkLocalID" - let id = UserDefaults.standard.string(forKey: key) ?? { - let fresh = UUID().uuidString - UserDefaults.standard.set(fresh, forKey: key) - return fresh - }() - return DeviceIdentity(id: id, name: Host.current().localizedName ?? "Mac") - } - // Per-app keyboard language (opt-in; remembers and re-selects the input source per app/site). Gated // on its OWN toggle, independent of the switcher master enable. The store holds the learned // context-key → source map; the service ties it to the pure policy and the Carbon @@ -273,7 +248,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate observeCommandTabToggle() observeLauncherToggle() observeClipboardToggle() - observeDeviceLinkToggle() observeKeyboardLanguageToggle() observeKeyboardLanguagePerSiteToggle() observeKeyboardLanguageBrowserControlToggle() @@ -388,7 +362,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate overlay.hide() stopPreviewRefresh() launcherOverlay.cancel() - receiveHUD.hide() clipboardMonitor.stop() isEnabled = false settings.enabled = false @@ -1048,74 +1021,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate .store(in: &cancellables) } - // MARK: - Device link lifecycle - - /// React to the `enableDeviceLink` toggle (mirrors `observeClipboardToggle` — uses the emitted value, - /// since `@Published` fires in `willSet`). - private func observeDeviceLinkToggle() { - settings.$enableDeviceLink - .dropFirst() - .sink { [weak self] on in MainActor.assumeIsolated { self?.setDeviceLink(on) } } - .store(in: &cancellables) - } - - /// Start the device-link service when opted in AND the app is enabled; otherwise stop it. Received - /// items are adapted into the clipboard store (where the Clipboard band surfaces them, tagged by - /// device). `on` is the authoritative state (the toggle's emitted value, or a stable read at launch). - private func setDeviceLink(_ on: Bool) { - if on && isEnabled { - guard deviceLinkService == nil else { return } - let service = DeviceLinkService( - localIdentity: localDeviceIdentity, - staticKey: MacLocalIdentity.privateKey, - pinnedFingerprints: { [weak self] in self?.pairedDeviceStore.pinnedFingerprints() ?? [] }, - device: { [weak self] fingerprint in self?.pairedDeviceStore.device(forFingerprint: fingerprint) }) - service.onItem = { [weak self] item in self?.receiveLinkItem(item) } - do { - try service.start() - deviceLinkService = service - } catch { - // Local-network start can fail (e.g. permission not yet granted) — leave the service nil - // so a later toggle / permission grant retries. Surfaced on the Devices page. - deviceLinkService = nil - } - } else { - deviceLinkService?.stop() - deviceLinkService = nil - } - } - - /// A received item → `ClipboardEntry` (files written to the inbox) → the existing store. Runs on the - /// main queue (the service hops there before calling `onItem`). - private func receiveLinkItem(_ item: LinkItem) { - do { - let entry = try linkInboundAdapter.entry(from: item) - clipboardStore.insert(entry) - // Auto-paste: make the received item the system clipboard so it's immediately pasteable. Reuse - // the launcher's pasteboard writer (PNG+TIFF / color-hex / path fallbacks), without synthesizing - // a ⌘V. Then suppress the monitor's re-capture of THIS write by its `changeCount`, so the - // already-inserted `.peer(deviceName:)` entry keeps its origin/`capturedAt` and no duplicate is - // recorded. Suppression is a no-op when clipboard history is off (the monitor isn't polling). - LaunchService.writeToPasteboard(entry) - clipboardMonitor.suppressSelfWrite(changeCount: NSPasteboard.general.changeCount) - // Fire-and-forget feedback — the LAST, non-throwing step, so a HUD problem can never break - // receive/storage. On the failure path (a malformed representation / unwritable file, which - // `entry(from:)` throws) the same HUD surfaces what was previously a silent drop. - receiveHUD.show(kind: item.kind, from: item.origin?.name, success: true) - } catch { - receiveHUD.show(kind: item.kind, from: item.origin?.name, success: false) - } - } - - /// v1 outbound trigger: send the most recent clipboard entry to every online paired peer. Per-device - /// targeting (a picker over `send(_:to:)`) is a later change; `sendToAll` is the thin convenience over - /// the per-peer primitive until then. - private func sendLatestClipboardToDevices() { - guard let entry = clipboardStore.recentWindow(limit: 1).first, - let item = try? linkOutboundAdapter.linkItem(from: entry, origin: localDeviceIdentity) else { return } - deviceLinkService?.sendToAll(item) - } - /// Push the tunables (retention, poll interval, exclusions) into the store/monitor. Does NOT read /// the start/stop or pause toggles (those are driven by their emitted values to avoid the /// `@Published` willSet staleness). @@ -2215,11 +2120,6 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate ctx.onToggleOpenAtLogin = { [weak self] in self?.toggleOpenAtLogin() } ctx.onWriteDiagnostics = { [weak self] in self?.writeDiagnostics() } ctx.onCopyFocusLog = { [weak self] in self?.copyFocusLog() } - // Devices (device link). - ctx.pairedDevices = { [weak self] in self?.pairedDeviceStore.all() ?? [] } - ctx.onForgetDevice = { [weak self] id in self?.pairedDeviceStore.remove(id: id) } - ctx.onSendLatestToDevices = { [weak self] in self?.sendLatestClipboardToDevices() } - ctx.pairingCoordinator = macPairingCoordinator // Danger zone. ctx.onDangerZoneClear = { [weak self] selection in self?.dangerZoneClear(selection) } ctx.onRestoreAllGestures = { [weak self] in self?.restoreAllNativeGestures() } diff --git a/Sources/ThreeFingerSwitcher/Clipboard/ClipboardEntry.swift b/Sources/ThreeFingerSwitcher/Clipboard/ClipboardEntry.swift index 8776b54..6c2a616 100644 --- a/Sources/ThreeFingerSwitcher/Clipboard/ClipboardEntry.swift +++ b/Sources/ThreeFingerSwitcher/Clipboard/ClipboardEntry.swift @@ -50,15 +50,6 @@ extension ClipboardPayload: Codable { } } -/// Where an entry came from — a *device* provenance, distinct from `sourceApp` (the app that made a -/// local copy). Additive and backward-compatible: it is optional on `ClipboardEntry`, so an index -/// persisted before this existed loads with `origin == nil` (treated as local). Local pasteboard -/// capture leaves it unset; an item received over the device link is stamped `.peer`. -enum ClipboardOrigin: Codable, Equatable { - case local - case peer(deviceName: String?) -} - /// A single recorded clipboard item: stable identity + when/where it was copied + the kind + the /// representations needed for a faithful re-paste + a derived single-line `key` for the list column. struct ClipboardEntry: Codable, Equatable, Identifiable { @@ -78,9 +69,6 @@ struct ClipboardEntry: Codable, Equatable, Identifiable { /// Stable content fingerprint used for de-duplication (two copies with the same fingerprint are /// the same entry). Derived from the canonical representation at capture time. var fingerprint: String - /// Device provenance. Optional + additive: `nil` (absent in legacy indexes) and `.local` both mean - /// "captured on this Mac"; `.peer` means it arrived over the device link. See `isPeer`. - var origin: ClipboardOrigin? /// Transient, **band-only** marker: true when the band's bounded preview omitted part of a larger /// payload (so the preview UI can say "truncated — full content will paste"). Derived at band-build /// time (`ClipboardStore.boundedForBand`); deliberately **excluded from `CodingKeys`** so it is never @@ -88,9 +76,10 @@ struct ClipboardEntry: Codable, Equatable, Identifiable { var isPreviewTruncated: Bool = false /// Persisted keys only — `isPreviewTruncated` is intentionally absent so it stays transient and the - /// index format is unchanged. Optionals (`sourceApp`, `origin`) still decode-if-present (legacy compat). + /// index format is unchanged. `sourceApp` still decodes-if-present (legacy compat; a legacy + /// `origin` key from the removed device link is simply ignored on decode). private enum CodingKeys: String, CodingKey { - case id, capturedAt, kind, key, sourceApp, pinned, representations, fingerprint, origin + case id, capturedAt, kind, key, sourceApp, pinned, representations, fingerprint } init(id: UUID = UUID(), @@ -100,8 +89,7 @@ struct ClipboardEntry: Codable, Equatable, Identifiable { sourceApp: String? = nil, pinned: Bool = false, representations: [String: ClipboardPayload], - fingerprint: String, - origin: ClipboardOrigin? = nil) { + fingerprint: String) { self.id = id self.capturedAt = capturedAt self.kind = kind @@ -110,20 +98,6 @@ struct ClipboardEntry: Codable, Equatable, Identifiable { self.pinned = pinned self.representations = representations self.fingerprint = fingerprint - self.origin = origin - } - - /// The originating device name when this came from a paired device, else nil. Reads `nil`/`.local` - /// origin as not-a-peer (centralizes the "absent == local" convention). - var peerDeviceName: String? { - if case let .peer(name) = origin { return name } - return nil - } - - /// True when this entry was received from a paired device (not a local copy). - var isPeer: Bool { - if case .peer = origin { return true } - return false } // MARK: Convenience accessors (used by the preview + paste paths) diff --git a/Sources/ThreeFingerSwitcher/Clipboard/ClipboardMonitor.swift b/Sources/ThreeFingerSwitcher/Clipboard/ClipboardMonitor.swift index fda7ee2..1793e3e 100644 --- a/Sources/ThreeFingerSwitcher/Clipboard/ClipboardMonitor.swift +++ b/Sources/ThreeFingerSwitcher/Clipboard/ClipboardMonitor.swift @@ -29,14 +29,6 @@ final class ClipboardMonitor { private var timer: Timer? private var lastChangeCount: Int - /// One pasteboard `changeCount` to skip capturing (a self-write we just made, e.g. auto-pasting a - /// received item). On the next poll, if the board's `changeCount` equals this, we advance - /// `lastChangeCount` without capturing and clear the suppression — so the already-recorded entry - /// keeps its origin/`capturedAt`. A *newer* change arriving first means a real user copy happened in - /// between, so the suppression is dropped without skipping it (that copy is still captured). Nil = no - /// pending suppression. Safe whether or not the monitor is running (a stopped monitor catches it up - /// on `start()` / next `poll()`). - private var suppressedChangeCount: Int? init(store: ClipboardStore, pasteboard: NSPasteboard = .general, @@ -70,30 +62,10 @@ final class ClipboardMonitor { timer = t } - /// Tell the monitor to ignore exactly ONE upcoming pasteboard change — the one we are about to make - /// (or just made) ourselves. Pass the `changeCount` read right after writing to `NSPasteboard.general`. - /// On the next poll, if the board is still at that exact count, we advance past it without capturing - /// (so a received item's `.peer` origin/`capturedAt` survive — no self-capture). If a *different* - /// (newer) change lands first, a real user copy intervened, so we drop the suppression and capture - /// normally. Idempotent and safe whether or not the monitor is running. - func suppressSelfWrite(changeCount: Int) { - suppressedChangeCount = changeCount - } - - /// Visible for testing: a single poll tick (the timer also calls this). Honors the self-write - /// suppression, then captures on a genuine change. + /// Visible for testing: a single poll tick (the timer also calls this). Captures on a genuine change. func poll() { guard !isPaused else { return } let current = pasteboard.changeCount - if let suppressed = suppressedChangeCount { - suppressedChangeCount = nil - if current == suppressed { - // Our own write — advance past it without capturing, so the peer entry keeps its origin. - lastChangeCount = current - return - } - // A different (newer) change arrived first: a real user copy. Fall through and capture it. - } guard current != lastChangeCount else { return } lastChangeCount = current capture() diff --git a/Sources/ThreeFingerSwitcher/Clipboard/ClipboardStore.swift b/Sources/ThreeFingerSwitcher/Clipboard/ClipboardStore.swift index 390e320..7ffd319 100644 --- a/Sources/ThreeFingerSwitcher/Clipboard/ClipboardStore.swift +++ b/Sources/ThreeFingerSwitcher/Clipboard/ClipboardStore.swift @@ -73,7 +73,7 @@ final class ClipboardStore { /// image bytes dropped, RTF dropped — so building the band never loads a full large payload into memory /// (opening the band with big entries can't OOM). The full content is materialized on demand for the /// selected preview / paste (`materializedEntry(id:)`). Use this for the launcher band; use - /// `recentWindow` where full bytes are needed (e.g. the device-link outbound send). + /// `recentWindow` where full bytes are needed. func bandWindow(limit: Int) -> [ClipboardEntry] { Self.recentWindow(entries, limit: limit).map(boundedForBand) } diff --git a/Sources/ThreeFingerSwitcher/Clipboard/LinkInboundAdapter.swift b/Sources/ThreeFingerSwitcher/Clipboard/LinkInboundAdapter.swift deleted file mode 100644 index 9c5aba4..0000000 --- a/Sources/ThreeFingerSwitcher/Clipboard/LinkInboundAdapter.swift +++ /dev/null @@ -1,127 +0,0 @@ -import Foundation -import AppKit -import DeviceLinkProtocol - -/// Converts a received `LinkItem` (the device-link wire DTO) into a `ClipboardEntry`, mirroring -/// `ClipboardMonitor.makeEntry`'s per-kind representation building so a peer item and a local copy of -/// the same content derive the **same fingerprint** (and therefore de-duplicate). Received files are -/// persisted to a dedicated `inbox/` directory and referenced as a `.file` entry. The adapter does NOT -/// insert — the caller (the transport, on `@MainActor`) inserts via `ClipboardStore.insert`, keeping -/// this type free of store/actor coupling and trivially unit-testable. -struct LinkInboundAdapter { - /// Where received file bytes are written (a sibling of the store's `blobs/`; injectable for tests). - let inboxDirectory: URL - - init(inboxDirectory: URL) { - self.inboxDirectory = inboxDirectory - } - - func entry(from item: LinkItem) throws -> ClipboardEntry { - let origin = ClipboardOrigin.peer(deviceName: item.origin?.name) - let capturedAt = item.capturedAt ?? Date() - var reps: [String: ClipboardPayload] = [:] - let key: String - let fingerprint: String - let kind: ClipboardKind - - switch item.kind { - case .text: - guard let data = item.representations[LinkUTI.plainText], - let str = String(data: data, encoding: .utf8) else { - throw LinkInboundError.missingRepresentation(.text) - } - reps[ClipboardUTI.plainText] = .inline(data) - key = ClipboardKey.fromText(str) - fingerprint = "text:\(str)" - kind = .text - - case .richText: - guard let rtf = item.representations[LinkUTI.rtf] else { - throw LinkInboundError.missingRepresentation(.richText) - } - reps[ClipboardUTI.rtf] = .inline(rtf) - let plain = item.representations[LinkUTI.plainText].flatMap { String(data: $0, encoding: .utf8) } ?? "" - if let pdata = plain.data(using: .utf8) { reps[ClipboardUTI.plainText] = .inline(pdata) } - key = ClipboardKey.fromText(plain.isEmpty ? "Rich text" : plain) - fingerprint = "rich:\(Self.hash(rtf))" - kind = .richText - - case .url: - guard let data = item.representations[LinkUTI.url] ?? item.representations[LinkUTI.plainText], - let str = String(data: data, encoding: .utf8), !str.isEmpty else { - throw LinkInboundError.missingRepresentation(.url) - } - let d = Data(str.utf8) - reps[ClipboardUTI.url] = .inline(d) - reps[ClipboardUTI.plainText] = .inline(d) - key = ClipboardKey.fromText(str) - fingerprint = "url:\(str)" - kind = .url - - case .color: - guard let data = item.representations[LinkUTI.color] else { - throw LinkInboundError.missingRepresentation(.color) - } - reps[ClipboardUTI.color] = .inline(data) - key = "Color" - fingerprint = "color:\(Self.hash(data))" - kind = .color - - case .image: - guard let data = item.representations[LinkUTI.png] ?? item.representations[LinkUTI.tiff] else { - throw LinkInboundError.missingRepresentation(.image) - } - let uti = item.representations[LinkUTI.png] != nil ? ClipboardUTI.png : ClipboardUTI.tiff - reps[uti] = .inline(data) - let rep = NSBitmapImageRep(data: data) - key = ClipboardKey.fromImage(width: rep?.pixelsWide ?? 0, height: rep?.pixelsHigh ?? 0) - fingerprint = "image:\(Self.hash(data))" - kind = .image - - case .file: - // The file's bytes are carried under whatever UTI key the sender chose; take the first - // non-empty representation. The filename comes from `suggestedName`. - guard let data = (item.representations.first(where: { !$0.value.isEmpty }) ?? item.representations.first)?.value else { - throw LinkInboundError.missingRepresentation(.file) - } - let name = Self.sanitizedFileName(item.suggestedName) ?? "received-file" - try FileManager.default.createDirectory(at: inboxDirectory, withIntermediateDirectories: true) - let dest = inboxDirectory.appendingPathComponent("received-\(item.messageID.uuidString)-\(name)") - try data.write(to: dest, options: .atomic) - reps[ClipboardUTI.fileURL] = .inline(Data(dest.absoluteString.utf8)) - key = name - fingerprint = "file:\(dest.path)" - kind = .file - } - - return ClipboardEntry(capturedAt: capturedAt, kind: kind, key: key, - sourceApp: nil, representations: reps, fingerprint: fingerprint, - origin: origin) - } - - /// Strip path separators so a hostile/odd suggested name can't escape the inbox directory. - static func sanitizedFileName(_ name: String?) -> String? { - guard let name, !name.isEmpty else { return nil } - let cleaned = name.replacingOccurrences(of: "/", with: "_").replacingOccurrences(of: ":", with: "_") - return cleaned.isEmpty ? nil : cleaned - } - - /// FNV-1a 64-bit — identical to `ClipboardMonitor.hash` so peer and local fingerprints match. - static func hash(_ data: Data) -> String { - var h: UInt64 = 0xcbf29ce484222325 - for byte in data { h ^= UInt64(byte); h = h &* 0x100000001b3 } - return String(h, radix: 16) - } -} - -/// The inbound adapter's error taxonomy (mapped at the network boundary by the transport). -enum LinkInboundError: Error, Equatable, LocalizedError { - case missingRepresentation(ClipboardKind) - - var errorDescription: String? { - switch self { - case .missingRepresentation(let kind): - return "The received \(kind.rawValue) item had no usable content." - } - } -} diff --git a/Sources/ThreeFingerSwitcher/Clipboard/LinkOutboundAdapter.swift b/Sources/ThreeFingerSwitcher/Clipboard/LinkOutboundAdapter.swift deleted file mode 100644 index 1df668b..0000000 --- a/Sources/ThreeFingerSwitcher/Clipboard/LinkOutboundAdapter.swift +++ /dev/null @@ -1,61 +0,0 @@ -import Foundation -import DeviceLinkProtocol - -/// The inverse of `LinkInboundAdapter`: turns a materialized `ClipboardEntry` into a wire `LinkItem` -/// for sending to a paired device. Non-file kinds carry their inline representation bytes verbatim; -/// a `.file` entry resolves its `file://` reference and sends the **file's bytes** (so the other device -/// receives content, not a meaningless local path). The item is stamped with the local device identity. -struct LinkOutboundAdapter { - - /// A generic content UTI used to carry a file's raw bytes on the wire. - static let fileContentUTI = "public.data" - - func linkItem(from entry: ClipboardEntry, origin: DeviceIdentity, messageID: UUID = UUID()) throws -> LinkItem { - let kind = Self.linkKind(for: entry.kind) - - if entry.kind == .file { - guard let urlData = entry.data(for: ClipboardUTI.fileURL), - let url = URL(string: String(decoding: urlData, as: UTF8.self)), url.isFileURL else { - throw LinkOutboundError.unreadableFile - } - let bytes: Data - do { bytes = try Data(contentsOf: url) } catch { throw LinkOutboundError.unreadableFile } - return LinkItem(messageID: messageID, kind: .file, - representations: [Self.fileContentUTI: bytes], - suggestedName: url.lastPathComponent, - capturedAt: entry.capturedAt, origin: origin) - } - - var reps: [String: Data] = [:] - for (uti, payload) in entry.representations { - if let data = payload.inlineData { reps[uti] = data } - } - guard !reps.isEmpty else { throw LinkOutboundError.noContent } - return LinkItem(messageID: messageID, kind: kind, representations: reps, - suggestedName: nil, capturedAt: entry.capturedAt, origin: origin) - } - - static func linkKind(for kind: ClipboardKind) -> LinkItemKind { - switch kind { - case .text: return .text - case .richText: return .richText - case .image: return .image - case .color: return .color - case .url: return .url - case .file: return .file - } - } -} - -/// The outbound adapter's error taxonomy (mapped at the send boundary). -enum LinkOutboundError: Error, Equatable, LocalizedError { - case noContent - case unreadableFile - - var errorDescription: String? { - switch self { - case .noContent: return "This item has no content to send." - case .unreadableFile: return "The file could not be read to send." - } - } -} diff --git a/Sources/ThreeFingerSwitcher/DeviceLink/DeviceLinkService.swift b/Sources/ThreeFingerSwitcher/DeviceLink/DeviceLinkService.swift deleted file mode 100644 index b9470a7..0000000 --- a/Sources/ThreeFingerSwitcher/DeviceLink/DeviceLinkService.swift +++ /dev/null @@ -1,169 +0,0 @@ -import Foundation -import CryptoKit -import Network -import DeviceLinkProtocol -import DeviceLinkPairing - -/// The Mac's always-on receive anchor: advertises a Bonjour service (peer-to-peer enabled so it can -/// use peer-to-peer Wi-Fi/AWDL for high-bandwidth transfer), accepts incoming connections, and surfaces -/// reassembled `LinkItem`s. The app wires `onItem` to the inbound adapter + `ClipboardStore`. Connection -/// state is confined to the serial `queue`; `onItem`/`onOnlineChange` are delivered on the main queue so -/// the app can touch its `@MainActor` store directly. -/// -/// Every accepted connection runs the authenticated `LinkSession` handshake first (this Mac's long-lived -/// key + the pinned-fingerprint set): an unpinned or unconfirmed peer is dropped and surfaces nothing -/// (fail closed, D6). After mutual confirm the byte channel is wrapped in `SealingByteTransport` so all -/// item traffic is encrypted (`ChaChaPoly`), below the unchanged `LinkPump`. Authenticated connections are -/// held in a per-peer registry keyed by the pinned identity id, so sends can target a chosen paired peer -/// and per-device online state is observable. -final class DeviceLinkService { - static let serviceType = "_tfslink._tcp" - - /// A reassembled item from a peer, delivered on the **main** queue. - var onItem: ((LinkItem) -> Void)? - /// Per-paired-device online state changed (peer id → connected). Delivered on the **main** queue. - var onOnlineChange: ((_ peerID: String, _ online: Bool) -> Void)? - - private let localIdentity: DeviceIdentity - private let staticKey: Curve25519.KeyAgreement.PrivateKey - /// Supplies the current pinned-fingerprint set (`SHA256(peer staticPub)`); read per accept so a - /// freshly-paired or "forgotten" device is reflected without restarting the service. - private let pinnedFingerprints: () -> Set - /// Maps a verified peer fingerprint back to its stable pinned id/name for the registry. - private let device: (Data) -> PairedDevice? - - private let queue = DispatchQueue(label: "com.threefingerswitcher.devicelink") - private var listener: NWListener? - /// In-flight handshakes (not yet authenticated) — retained so they aren't deallocated mid-handshake. - private var pending: [ObjectIdentifier: LinkHandshake] = [:] // queue-confined - /// Authenticated connections keyed by the pinned peer id (D4/D5). One entry per online paired device. - private var peers: [String: LinkConnection] = [:] // queue-confined - - init(localIdentity: DeviceIdentity, - staticKey: Curve25519.KeyAgreement.PrivateKey, - pinnedFingerprints: @escaping () -> Set, - device: @escaping (Data) -> PairedDevice?) { - self.localIdentity = localIdentity - self.staticKey = staticKey - self.pinnedFingerprints = pinnedFingerprints - self.device = device - } - - /// Begin advertising + accepting. Call from the main thread. - func start() throws { - guard listener == nil else { return } - let params = NWParameters.tcp - params.includePeerToPeer = true - let listener = try NWListener(using: params) - listener.service = NWListener.Service(name: localIdentity.name, type: Self.serviceType) - listener.newConnectionHandler = { [weak self] connection in - self?.queue.async { self?.accept(connection) } - } - listener.start(queue: queue) - self.listener = listener - } - - /// Send an item to a specific paired peer by its pinned id. No-op if that peer is not online. - func send(_ item: LinkItem, to peerID: String) { - queue.async { [weak self] in - self?.peers[peerID]?.send(item) - } - } - - /// Send an item to every currently-online paired peer (a thin convenience over the per-peer send). - func sendToAll(_ item: LinkItem) { - queue.async { [weak self] in - self?.peers.values.forEach { $0.send(item) } - } - } - - /// The pinned ids of currently-online (authenticated) peers. - func onlinePeerIDs() -> Set { - queue.sync { Set(peers.keys) } - } - - /// Stop advertising and close all connections. - func stop() { - listener?.cancel() - listener = nil - queue.async { [weak self] in - guard let self else { return } - self.pending.removeAll() - // Detach the registry first so each close()'s `onClose` → `drop` is a no-op (=== fails), then - // close + notify offline exactly once per peer. - let closing = self.peers - self.peers.removeAll() - for (peerID, connection) in closing { - connection.close() - self.notifyOnline(peerID, false) - } - } - } - - // MARK: - Accept + handshake (runs on `queue`) - - private func accept(_ nwConnection: NWConnection) { - let transport = NWByteTransport(connection: nwConnection, queue: queue) - let session = LinkSession(identity: localIdentity, staticKey: staticKey) - let handshake = LinkHandshake(transport: transport, - session: session, - pinnedFingerprints: pinnedFingerprints()) - let token = ObjectIdentifier(handshake) - handshake.onEstablished = { [weak self] established, residual in - self?.adopt(established, transport: transport, residual: residual, token: token) - } - handshake.onFailure = { [weak self] _ in - self?.pending[token] = nil // connection already closed by the handshake; just drop the holder - } - pending[token] = handshake - transport.start() - handshake.start() - } - - /// A handshake completed: wrap the transport in the sealing layer, register the authenticated peer by - /// its pinned id (replacing any stale connection for the same device), and start the sealed pump. - private func adopt(_ established: LinkSession.Established, - transport: LinkByteTransport, - residual: Data, - token: ObjectIdentifier) { - pending[token] = nil - // The verified peer fingerprint resolves to the pinned record → stable peer id for the registry. - guard let paired = device(established.peerStaticFingerprint) else { return } - let peerID = paired.id - - let sealing = SealingByteTransport(inner: transport, sealKey: established.sealKey, openKey: established.openKey) - let connection = LinkConnection(transport: sealing, localIdentity: localIdentity) - connection.onItem = { [weak self] item in - DispatchQueue.main.async { self?.onItem?(item) } - } - // `onClose` fires once on any close (clean disconnect or error), already on `queue`; drop the peer - // and update online state. Idempotent in `drop` (guards against a superseded connection). - connection.onClose = { [weak self] in - self?.drop(peerID: peerID, connection: connection) - } - - // Replace any prior connection for this device (a reconnect supersedes the stale one). Install the - // new entry first so the stale close's `onClose` → `drop` is a no-op (=== fails) and doesn't clear - // the fresh registration or emit a spurious offline. - let stale = peers[peerID] - peers[peerID] = connection - stale?.close() - - connection.start() - sealing.feed(residual: residual) // replay any post-confirm bytes the handshake read ahead - notifyOnline(peerID, true) - } - - /// Remove a peer's connection from the registry iff it is still the registered one, updating online - /// state. Guards against a late error from a superseded connection clobbering a fresh reconnect. - private func drop(peerID: String, connection: LinkConnection) { - guard peers[peerID] === connection else { return } - connection.close() - peers[peerID] = nil - notifyOnline(peerID, false) - } - - private func notifyOnline(_ peerID: String, _ online: Bool) { - DispatchQueue.main.async { [weak self] in self?.onOnlineChange?(peerID, online) } - } -} diff --git a/Sources/ThreeFingerSwitcher/DeviceLink/LinkByteTransport.swift b/Sources/ThreeFingerSwitcher/DeviceLink/LinkByteTransport.swift deleted file mode 100644 index 5ca684a..0000000 --- a/Sources/ThreeFingerSwitcher/DeviceLink/LinkByteTransport.swift +++ /dev/null @@ -1,15 +0,0 @@ -import Foundation - -/// An abstract bidirectional byte channel. `LinkConnection` depends only on this seam (never on -/// `Network.framework`), so its logic is unit-tested with a mock loopback transport while the real -/// `NWConnection`-backed transport is swapped in at runtime. -protocol LinkByteTransport: AnyObject { - /// Invoked with each received buffer (may hold partial or multiple frames). - var onReceive: ((Data) -> Void)? { get set } - /// Invoked once when the channel closes, with an error if it failed. - var onClose: ((Error?) -> Void)? { get set } - /// Write a buffer to the channel. - func send(_ data: Data) - /// Close the channel. - func close() -} diff --git a/Sources/ThreeFingerSwitcher/DeviceLink/LinkConnection.swift b/Sources/ThreeFingerSwitcher/DeviceLink/LinkConnection.swift deleted file mode 100644 index 6ae714d..0000000 --- a/Sources/ThreeFingerSwitcher/DeviceLink/LinkConnection.swift +++ /dev/null @@ -1,112 +0,0 @@ -import Foundation -import DeviceLinkProtocol - -/// Drives a `LinkPump` over a `LinkByteTransport`: performs the `hello` version handshake, sends -/// `LinkItem`s as pump-encoded buffers, and surfaces received items. Transport-agnostic, so it is -/// unit-tested with a mock loopback. Driven from a single serial context (the transport's queue); -/// it is not internally synchronized. -final class LinkConnection { - /// A fully-received item from the peer. - var onItem: ((LinkItem) -> Void)? - /// The peer's identity, learned from its `hello`. - var onHandshake: ((DeviceIdentity) -> Void)? - /// A protocol/transport error (after which the connection is closed). - var onError: ((Error) -> Void)? - /// The connection closed (cleanly or after an error) — fired once. Lets a registry drop the peer and - /// update online state on a peer disconnect, not just on a protocol error. - var onClose: (() -> Void)? - - private(set) var peer: DeviceIdentity? - - private let localIdentity: DeviceIdentity - private let transport: LinkByteTransport - private var pump: LinkPump - private var closed = false - - init(transport: LinkByteTransport, localIdentity: DeviceIdentity, - chunkByteBound: Int = LinkProtocol.defaultChunkByteBound) { - self.transport = transport - self.localIdentity = localIdentity - self.pump = LinkPump(chunkByteBound: chunkByteBound) - transport.onReceive = { [weak self] data in self?.handle(data) } - transport.onClose = { [weak self] error in self?.handleClose(error) } - } - - /// Begin the session: announce ourselves with a `hello`. - func start() { - sendControl(.hello(localIdentity, LinkProtocol.version)) - } - - /// Send an item to the peer (after the handshake; ordering with the hello is preserved by the channel). - func send(_ item: LinkItem) { - guard !closed else { return } - do { - for buffer in try pump.outbound(item) { transport.send(buffer) } - } catch { - fail(error) - } - } - - func close() { - guard !closed else { return } - closed = true - transport.close() - onClose?() - } - - // MARK: - Inbound - - private func handle(_ data: Data) { - guard !closed else { return } - do { - for inbound in try pump.ingest(data) { - switch inbound { - case .item(let item): - onItem?(item) - case .control(let frame): - handleControl(frame) - } - } - } catch { - fail(error) - } - } - - private func handleControl(_ frame: Frame) { - switch frame { - case let .hello(identity, version): - guard LinkProtocol.version.isCompatible(with: version) else { - fail(LinkProtocolError(.unsupportedVersion)) - return - } - peer = identity - onHandshake?(identity) - case .ack, .error: - break // v1: acks are advisory; a peer-reported error is left to a later change to surface - default: - break // item-bearing frames are handled by the pump, not here - } - } - - private func sendControl(_ frame: Frame) { - guard !closed else { return } - do { - transport.send(try pump.outbound(control: frame)) - } catch { - fail(error) - } - } - - private func handleClose(_ error: Error?) { - guard !closed else { return } - closed = true - if let error { onError?(error) } - onClose?() - } - - private func fail(_ error: Error) { - guard !closed else { return } - onError?(error) - close() - } -} diff --git a/Sources/ThreeFingerSwitcher/DeviceLink/LinkHandshake.swift b/Sources/ThreeFingerSwitcher/DeviceLink/LinkHandshake.swift deleted file mode 100644 index 360b184..0000000 --- a/Sources/ThreeFingerSwitcher/DeviceLink/LinkHandshake.swift +++ /dev/null @@ -1,118 +0,0 @@ -import Foundation -import DeviceLinkProtocol -import DeviceLinkPairing - -/// Runs the authenticated `LinkSession` handshake on a freshly-accepted connection, *before* any item or -/// control traffic is allowed (D6: fail-closed gate). The handshake's `authHello`/`authConfirm` frames are -/// exchanged in the clear over the raw `NWByteTransport` (they carry only public keys + a MAC), framed with -/// the same `LinkCodec` envelope as the pump but driven directly here so the `LinkPump` never sees them. -/// -/// On mutual confirm it reports the derived `Established` session (session key + verified peer identity + -/// pinned fingerprint); the caller then wraps the transport in `SealingByteTransport` and drives a normal -/// `LinkConnection` over it. Any failure — malformed key, an unpinned static (not in `pinnedFingerprints`), -/// or a confirmation mismatch — closes the connection and surfaces nothing. -final class LinkHandshake { - /// Fired once on mutual confirm with the established session, plus any post-confirm bytes already - /// read (to be replayed into the sealing layer). Delivered on the transport's queue. - var onEstablished: ((LinkSession.Established, _ residual: Data) -> Void)? - /// Fired once if the handshake fails / the transport closes first. The connection is dropped. - var onFailure: ((Error) -> Void)? - - private let transport: LinkByteTransport - private let session: LinkSession - private let pinnedFingerprints: Set - private var decoder = FrameDecoder() - /// Every byte received during the handshake; lets us recover the post-confirm tail (the peer may - /// coalesce its `authConfirm` with the first sealed bytes into one TCP read) and hand it to the - /// sealing layer rather than dropping it. - private var rawReceived = Data() - - private var established: LinkSession.Established? - private var sentConfirm = false - private var done = false - - init(transport: LinkByteTransport, session: LinkSession, pinnedFingerprints: Set) { - self.transport = transport - self.session = session - self.pinnedFingerprints = pinnedFingerprints - transport.onReceive = { [weak self] data in self?.handle(data) } - transport.onClose = { [weak self] error in self?.fail(error ?? LinkSession.Failure.confirmationFailed) } - } - - /// Send our `authHello` to open the handshake. Both ends do this symmetrically (no caller-supplied role). - func start() { - sendFrame(session.hello()) - } - - // MARK: - Inbound - - private func handle(_ data: Data) { - guard !done else { return } - rawReceived.append(data) - decoder.push(data) - do { - while let frame = try decoder.next() { - try consume(frame) - if done { return } - } - } catch { - fail(error) - } - } - - /// Process one handshake frame. The two ends are symmetric: each consumes the peer's `authHello` to - /// derive the session + send its own `authConfirm`, then verifies the peer's `authConfirm`. - private func consume(_ frame: Frame) throws { - switch frame { - case .authHello: - guard established == nil else { return } // ignore a duplicate hello - let est = try session.accept(peerHello: frame, pinnedFingerprints: pinnedFingerprints) - established = est - if !sentConfirm { - sentConfirm = true - sendFrame(est.confirm()) - } - case .authConfirm: - guard let est = established else { - // A confirm before we have a session (no peer hello yet) can't verify → fail closed. - throw LinkSession.Failure.confirmationFailed - } - guard est.verify(peerConfirm: frame) else { - throw LinkSession.Failure.confirmationFailed - } - finish(est) - default: - // No other frame may precede mutual confirm (D6) — treat as a protocol violation. - throw LinkSession.Failure.confirmationFailed - } - } - - private func sendFrame(_ frame: Frame) { - guard !done else { return } - do { - transport.send(try LinkCodec.encode(frame)) - } catch { - fail(error) - } - } - - private func finish(_ established: LinkSession.Established) { - guard !done else { return } - done = true - transport.onReceive = nil - transport.onClose = nil - // Bytes pushed but not yet framed are the post-confirm tail (sealed item bytes); replay them. - let residual = decoder.bufferedByteCount > 0 - ? Data(rawReceived.suffix(decoder.bufferedByteCount)) : Data() - onEstablished?(established, residual) - } - - private func fail(_ error: Error) { - guard !done else { return } - done = true - transport.onReceive = nil - transport.onClose = nil - transport.close() - onFailure?(error) - } -} diff --git a/Sources/ThreeFingerSwitcher/DeviceLink/NWByteTransport.swift b/Sources/ThreeFingerSwitcher/DeviceLink/NWByteTransport.swift deleted file mode 100644 index cc8403b..0000000 --- a/Sources/ThreeFingerSwitcher/DeviceLink/NWByteTransport.swift +++ /dev/null @@ -1,53 +0,0 @@ -import Foundation -import Network - -/// A `LinkByteTransport` backed by an `NWConnection`. A continuous receive loop forwards bytes to -/// `onReceive`; `send` writes to the connection; failure/cancellation reports `onClose`. All callbacks -/// run on the connection's `queue` (the device-link serial queue), which is where `LinkConnection` is -/// driven. Compile-verified; runtime behavior is user-verified on devices. -final class NWByteTransport: LinkByteTransport { - var onReceive: ((Data) -> Void)? - var onClose: ((Error?) -> Void)? - - private let connection: NWConnection - private let queue: DispatchQueue - private var started = false - - init(connection: NWConnection, queue: DispatchQueue) { - self.connection = connection - self.queue = queue - } - - /// Start the connection and the receive loop. Call once. - func start() { - guard !started else { return } - started = true - connection.stateUpdateHandler = { [weak self] state in - switch state { - case .failed(let error): self?.onClose?(error) - case .cancelled: self?.onClose?(nil) - default: break - } - } - connection.start(queue: queue) - receiveLoop() - } - - private func receiveLoop() { - connection.receive(minimumIncompleteLength: 1, maximumLength: 65536) { [weak self] data, _, isComplete, error in - guard let self else { return } - if let data, !data.isEmpty { self.onReceive?(data) } - if let error { self.onClose?(error); return } - if isComplete { self.onClose?(nil); return } - self.receiveLoop() - } - } - - func send(_ data: Data) { - connection.send(content: data, completion: .contentProcessed { _ in }) - } - - func close() { - connection.cancel() - } -} diff --git a/Sources/ThreeFingerSwitcher/DeviceLink/Pairing/MacLocalIdentity.swift b/Sources/ThreeFingerSwitcher/DeviceLink/Pairing/MacLocalIdentity.swift deleted file mode 100644 index ebe2e66..0000000 --- a/Sources/ThreeFingerSwitcher/DeviceLink/Pairing/MacLocalIdentity.swift +++ /dev/null @@ -1,55 +0,0 @@ -import Foundation -import CryptoKit -import DeviceLinkProtocol -import DeviceLinkPairing - -/// The Mac's long-lived Curve25519 key in the Keychain + its public-key fingerprint — mirrors the iOS -/// `LocalIdentity`. Goes in the Mac's pairing QR and (later) is what a peer pins for the encrypted link. -enum MacLocalIdentity { - private static let service = "com.threefingerswitcher.identity" - private static let account = "macLongLivedKey" - - static var privateKey: Curve25519.KeyAgreement.PrivateKey { - if let data = load(), let key = try? Curve25519.KeyAgreement.PrivateKey(rawRepresentation: data) { - return key - } - let key = Curve25519.KeyAgreement.PrivateKey() - save(key.rawRepresentation) - return key - } - - static var fingerprint: Data { - Data(SHA256.hash(data: privateKey.publicKey.rawRepresentation)) - } - - static func payload(device: DeviceIdentity, secret: Data, - addresses: [String] = [], port: UInt16? = nil) -> PairingQRPayload { - PairingQRPayload(device: device, secret: secret, spkiFingerprint: fingerprint, - addresses: addresses, port: port) - } - - // MARK: Keychain - - private static func baseQuery() -> [String: Any] { - [kSecClass as String: kSecClassGenericPassword, - kSecAttrService as String: service, - kSecAttrAccount as String: account] - } - - private static func load() -> Data? { - var q = baseQuery() - q[kSecReturnData as String] = true - q[kSecMatchLimit as String] = kSecMatchLimitOne - var out: CFTypeRef? - guard SecItemCopyMatching(q as CFDictionary, &out) == errSecSuccess else { return nil } - return out as? Data - } - - private static func save(_ data: Data) { - SecItemDelete(baseQuery() as CFDictionary) - var q = baseQuery() - q[kSecAttrAccessible as String] = kSecAttrAccessibleAfterFirstUnlock - q[kSecValueData as String] = data - SecItemAdd(q as CFDictionary, nil) - } -} diff --git a/Sources/ThreeFingerSwitcher/DeviceLink/Pairing/MacPairingChannel.swift b/Sources/ThreeFingerSwitcher/DeviceLink/Pairing/MacPairingChannel.swift deleted file mode 100644 index d93e3ad..0000000 --- a/Sources/ThreeFingerSwitcher/DeviceLink/Pairing/MacPairingChannel.swift +++ /dev/null @@ -1,135 +0,0 @@ -import Foundation -import Network -import os -import DeviceLinkPairing - -private let pairingLog = Logger(subsystem: "com.threefingerswitcher.app", category: "pairing") - -/// A length-prefixed JSON `PairingMessage` channel over one `NWConnection` — the Mac mirror of the iOS -/// `PairingChannel`. Pairing uses `includePeerToPeer = true` (AWDL + Wi-Fi) so discovery works even when -/// the router blocks mDNS between devices (AP/client isolation, mesh). `@unchecked Sendable`. -final class MacPairingChannel: @unchecked Sendable { - static let serviceType = "_tfspair._tcp" - - var onMessage: ((PairingMessage) -> Void)? // delivered on main - var onClosed: (() -> Void)? // delivered on main - - private let connection: NWConnection - private let queue: DispatchQueue - private var buffer = Data() - - init(connection: NWConnection, queue: DispatchQueue) { - self.connection = connection - self.queue = queue - } - - static func parameters() -> NWParameters { - let params = NWParameters.tcp - params.includePeerToPeer = true // AWDL + Wi-Fi — robust across routers that block mDNS bridging - return params - } - - func start() { - connection.stateUpdateHandler = { [weak self] state in - switch state { - case .ready: pairingLog.info("channel READY"); self?.receiveLoop() - case .waiting(let error): pairingLog.error("channel waiting: \(String(describing: error), privacy: .public)") - case .failed(let error): pairingLog.error("channel failed: \(String(describing: error), privacy: .public)"); DispatchQueue.main.async { self?.onClosed?() } - case .cancelled: DispatchQueue.main.async { self?.onClosed?() } - default: break - } - } - connection.start(queue: queue) - } - - func send(_ message: PairingMessage) { - guard let json = try? JSONEncoder().encode(message) else { return } - pairingLog.info("→ send \(self.label(message), privacy: .public)") - var framed = Data() - let length = UInt32(json.count) - framed.append(UInt8((length >> 24) & 0xff)); framed.append(UInt8((length >> 16) & 0xff)) - framed.append(UInt8((length >> 8) & 0xff)); framed.append(UInt8(length & 0xff)) - framed.append(json) - connection.send(content: framed, completion: .contentProcessed { _ in }) - } - - func close() { connection.cancel() } - - private func receiveLoop() { - connection.receive(minimumIncompleteLength: 1, maximumLength: 65536) { [weak self] data, _, isComplete, error in - guard let self else { return } - if let data { self.buffer.append(data); self.drain() } - if error != nil || isComplete { DispatchQueue.main.async { self.onClosed?() }; return } - self.receiveLoop() - } - } - - private func drain() { - while buffer.count >= 4 { - let length = (UInt32(buffer[buffer.startIndex]) << 24) | (UInt32(buffer[buffer.startIndex + 1]) << 16) - | (UInt32(buffer[buffer.startIndex + 2]) << 8) | UInt32(buffer[buffer.startIndex + 3]) - let total = 4 + Int(length) - guard buffer.count >= total else { break } - let json = buffer.subdata(in: (buffer.startIndex + 4)..<(buffer.startIndex + total)) - buffer = Data(buffer[(buffer.startIndex + total)...]) - if let message = try? JSONDecoder().decode(PairingMessage.self, from: json) { - pairingLog.info("← recv \(self.label(message), privacy: .public)") - DispatchQueue.main.async { self.onMessage?(message) } - } - } - } - - private func label(_ message: PairingMessage) -> String { - switch message { - case .joinerHello: return "joinerHello" - case .hostHello: return "hostHello" - case .joinerConfirm: return "joinerConfirm" - } - } -} - -/// Advertises `_tfspair._tcp` under the Mac's id and accepts a scanner. `onReady` delivers the listener's -/// bound port so the shown QR can carry a directly-dialable endpoint (unicast, no mDNS needed). -final class MacPairingListener: @unchecked Sendable { - var onChannel: ((MacPairingChannel) -> Void)? // main - var onReady: ((UInt16) -> Void)? // main — fired once when ready + has a port - - private let serviceName: String - private let queue: DispatchQueue - private var listener: NWListener? - private var announcedPort = false - - init(serviceName: String, queue: DispatchQueue) { - self.serviceName = serviceName - self.queue = queue - } - - func start() { - guard let listener = try? NWListener(using: MacPairingChannel.parameters()) else { - pairingLog.error("listener: failed to create NWListener") - return - } - listener.service = NWListener.Service(name: serviceName, type: MacPairingChannel.serviceType) - listener.stateUpdateHandler = { [weak self] state in - pairingLog.info("listener state: \(String(describing: state), privacy: .public)") - guard let self else { return } - if case .ready = state, !self.announcedPort, let port = listener.port?.rawValue { - self.announcedPort = true - pairingLog.info("listener: bound port \(port, privacy: .public)") - DispatchQueue.main.async { self.onReady?(port) } - } - } - listener.newConnectionHandler = { [weak self] connection in - guard let self else { connection.cancel(); return } - pairingLog.info("listener: scanner CONNECTED") - let channel = MacPairingChannel(connection: connection, queue: self.queue) - channel.start() - DispatchQueue.main.async { self.onChannel?(channel) } - } - listener.start(queue: queue) - self.listener = listener - pairingLog.info("listener: advertising _tfspair._tcp as \(self.serviceName, privacy: .public)") - } - - func stop() { listener?.cancel(); listener = nil } -} diff --git a/Sources/ThreeFingerSwitcher/DeviceLink/Pairing/MacPairingCoordinator.swift b/Sources/ThreeFingerSwitcher/DeviceLink/Pairing/MacPairingCoordinator.swift deleted file mode 100644 index 8f0bb7d..0000000 --- a/Sources/ThreeFingerSwitcher/DeviceLink/Pairing/MacPairingCoordinator.swift +++ /dev/null @@ -1,98 +0,0 @@ -import Foundation -import os -import DeviceLinkProtocol -import DeviceLinkPairing - -private let coordLog = Logger(subsystem: "com.threefingerswitcher.app", category: "pairing") - -/// The Mac host side of QR pairing: generate a secret, expose the QR string, advertise the pairing -/// service, run the host `PairingExchange`, and pin the scanner into `PairedDeviceStore`. -@MainActor -final class MacPairingCoordinator: ObservableObject { - enum Status: Equatable { - case idle, waiting, pairing - case success(String) - case failed - } - - @Published var status: Status = .idle - @Published var qrString: String? - - private let queue = DispatchQueue(label: "com.threefingerswitcher.macpair") - private let store: PairedDeviceStore - private let identity: DeviceIdentity - - private var listener: MacPairingListener? - private var channel: MacPairingChannel? - private var exchange: PairingExchange? - private var secret = Data() - - init(store: PairedDeviceStore, identity: DeviceIdentity) { - self.store = store - self.identity = identity - } - - /// Show a QR + advertise the pairing service, waiting to be scanned. Start the listener FIRST so the - /// QR can carry the listener's bound port + this Mac's reachable address(es) — letting the scanner dial - /// directly (unicast) without mDNS/Bonjour discovery. The QR is published once the listener is ready. - func showCode() { - guard status == .idle else { return } - coordLog.info("HOST showCode (id \(self.identity.id, privacy: .public))") - secret = PairingQRPayload.makeSecret() - status = .waiting - - let listener = MacPairingListener(serviceName: identity.id, queue: queue) - listener.onReady = { [weak self] port in - guard let self else { return } - let addresses = LocalAddresses.current() - coordLog.info("HOST endpoint [\(addresses.joined(separator: ", "), privacy: .public)]:\(port, privacy: .public)") - self.qrString = MacLocalIdentity.payload(device: self.identity, secret: self.secret, - addresses: addresses, port: port).encodedString() - } - listener.onChannel = { [weak self] channel in self?.adopt(channel) } - listener.start() - self.listener = listener - } - - func stop() { - listener?.stop(); channel?.close() - listener = nil; channel = nil; exchange = nil - status = .idle; qrString = nil - } - - // MARK: - Exchange (main queue) - - private func adopt(_ channel: MacPairingChannel) { - guard self.channel == nil else { // already pairing on a connection — ignore extras, don't clobber - coordLog.info("HOST ignoring extra scanner connection") - channel.close(); return - } - self.channel = channel - exchange = PairingExchange(role: .host, secret: secret, - identity: identity, spkiFingerprint: MacLocalIdentity.fingerprint) - status = .pairing - channel.onMessage = { [weak self] message in self?.handle(message) } - } - - private func handle(_ message: PairingMessage) { - guard var exch = exchange, let channel else { return } - do { - let (reply, result) = try exch.consume(message) - exchange = exch - if let reply { channel.send(reply) } - if let result { - switch result { - case let .pinned(peer, fingerprint): - coordLog.info("PINNED \(peer.name, privacy: .public) ✅") - store.add(PairedDevice(id: peer.id, name: peer.name, pinnedSPKIHash: fingerprint, pairedAt: Date())) - status = .success(peer.name) - case .failed: - coordLog.error("exchange FAILED (confirmation mismatch)") - status = .failed - } - } - } catch { - status = .failed - } - } -} diff --git a/Sources/ThreeFingerSwitcher/DeviceLink/Pairing/PairedDevice.swift b/Sources/ThreeFingerSwitcher/DeviceLink/Pairing/PairedDevice.swift deleted file mode 100644 index 198c552..0000000 --- a/Sources/ThreeFingerSwitcher/DeviceLink/Pairing/PairedDevice.swift +++ /dev/null @@ -1,17 +0,0 @@ -import Foundation - -/// A durable trust record created by a successful pairing: the peer's identity plus the pinned -/// public-key (SPKI) SHA-256 hash, so future sessions authenticate the peer by pin without the code. -struct PairedDevice: Codable, Equatable, Identifiable { - var id: String // the peer device id - var name: String // human-readable (shown in the Devices list) - var pinnedSPKIHash: Data // SHA-256 of the peer's certificate SubjectPublicKeyInfo - var pairedAt: Date - - init(id: String, name: String, pinnedSPKIHash: Data, pairedAt: Date) { - self.id = id - self.name = name - self.pinnedSPKIHash = pinnedSPKIHash - self.pairedAt = pairedAt - } -} diff --git a/Sources/ThreeFingerSwitcher/DeviceLink/Pairing/PairedDeviceStore.swift b/Sources/ThreeFingerSwitcher/DeviceLink/Pairing/PairedDeviceStore.swift deleted file mode 100644 index e40eaa5..0000000 --- a/Sources/ThreeFingerSwitcher/DeviceLink/Pairing/PairedDeviceStore.swift +++ /dev/null @@ -1,68 +0,0 @@ -import Foundation - -/// Persists the pinned-peer trust records. The pinned value is a public-key hash (a public value), so a -/// Codable file under Application Support is acceptable on the Mac; the device's own *private* long-lived -/// identity goes in the Keychain/Secure Enclave in the TLS follow-up. Injectable directory for tests. -final class PairedDeviceStore { - private let fileURL: URL - private var devices: [PairedDevice] = [] - - init(directory: URL) { - self.fileURL = directory.appendingPathComponent("paired-devices.json") - load() - } - - /// `~/Library/Application Support/ThreeFingerSwitcher/devicelink`. - static func defaultDirectory() -> URL { - let base = (try? FileManager.default.url(for: .applicationSupportDirectory, in: .userDomainMask, - appropriateFor: nil, create: true)) - ?? URL(fileURLWithPath: NSTemporaryDirectory()) - return base.appendingPathComponent("ThreeFingerSwitcher/devicelink", isDirectory: true) - } - - func all() -> [PairedDevice] { devices.sorted { $0.pairedAt > $1.pairedAt } } - - /// Add or replace the record for a peer id. - func add(_ device: PairedDevice) { - devices.removeAll { $0.id == device.id } - devices.append(device) - save() - } - - func remove(id: String) { - devices.removeAll { $0.id == id } - save() - } - - /// True iff some paired peer pins this SPKI hash (the check the TLS verify block will use). - func isPinned(spkiHash: Data) -> Bool { - devices.contains { $0.pinnedSPKIHash == spkiHash } - } - - /// The set of pinned long-lived fingerprints — `SHA256(peer staticPub raw)` — fed to the - /// `LinkSession` handshake so an inbound connection is accepted only if its presented static key is - /// pinned (fail closed). The pin stored at pairing (`pinnedSPKIHash`) IS this fingerprint. - func pinnedFingerprints() -> Set { - Set(devices.map { $0.pinnedSPKIHash }) - } - - /// The paired device whose pinned fingerprint matches, if any — lets the link map a verified peer - /// fingerprint back to its stable pinned id/name for the per-peer registry and online state. - func device(forFingerprint fingerprint: Data) -> PairedDevice? { - devices.first { $0.pinnedSPKIHash == fingerprint } - } - - // MARK: Persistence - - private func load() { - guard let data = try? Data(contentsOf: fileURL), - let decoded = try? JSONDecoder().decode([PairedDevice].self, from: data) else { return } - devices = decoded - } - - private func save() { - try? FileManager.default.createDirectory(at: fileURL.deletingLastPathComponent(), withIntermediateDirectories: true) - guard let data = try? JSONEncoder().encode(devices) else { return } - try? data.write(to: fileURL, options: .atomic) - } -} diff --git a/Sources/ThreeFingerSwitcher/DeviceLink/Pairing/QRImage.swift b/Sources/ThreeFingerSwitcher/DeviceLink/Pairing/QRImage.swift deleted file mode 100644 index 18e698e..0000000 --- a/Sources/ThreeFingerSwitcher/DeviceLink/Pairing/QRImage.swift +++ /dev/null @@ -1,32 +0,0 @@ -import Foundation -import AppKit -import CoreImage - -/// QR generation + decoding (CoreImage). Generation feeds the Hub "Show pairing code" surface; decoding -/// backs the unit-tested round-trip (the actual phone scan uses the iOS camera). -enum QRImage { - /// A crisp QR `NSImage` for `string`, or nil. - static func image(from string: String) -> NSImage? { - guard let filter = CIFilter(name: "CIQRCodeGenerator") else { return nil } - filter.setValue(Data(string.utf8), forKey: "inputMessage") - filter.setValue("M", forKey: "inputCorrectionLevel") - guard let output = filter.outputImage else { return nil } - let scaled = output.transformed(by: CGAffineTransform(scaleX: 10, y: 10)) - let context = CIContext() - guard let cgImage = context.createCGImage(scaled, from: scaled.extent) else { return nil } - return NSImage(cgImage: cgImage, size: NSSize(width: scaled.extent.width, height: scaled.extent.height)) - } - - /// Decode the first QR message in a generated CIImage (for the round-trip test). - static func decode(_ string: String) -> String? { - guard let filter = CIFilter(name: "CIQRCodeGenerator") else { return nil } - filter.setValue(Data(string.utf8), forKey: "inputMessage") - filter.setValue("M", forKey: "inputCorrectionLevel") - guard let output = filter.outputImage else { return nil } - let scaled = output.transformed(by: CGAffineTransform(scaleX: 10, y: 10)) - let detector = CIDetector(ofType: CIDetectorTypeQRCode, context: CIContext(), - options: [CIDetectorAccuracy: CIDetectorAccuracyHigh]) - let features = detector?.features(in: scaled) ?? [] - return (features.first as? CIQRCodeFeature)?.messageString - } -} diff --git a/Sources/ThreeFingerSwitcher/DeviceLink/ReceiveHUD.swift b/Sources/ThreeFingerSwitcher/DeviceLink/ReceiveHUD.swift deleted file mode 100644 index 7c367b2..0000000 --- a/Sources/ThreeFingerSwitcher/DeviceLink/ReceiveHUD.swift +++ /dev/null @@ -1,178 +0,0 @@ -import AppKit -import SwiftUI -import DeviceLinkProtocol - -/// The transient on-receive notch HUD: a small, non-activating glass capsule near the top-center of the -/// active screen confirming that a device-link item just landed (or that it couldn't). Mirrors the -/// `LanesLiveToast` panel pattern — a borderless `.nonactivatingPanel` that never steals focus, ignores -/// the mouse (click-through), floats below the notch/menu bar, fades+drifts in, rests, then lifts away. -/// A burst of receives coalesces into the single live panel (content updated + timer re-armed) rather -/// than stacking windows. Fire-and-forget: `show` is the last, non-throwing step on the receive path, so -/// a HUD problem can never affect storage/auto-paste. -@MainActor -final class ReceiveHUDController { - private var panel: NSPanel? - /// The pending auto-dismiss; cancelled + re-armed when a new receive coalesces into the live panel. - private var dismissWork: DispatchWorkItem? - /// Bumped on every show/hide. A dismiss fade-out captures the value and only tears the panel down if it - /// still matches in its completion — so a receive that coalesces DURING the fade revives the panel - /// instead of having it nil'd out from under the new content. - private var epoch = 0 - private let restDuration: TimeInterval = 3.5 - - /// Show (or, if already visible, update) the HUD for a received item. `kind` drives the icon/label, - /// `deviceName` the source (falls back to "a device"), `success == false` swaps to the failure state. - func show(kind: LinkItemKind, from deviceName: String?, success: Bool) { - epoch += 1 // supersede any in-flight dismiss fade-out's completion (see hide()) - let content = ReceiveHUDView(kind: kind, deviceName: deviceName, success: success) - - // Coalesce: a receive while the HUD is still up just swaps the content and re-arms the timer — - // bursts update one panel instead of stacking (mirrors LanesLiveToast's single-instance guard). - if let panel { - (panel.contentView as? NSHostingView)?.rootView = content - panel.contentView?.layoutSubtreeIfNeeded() - layout(panel) - // Snap back to opaque in case a dismiss fade-out was mid-flight (the epoch bump above already - // stops that stale fade's completion from tearing the panel down). - NSAnimationContext.runAnimationGroup { context in - context.duration = 0.15 - panel.animator().alphaValue = 1 - } - armDismiss() - return - } - - let view = NSHostingView(rootView: content) - view.layoutSubtreeIfNeeded() - let size = view.fittingSize - let rect = topCenterRect(size: size) - let panel = NSPanel(contentRect: rect, - styleMask: [.borderless, .nonactivatingPanel], - backing: .buffered, defer: false) - panel.isReleasedWhenClosed = false - panel.isOpaque = false - panel.backgroundColor = .clear - panel.hasShadow = false - panel.level = .popUpMenu - panel.ignoresMouseEvents = true - // NOT `.canJoinAllSpaces` — that causes a documented Space-switch ghost; `.ignoresCycle` keeps it - // out of window cycling while `.fullScreenAuxiliary` lets it ride over a full-screen app. - panel.collectionBehavior = [.fullScreenAuxiliary, .ignoresCycle] - panel.contentView = view - // Drift down into place + fade in, like the lanes toast — never a hard pop on the desktop. - panel.alphaValue = 0 - panel.setFrame(rect.offsetBy(dx: 0, dy: 12), display: false) - panel.orderFrontRegardless() - self.panel = panel - NSAnimationContext.runAnimationGroup { context in - context.duration = 0.4 - context.timingFunction = CAMediaTimingFunction(name: .easeOut) - panel.animator().alphaValue = 1 - panel.animator().setFrame(rect, display: true) - } - armDismiss() - } - - /// Dismiss any visible HUD immediately (fade out, then nil the panel). Called on feature teardown. - func hide() { - dismissWork?.cancel() - dismissWork = nil - guard let panel else { return } - epoch += 1 - let token = epoch - NSAnimationContext.runAnimationGroup({ context in - context.duration = 0.35 - context.timingFunction = CAMediaTimingFunction(name: .easeIn) - panel.animator().alphaValue = 0 - panel.animator().setFrame(panel.frame.offsetBy(dx: 0, dy: 8), display: true) - }, completionHandler: { [weak self] in - // A receive (or another hide) during the fade bumped the epoch — it revived this panel, so - // don't tear it down. - guard let self, self.epoch == token else { return } - self.panel?.orderOut(nil) - self.panel?.close() - self.panel = nil - }) - } - - /// (Re-)schedule the rest → fade-out. Cancelling the prior work item is what lets a coalesced receive - /// keep the panel alive for a fresh full duration instead of inheriting the old deadline. - private func armDismiss() { - dismissWork?.cancel() - let work = DispatchWorkItem { [weak self] in self?.hide() } - dismissWork = work - DispatchQueue.main.asyncAfter(deadline: .now() + restDuration, execute: work) - } - - /// Re-place an already-shown panel after a content swap (the coalesced item may resize the capsule). - private func layout(_ panel: NSPanel) { - let size = panel.contentView?.fittingSize ?? panel.frame.size - panel.setFrame(topCenterRect(size: size), display: true) - } - - /// Notch-aware top-center geometry on the active screen: centered on the visible frame, with the top - /// edge a margin below the menu bar/notch (`safeAreaInsets.top` where the screen reports one — the - /// notch — else a fixed margin under the menu bar). - private func topCenterRect(size: NSSize) -> NSRect { - let screen = screenUnderMouse() ?? NSScreen.main ?? NSScreen.screens.first - let frame = screen?.visibleFrame ?? .zero - let safeTop = screen?.safeAreaInsets.top ?? 0 - let topMargin: CGFloat = safeTop > 0 ? safeTop + 8 : 12 - let x = (frame.minX + (frame.width - size.width) / 2).rounded() - let y = (frame.maxY - size.height - topMargin).rounded() - return NSRect(x: x, y: y, width: size.width, height: size.height) - } - - private func screenUnderMouse() -> NSScreen? { - let mouse = NSEvent.mouseLocation - return NSScreen.screens.first { NSMouseInRect(mouse, $0.frame, false) } - } -} - -/// The glass capsule the HUD draws: an SF Symbol for the item kind (or a warning on failure) beside a -/// short "Received from " / "Couldn't receive from " line, in the app's `HubGlass`. -private struct ReceiveHUDView: View { - let kind: LinkItemKind - let deviceName: String? - let success: Bool - - var body: some View { - HStack(spacing: 10) { - Image(systemName: symbol) - .foregroundStyle(success ? AnyShapeStyle(.tint) : AnyShapeStyle(Color.orange)) - Text(message) - .font(.system(size: 13, weight: .medium)) - } - .padding(.horizontal, 18) - .padding(.vertical, 11) - .background(HubGlass(cornerRadius: 22)) - .clipShape(RoundedRectangle(cornerRadius: 22, style: .continuous)) - } - - private var source: String { deviceName ?? "a device" } - - private var message: String { - success ? "Received \(kindLabel) from \(source)" : "Couldn't receive from \(source)" - } - - private var symbol: String { - guard success else { return "exclamationmark.triangle.fill" } - switch kind { - case .text, .richText: return "doc.text" - case .url: return "link" - case .image: return "photo" - case .color: return "paintpalette" - case .file: return "doc" - } - } - - private var kindLabel: String { - switch kind { - case .text, .richText: return "text" - case .url: return "link" - case .image: return "image" - case .color: return "value" - case .file: return "file" - } - } -} diff --git a/Sources/ThreeFingerSwitcher/DeviceLink/SealingByteTransport.swift b/Sources/ThreeFingerSwitcher/DeviceLink/SealingByteTransport.swift deleted file mode 100644 index 9a33c90..0000000 --- a/Sources/ThreeFingerSwitcher/DeviceLink/SealingByteTransport.swift +++ /dev/null @@ -1,87 +0,0 @@ -import Foundation -import CryptoKit -import DeviceLinkPairing - -/// A `LinkByteTransport` decorator that wraps an inner transport (the real `NWByteTransport`) in the -/// `SealedRecord` confidential layer once the link handshake has derived a session key. Every outbound -/// pump buffer is sealed into a length-prefixed `ChaChaPoly` record; every inbound buffer is reassembled -/// into whole records and opened in order. This sits *below* `LinkConnection`/`LinkPump`, so the tested -/// framing is unchanged — the pump only ever sees plaintext item bytes. -/// -/// **Ordering / nonce discipline:** the `Opener` uses an implicit per-direction monotonic counter, so -/// records MUST be opened in the exact order the peer sealed them. `NWByteTransport` may deliver a record -/// split across receives or several records coalesced into one; we buffer and frame on the 4-byte BE -/// length prefix and open each complete record. Any AEAD failure (tamper / reorder / wrong key) throws — -/// we drop the connection (no partial item is surfaced), matching the fail-closed contract. -final class SealingByteTransport: LinkByteTransport { - var onReceive: ((Data) -> Void)? - var onClose: ((Error?) -> Void)? - - private let inner: LinkByteTransport - private var sealer: SealedRecord.Sealer - private var opener: SealedRecord.Opener - private var inboundBuffer = Data() - private var closed = false - - init(inner: LinkByteTransport, sealKey: SymmetricKey, openKey: SymmetricKey) { - self.inner = inner - // Distinct per-direction keys (sealKey == the peer's openKey) so the two stream directions never - // reuse a (key, counter-nonce) pair under the role-independent session key. - self.sealer = SealedRecord.Sealer(key: sealKey) - self.opener = SealedRecord.Opener(key: openKey) - inner.onReceive = { [weak self] data in self?.handleInbound(data) } - inner.onClose = { [weak self] error in self?.onClose?(error) } - } - - /// Replay bytes that the handshake layer read past `authConfirm` (a coalesced TCP segment) so the - /// first sealed record(s) are not lost when the transport is swapped in. Call once, right after init, - /// before `LinkConnection.start`. - func feed(residual: Data) { - guard !residual.isEmpty else { return } - handleInbound(residual) - } - - func send(_ data: Data) { - guard !closed else { return } - do { - inner.send(try sealer.seal(data)) - } catch { - fail(error) - } - } - - func close() { - guard !closed else { return } - closed = true - inner.close() - } - - // MARK: - Inbound - - /// Accumulate bytes and open every complete sealed record in order. Surfaces each record's plaintext - /// to `onReceive` (the pump). A `.truncated` open means we need more bytes — wait; any other error is - /// fatal (fail closed). - private func handleInbound(_ data: Data) { - guard !closed else { return } - inboundBuffer.append(data) - while !inboundBuffer.isEmpty { - do { - let (plaintext, consumed) = try opener.open(inboundBuffer) - inboundBuffer.removeFirst(consumed) - onReceive?(plaintext) - } catch SealedRecord.Error.truncated { - return // incomplete record: keep buffering - } catch { - fail(error) - return - } - } - } - - private func fail(_ error: Error) { - guard !closed else { return } - closed = true - inner.close() - onClose?(error) - } -} diff --git a/Sources/ThreeFingerSwitcher/Hub/HubDevicesPage.swift b/Sources/ThreeFingerSwitcher/Hub/HubDevicesPage.swift deleted file mode 100644 index 7cf7bf9..0000000 --- a/Sources/ThreeFingerSwitcher/Hub/HubDevicesPage.swift +++ /dev/null @@ -1,113 +0,0 @@ -import SwiftUI - -/// The Hub **Devices** page: the device-link opt-in, the paired-device list (with Forget), and the v1 -/// outbound trigger. Honest copy about the link being local-network-only and not yet end-to-end -/// encrypted (that arrives with device pairing's TLS follow-up). -struct DevicesPage: View { - let context: HubContext - @ObservedObject private var settings: AppSettings - - init(context: HubContext) { - self.context = context - _settings = ObservedObject(wrappedValue: context.settings) - } - - var body: some View { - HubPage(HubDestination.devices.title, - subtitle: "Move clipboard items and files between this Mac and your iPhone over your local network.") { - HubSection(footnote: "Opens a direct local-network link to your paired iPhone — no servers, nothing leaves your network. Items you receive appear in your Clipboard band (turn on Clipboard history to see them there). Pairing is completed on the devices. Note: the link is not yet end-to-end encrypted — that arrives with device pairing — so keep this off on untrusted networks until then.") { - ToggleRow(title: "Enable the device link", isOn: $settings.enableDeviceLink) - } - - if let coordinator = context.pairingCoordinator { - HubSection("Pair a device", - footnote: "Show this code and scan it with your iPhone to pair securely — the secret never leaves the screen, and each device pins the other.") { - ShowPairingCodeView(coordinator: coordinator) - } - } - - HubSection("Paired devices") { - let devices = context.pairedDevices() - if devices.isEmpty { - Text("No paired devices yet. Pair your iPhone from the companion app to start moving things between them.") - .font(.system(size: 12)) - .foregroundStyle(.secondary) - .frame(maxWidth: .infinity, alignment: .leading) - } else { - ForEach(devices) { device in - HStack(spacing: 10) { - Image(systemName: "iphone").foregroundStyle(.secondary) - VStack(alignment: .leading, spacing: 1) { - Text(device.name).font(.system(size: 13, weight: .medium)) - Text("Paired \(device.pairedAt.formatted(date: .abbreviated, time: .omitted))") - .font(.system(size: 11)).foregroundStyle(.secondary) - } - Spacer() - Button("Forget") { context.onForgetDevice(device.id) } - .buttonStyle(.borderless) - .foregroundStyle(.red) - } - .padding(.vertical, 2) - } - } - } - - HubSection("Send", - footnote: "Sends the most recent item from your clipboard history to your connected devices.") { - Button { - context.onSendLatestToDevices() - } label: { - Label("Send latest clipboard item to my devices", systemImage: "paperplane") - } - .disabled(!settings.enableDeviceLink) - } - } - } -} - -/// Shows the Mac's pairing QR and runs the host exchange while visible (advertises only while shown). -private struct ShowPairingCodeView: View { - @ObservedObject var coordinator: MacPairingCoordinator - @State private var showing = false - - var body: some View { - VStack(alignment: .leading, spacing: 10) { - if showing { - if let string = coordinator.qrString, let image = QRImage.image(from: string) { - Image(nsImage: image) - .interpolation(.none) - .resizable() - .scaledToFit() - .frame(width: 180, height: 180) - .background(Color.white) - .padding(6) - } else { - ProgressView().frame(width: 180, height: 180) - } - status - Button("Hide code") { coordinator.stop(); showing = false } - .buttonStyle(.borderless) - } else { - Button { - coordinator.showCode() - showing = true - } label: { - Label("Show pairing code", systemImage: "qrcode") - } - } - } - } - - @ViewBuilder private var status: some View { - switch coordinator.status { - case let .success(name): - Label("Paired with \(name)", systemImage: "checkmark.seal.fill").foregroundStyle(.green) - case .failed: - Label("Pairing failed — show a fresh code.", systemImage: "xmark.octagon").foregroundStyle(.red) - case .pairing: - HStack { ProgressView().controlSize(.small); Text("Pairing…") } - default: - Text("Scan this with your iPhone.").font(.system(size: 12)).foregroundStyle(.secondary) - } - } -} diff --git a/Sources/ThreeFingerSwitcher/Hub/HubView.swift b/Sources/ThreeFingerSwitcher/Hub/HubView.swift index b63a6d9..674da77 100644 --- a/Sources/ThreeFingerSwitcher/Hub/HubView.swift +++ b/Sources/ThreeFingerSwitcher/Hub/HubView.swift @@ -8,7 +8,6 @@ enum HubDestination: Hashable, CaseIterable { case overview case bands case switcher, launcher, clipboard, keyboardLanguage - case devices case setup, general var title: String { @@ -19,7 +18,6 @@ enum HubDestination: Hashable, CaseIterable { case .launcher: return "Launcher" case .clipboard: return "Clipboard" case .keyboardLanguage: return "Keyboard Language" - case .devices: return "Devices" case .setup: return "Setup & Permissions" case .general: return "General" } @@ -43,7 +41,6 @@ enum HubDestination: Hashable, CaseIterable { case .launcher: return "square.grid.3x3.fill" case .clipboard: return "doc.on.clipboard" case .keyboardLanguage: return "globe" - case .devices: return "iphone.and.arrow.forward" case .setup: return "gearshape.2" case .general: return "slider.horizontal.3" } @@ -138,12 +135,6 @@ final class HubContext { var onDangerZoneClear: (DangerZoneSelection) -> Void = { _ in } var onRestoreAllGestures: () -> Void = {} - // Devices page (device link). - var pairedDevices: () -> [PairedDevice] = { [] } - var onForgetDevice: (String) -> Void = { _ in } - var onSendLatestToDevices: () -> Void = {} - /// QR pairing host coordinator (show a code + accept a scanner). Observed by the Devices page. - var pairingCoordinator: MacPairingCoordinator? init(settings: AppSettings, favorites: FavoritesStore, @@ -210,7 +201,7 @@ struct HubView: View { railDivider railButton(.bands) railDivider - railButton(.switcher); railButton(.launcher); railButton(.clipboard); railButton(.keyboardLanguage); railButton(.devices) + railButton(.switcher); railButton(.launcher); railButton(.clipboard); railButton(.keyboardLanguage) railDivider railButton(.setup); railButton(.general) } @@ -263,7 +254,6 @@ struct HubView: View { case .launcher: LauncherPage(context: context) case .clipboard: ClipboardPage(context: context) case .keyboardLanguage: KeyboardLanguagePage(context: context) - case .devices: DevicesPage(context: context) case .setup: SetupPage(context: context) case .general: GeneralPage(context: context) } diff --git a/Sources/ThreeFingerSwitcher/Launcher/LaunchService.swift b/Sources/ThreeFingerSwitcher/Launcher/LaunchService.swift index 70919f7..45cddb9 100644 --- a/Sources/ThreeFingerSwitcher/Launcher/LaunchService.swift +++ b/Sources/ThreeFingerSwitcher/Launcher/LaunchService.swift @@ -220,8 +220,7 @@ final class LaunchService { /// Write a `ClipboardEntry`'s representations onto a pasteboard (default `NSPasteboard.general`), /// with the same image PNG+TIFF expansion and color/path plain-text fallbacks `pasteEntry` uses — /// but WITHOUT synthesizing a ⌘V. The single reusable writer for "make this entry the clipboard": - /// the launcher's paste path calls it (then pastes), and the device-link receive path calls it to - /// auto-paste a received item (no key synthesis). An entry with no inline bytes writes nothing. + /// the launcher's paste path calls it (then pastes). An entry with no inline bytes writes nothing. /// `nonisolated` + pasteboard-injectable so it's callable off the launcher and unit-testable. @discardableResult nonisolated static func writeToPasteboard(_ entry: ClipboardEntry, diff --git a/Sources/ThreeFingerSwitcher/Overlay/ClipboardBandView.swift b/Sources/ThreeFingerSwitcher/Overlay/ClipboardBandView.swift index 2c50f93..3b6703f 100644 --- a/Sources/ThreeFingerSwitcher/Overlay/ClipboardBandView.swift +++ b/Sources/ThreeFingerSwitcher/Overlay/ClipboardBandView.swift @@ -91,9 +91,6 @@ struct ClipboardBandView: View { .truncationMode(.tail) .foregroundStyle(selected ? .primary : .secondary) Spacer(minLength: 4) - if let entry = clipboardEntry(item), entry.isPeer { - ProvenanceChip(deviceName: entry.peerDeviceName, color: color) - } if model.isPinned(item) { Image(systemName: "pin.fill").font(.system(size: 10)).foregroundStyle(color) } @@ -376,30 +373,6 @@ private struct ClipboardImagePreview: View { } } -/// A small, unobtrusive "from \" marker on a Clipboard-band row whose entry arrived over the -/// device link. Shows the device name when known, else a generic phone glyph. Does not alter the row's -/// key text or value preview (per the provenance spec). -private struct ProvenanceChip: View { - let deviceName: String? - let color: Color - - var body: some View { - HStack(spacing: 3) { - Image(systemName: "iphone").font(.system(size: 9)) - if let deviceName, !deviceName.isEmpty { - Text(deviceName).font(.system(size: 9, weight: .medium)).lineLimit(1) - } - } - .foregroundStyle(color) - .padding(.horizontal, 5) - .padding(.vertical, 1) - .background( - Capsule(style: .continuous).fill(color.opacity(0.14)) - ) - .accessibilityLabel(deviceName.map { "from \($0)" } ?? "from a paired device") - } -} - /// A list-row dwell highlight: a Liquid Glass pill that starts nearly clear and tints over the dwell, /// then locks when armed — the list analog of the grid's `SelectionSquare`. It is a single persistent /// view that slides between rows (see `highlight`), so scrubbing doesn't strobe. `token` re-animates diff --git a/Sources/ThreeFingerSwitcher/Settings/AppSettings.swift b/Sources/ThreeFingerSwitcher/Settings/AppSettings.swift index 30654ef..b252b3b 100644 --- a/Sources/ThreeFingerSwitcher/Settings/AppSettings.swift +++ b/Sources/ThreeFingerSwitcher/Settings/AppSettings.swift @@ -130,12 +130,6 @@ final class AppSettings: ObservableObject { /// permission — it only enables local recording + the synthetic band. Default OFF (privacy). @Published var keepClipboardHistory: Bool { didSet { defaults.set(keepClipboardHistory, forKey: Keys.keepClipboardHistory) } } - /// Opt-in to the device link (iPhone↔Mac clipboard/file bridge). Like the clipboard opt-in it - /// relocates no native gesture, needs no re-login, and has no `is…Effective` gate — it just starts/ - /// stops the receive/send service. Default OFF (privacy; it opens a local-network listener). Adds the - /// macOS Local Network prompt the first time the service advertises/connects. - @Published var enableDeviceLink: Bool { didSet { defaults.set(enableDeviceLink, forKey: Keys.enableDeviceLink) } } - /// Temporarily stop recording without disabling the feature (the band still shows what's stored). @Published var clipboardPaused: Bool { didSet { defaults.set(clipboardPaused, forKey: Keys.clipboardPaused) } } @@ -299,7 +293,6 @@ final class AppSettings: ObservableObject { dwellToArmDuration = defaults.object(forKey: Keys.dwellToArmDuration) as? Double ?? Defaults.dwellToArmDuration showDiagnostics = defaults.object(forKey: Keys.showDiagnostics) as? Bool ?? Defaults.showDiagnostics keepClipboardHistory = defaults.object(forKey: Keys.keepClipboardHistory) as? Bool ?? Defaults.keepClipboardHistory - enableDeviceLink = defaults.object(forKey: Keys.enableDeviceLink) as? Bool ?? Defaults.enableDeviceLink clipboardPaused = defaults.object(forKey: Keys.clipboardPaused) as? Bool ?? Defaults.clipboardPaused clipboardRecentWindow = defaults.object(forKey: Keys.clipboardRecentWindow) as? Int ?? Defaults.clipboardRecentWindow clipboardMaxCount = defaults.object(forKey: Keys.clipboardMaxCount) as? Int ?? Defaults.clipboardMaxCount @@ -426,7 +419,6 @@ final class AppSettings: ObservableObject { static let dwellToArmDuration = 0.3 // quick tick; the charge stays readable static let showDiagnostics = false // troubleshooting tools hidden from the menu by default static let keepClipboardHistory = false // opt-in; records copied content locally (privacy) - static let enableDeviceLink = false // opt-in; opens a local-network link to the phone (privacy) static let clipboardPaused = false static let clipboardRecentWindow = 30 // entries shown in the band (pinned float to top) static let clipboardMaxCount = 200 // stored-entry cap (pinned exempt) @@ -479,7 +471,6 @@ final class AppSettings: ObservableObject { static let dwellToArmDuration = "dwellToArmDuration" static let showDiagnostics = "showDiagnostics" static let keepClipboardHistory = "keepClipboardHistory" - static let enableDeviceLink = "enableDeviceLink" static let clipboardPaused = "clipboardPaused" static let clipboardRecentWindow = "clipboardRecentWindow" static let clipboardMaxCount = "clipboardMaxCount" diff --git a/Tests/ThreeFingerSwitcherTests/AppSettingsTests.swift b/Tests/ThreeFingerSwitcherTests/AppSettingsTests.swift index 158dc81..7e60430 100644 --- a/Tests/ThreeFingerSwitcherTests/AppSettingsTests.swift +++ b/Tests/ThreeFingerSwitcherTests/AppSettingsTests.swift @@ -77,17 +77,6 @@ final class AppSettingsTests: XCTestCase { XCTAssertTrue(reader.manageVerticalGesture, "persists across instances") } - func testEnableDeviceLinkDefaultsFalseAndPersists() { - let writer = makeSettings() - XCTAssertFalse(writer.enableDeviceLink, "default must be off (opt-in)") - - writer.enableDeviceLink = true - XCTAssertEqual(defaults.object(forKey: "enableDeviceLink") as? Bool, true, "writes the documented key") - - let reader = AppSettings(defaults: defaults) - XCTAssertTrue(reader.enableDeviceLink, "persists across instances") - } - /// Spot-check the literal default values so a silent change to `Defaults` is caught. func testDefaultsEnumHasExpectedLiteralValues() { // The shipped gesture-feel numbers are the maintainer's dialed-in daily-use values diff --git a/Tests/ThreeFingerSwitcherTests/ClipboardMonitorTests.swift b/Tests/ThreeFingerSwitcherTests/ClipboardMonitorTests.swift index 1c04c77..abd811d 100644 --- a/Tests/ThreeFingerSwitcherTests/ClipboardMonitorTests.swift +++ b/Tests/ThreeFingerSwitcherTests/ClipboardMonitorTests.swift @@ -101,72 +101,4 @@ final class ClipboardMonitorTests: XCTestCase { XCTAssertEqual(store.count, 1, "re-capturing identical content does not duplicate") } - - // MARK: - Self-write suppression (auto-paste of a received item) - - /// A peer entry already in the store + our own pasteboard write (suppressed by its `changeCount`) is - /// NOT re-captured on the next poll, and the peer entry keeps its `.peer` origin. - func testSuppressedSelfWriteIsNotRecaptured() { - let pb = pasteboard() - let store = ClipboardStore(directory: tempDir()) - let monitor = ClipboardMonitor(store: store, pasteboard: pb, sourceAppProvider: { nil }) - - // Simulate the receive path: a `.peer` entry is inserted, then we write it to the board. - let peer = ClipboardEntry(capturedAt: Date(timeIntervalSince1970: 1000), - kind: .text, key: "from iPhone", - representations: [ClipboardUTI.plainText: .inline(Data("from iPhone".utf8))], - fingerprint: "text:from iPhone", - origin: .peer(deviceName: "iPhone")) - store.insert(peer) - pb.clearContents() - pb.setString("from iPhone", forType: .string) // our own write - - monitor.suppressSelfWrite(changeCount: pb.changeCount) - monitor.poll() - - XCTAssertEqual(store.count, 1, "the self-write is not captured as a second entry") - XCTAssertEqual(store.recentWindow(limit: 1).first?.origin, .peer(deviceName: "iPhone"), - "the peer entry keeps its origin/capturedAt (no self-capture overwrote it)") - XCTAssertEqual(store.recentWindow(limit: 1).first?.capturedAt, Date(timeIntervalSince1970: 1000)) - } - - /// Suppression matches exactly ONE `changeCount`: if a *newer* (real user) copy lands before the - /// poll, the suppression doesn't match and that copy IS captured (no lost captures). - func testNewerChangeBeforePollIsStillCaptured() { - let pb = pasteboard() - let store = ClipboardStore(directory: tempDir()) - let monitor = ClipboardMonitor(store: store, pasteboard: pb, sourceAppProvider: { nil }) - - pb.setString("our write", forType: .string) - let suppressed = pb.changeCount - monitor.suppressSelfWrite(changeCount: suppressed) - - // A real user copy lands before the poll → newer changeCount. - pb.clearContents() - pb.setString("user copy", forType: .string) - XCTAssertNotEqual(pb.changeCount, suppressed) - - monitor.poll() - - XCTAssertEqual(store.recentWindow(limit: 1).first?.key, "user copy", - "a real copy that superseded the suppressed change is still captured") - } - - /// Suppression is one-shot: after consuming it, the very next genuine change is captured normally. - func testSuppressionIsOneShot() { - let pb = pasteboard() - let store = ClipboardStore(directory: tempDir()) - let monitor = ClipboardMonitor(store: store, pasteboard: pb, sourceAppProvider: { nil }) - - pb.setString("self write", forType: .string) - monitor.suppressSelfWrite(changeCount: pb.changeCount) - monitor.poll() // consumes (skips) the suppression - XCTAssertTrue(store.isEmpty) - - pb.clearContents() - pb.setString("real copy", forType: .string) - monitor.poll() // next change is captured - - XCTAssertEqual(store.recentWindow(limit: 1).first?.key, "real copy") - } } diff --git a/Tests/ThreeFingerSwitcherTests/LinkConnectionTests.swift b/Tests/ThreeFingerSwitcherTests/LinkConnectionTests.swift deleted file mode 100644 index 29b8b5e..0000000 --- a/Tests/ThreeFingerSwitcherTests/LinkConnectionTests.swift +++ /dev/null @@ -1,94 +0,0 @@ -import XCTest -import DeviceLinkProtocol -@testable import ThreeFingerSwitcherCore - -/// `LinkConnection` logic, driven synchronously by a mock loopback transport — handshake, item -/// exchange, version refusal, malformed-stream teardown. No `Network.framework` involved. -final class LinkConnectionTests: XCTestCase { - - /// A synchronous loopback: each `send` delivers straight to the peer's `onReceive`. - final class MockByteTransport: LinkByteTransport { - var onReceive: ((Data) -> Void)? - var onClose: ((Error?) -> Void)? - weak var peer: MockByteTransport? - private(set) var closed = false - - func send(_ data: Data) { peer?.onReceive?(data) } - func close() { - guard !closed else { return } - closed = true - onClose?(nil) - } - - static func pair() -> (MockByteTransport, MockByteTransport) { - let a = MockByteTransport(); let b = MockByteTransport() - a.peer = b; b.peer = a - return (a, b) - } - } - - private let mac = DeviceIdentity(id: "mac", name: "Mac") - private let phone = DeviceIdentity(id: "phone", name: "iPhone") - - func testHandshakeLearnsPeerIdentities() { - let (ta, tb) = MockByteTransport.pair() - let connA = LinkConnection(transport: ta, localIdentity: mac) - let connB = LinkConnection(transport: tb, localIdentity: phone) - var aLearned: DeviceIdentity?; connA.onHandshake = { aLearned = $0 } - var bLearned: DeviceIdentity?; connB.onHandshake = { bLearned = $0 } - - connA.start() - connB.start() - - XCTAssertEqual(aLearned, phone) - XCTAssertEqual(bLearned, mac) - XCTAssertEqual(connA.peer, phone) - } - - func testItemExchangeBothDirections() { - let (ta, tb) = MockByteTransport.pair() - let connA = LinkConnection(transport: ta, localIdentity: mac) - let connB = LinkConnection(transport: tb, localIdentity: phone) - var aItems: [LinkItem] = []; connA.onItem = { aItems.append($0) } - var bItems: [LinkItem] = []; connB.onItem = { bItems.append($0) } - connA.start(); connB.start() - - let toPhone = LinkItem(messageID: UUID(), kind: .text, representations: [LinkUTI.plainText: Data("to phone".utf8)]) - let toMac = LinkItem(messageID: UUID(), kind: .url, representations: [LinkUTI.url: Data("https://x".utf8)]) - connA.send(toPhone) - connB.send(toMac) - - XCTAssertEqual(bItems, [toPhone]) - XCTAssertEqual(aItems, [toMac]) - } - - func testIncompatibleVersionIsRefused() throws { - let (ta, tb) = MockByteTransport.pair() - let connA = LinkConnection(transport: ta, localIdentity: mac) - var handshakes = 0; connA.onHandshake = { _ in handshakes += 1 } - var items = 0; connA.onItem = { _ in items += 1 } - var error: Error?; connA.onError = { error = $0 } - connA.start() - - // Inject a hello from a peer on an incompatible major version (2.x vs our 1.x). - _ = tb // keep the pair alive - let badHello = try LinkCodec.encode(.hello(phone, ProtocolVersion(major: 2, minor: 0))) - ta.onReceive?(badHello) - - XCTAssertEqual((error as? LinkProtocolError)?.code, .unsupportedVersion) - XCTAssertEqual(handshakes, 0) - XCTAssertEqual(items, 0) - XCTAssertTrue(ta.closed, "connection should close on version refusal") - } - - func testMalformedStreamTearsDown() { - let (ta, _) = MockByteTransport.pair() - let connA = LinkConnection(transport: ta, localIdentity: mac) - var error: Error?; connA.onError = { error = $0 } - connA.start() - - ta.onReceive?(Data([0xDE, 0xAD, 0xBE, 0xEF, 1, 2, 3, 4, 5, 6])) // bad magic - XCTAssertTrue(error is LinkProtocolError) - XCTAssertTrue(ta.closed) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/LinkInboundAdapterTests.swift b/Tests/ThreeFingerSwitcherTests/LinkInboundAdapterTests.swift deleted file mode 100644 index 540f2ff..0000000 --- a/Tests/ThreeFingerSwitcherTests/LinkInboundAdapterTests.swift +++ /dev/null @@ -1,144 +0,0 @@ -import XCTest -import DeviceLinkProtocol -@testable import ThreeFingerSwitcherCore - -/// The receive-side adapter: LinkItem → ClipboardEntry mapping, inbox file persistence, peer provenance, -/// and de-dup against an identical local entry through the existing store. -final class LinkInboundAdapterTests: XCTestCase { - - private var tempDir: URL! - private var adapter: LinkInboundAdapter! - - override func setUpWithError() throws { - tempDir = FileManager.default.temporaryDirectory.appendingPathComponent("tfs-inbox-\(UUID().uuidString)") - adapter = LinkInboundAdapter(inboxDirectory: tempDir.appendingPathComponent("inbox")) - } - - override func tearDownWithError() throws { - try? FileManager.default.removeItem(at: tempDir) - } - - private func linkItem(_ kind: LinkItemKind, - reps: [String: Data], - name: String? = nil, - device: String? = "iPhone") -> LinkItem { - LinkItem(messageID: UUID(), kind: kind, representations: reps, - suggestedName: name, capturedAt: nil, - origin: device.map { DeviceIdentity(id: "dev", name: $0) }) - } - - // MARK: Mapping - - func testTextItemMapping() throws { - let entry = try adapter.entry(from: linkItem(.text, reps: [LinkUTI.plainText: Data("hello".utf8)])) - XCTAssertEqual(entry.kind, .text) - XCTAssertEqual(entry.data(for: ClipboardUTI.plainText), Data("hello".utf8)) - XCTAssertEqual(entry.key, "hello") - XCTAssertEqual(entry.fingerprint, "text:hello", "must match the local capture fingerprint convention") - XCTAssertEqual(entry.origin, .peer(deviceName: "iPhone")) - } - - func testUrlItemMapping() throws { - let entry = try adapter.entry(from: linkItem(.url, reps: [LinkUTI.url: Data("https://example.com".utf8)])) - XCTAssertEqual(entry.kind, .url) - XCTAssertEqual(entry.fingerprint, "url:https://example.com") - XCTAssertEqual(entry.data(for: ClipboardUTI.url), Data("https://example.com".utf8)) - XCTAssertEqual(entry.data(for: ClipboardUTI.plainText), Data("https://example.com".utf8)) - } - - func testColorItemMapping() throws { - let colorBytes = Data([1, 2, 3, 4]) - let entry = try adapter.entry(from: linkItem(.color, reps: [LinkUTI.color: colorBytes])) - XCTAssertEqual(entry.kind, .color) - XCTAssertEqual(entry.key, "Color") - XCTAssertTrue(entry.fingerprint.hasPrefix("color:")) - XCTAssertEqual(entry.data(for: ClipboardUTI.color), colorBytes) - } - - func testRichTextItemMapping() throws { - let rtf = Data("{\\rtf1 hi}".utf8) - let entry = try adapter.entry(from: linkItem(.richText, reps: [LinkUTI.rtf: rtf, LinkUTI.plainText: Data("hi".utf8)])) - XCTAssertEqual(entry.kind, .richText) - XCTAssertEqual(entry.key, "hi") - XCTAssertTrue(entry.fingerprint.hasPrefix("rich:")) - XCTAssertEqual(entry.data(for: ClipboardUTI.rtf), rtf) - } - - func testImageItemMappingDimensionsKey() throws { - // Build a deterministic 4x2 PNG without a display (no lockFocus). - let rep = NSBitmapImageRep(bitmapDataPlanes: nil, pixelsWide: 4, pixelsHigh: 2, - bitsPerSample: 8, samplesPerPixel: 4, hasAlpha: true, isPlanar: false, - colorSpaceName: .deviceRGB, bytesPerRow: 0, bitsPerPixel: 0)! - let png = rep.representation(using: .png, properties: [:])! - let entry = try adapter.entry(from: linkItem(.image, reps: [LinkUTI.png: png])) - XCTAssertEqual(entry.kind, .image) - XCTAssertEqual(entry.key, "Image 4×2") - XCTAssertTrue(entry.fingerprint.hasPrefix("image:")) - XCTAssertEqual(entry.data(for: ClipboardUTI.png), png) - } - - func testMissingRepresentationThrows() { - XCTAssertThrowsError(try adapter.entry(from: linkItem(.text, reps: [:]))) { - XCTAssertEqual($0 as? LinkInboundError, .missingRepresentation(.text)) - } - } - - // MARK: Files / inbox - - func testFileItemWritesToInboxAndReferencesIt() throws { - let bytes = Data((0..<2048).map { UInt8($0 & 0xff) }) - let entry = try adapter.entry(from: linkItem(.file, reps: ["public.data": bytes], name: "report.pdf")) - XCTAssertEqual(entry.kind, .file) - XCTAssertEqual(entry.key, "report.pdf") - XCTAssertTrue(entry.fingerprint.hasPrefix("file:")) - - let urlData = try XCTUnwrap(entry.data(for: ClipboardUTI.fileURL)) - let url = try XCTUnwrap(URL(string: String(decoding: urlData, as: UTF8.self))) - XCTAssertTrue(url.isFileURL) - XCTAssertEqual(try Data(contentsOf: url), bytes, "the referenced file holds the received bytes") - XCTAssertTrue(url.lastPathComponent.hasSuffix("report.pdf")) - } - - func testInboxCreatedOnDemand() throws { - XCTAssertFalse(FileManager.default.fileExists(atPath: adapter.inboxDirectory.path)) - _ = try adapter.entry(from: linkItem(.file, reps: ["public.data": Data("x".utf8)], name: "a.txt")) - XCTAssertTrue(FileManager.default.fileExists(atPath: adapter.inboxDirectory.path)) - } - - // MARK: Provenance - - func testProvenanceStamping() throws { - let named = try adapter.entry(from: linkItem(.text, reps: [LinkUTI.plainText: Data("a".utf8)], device: "iPhone")) - XCTAssertEqual(named.origin, .peer(deviceName: "iPhone")) - XCTAssertTrue(named.isPeer) - XCTAssertEqual(named.peerDeviceName, "iPhone") - - let anon = try adapter.entry(from: linkItem(.text, reps: [LinkUTI.plainText: Data("b".utf8)], device: nil)) - XCTAssertEqual(anon.origin, .peer(deviceName: nil)) - XCTAssertTrue(anon.isPeer) - - let local = ClipboardEntry(capturedAt: Date(), kind: .text, key: "c", - representations: [:], fingerprint: "text:c") - XCTAssertFalse(local.isPeer) - XCTAssertNil(local.peerDeviceName) - } - - // MARK: De-dup through the real store - - @MainActor - func testPeerItemDedupsAgainstIdenticalLocal() throws { - let storeDir = tempDir.appendingPathComponent("store") - let store = ClipboardStore(directory: storeDir) - // A pre-existing local copy of "shared". - store.insert(ClipboardEntry(capturedAt: Date(), kind: .text, key: "shared", - representations: [ClipboardUTI.plainText: .inline(Data("shared".utf8))], - fingerprint: "text:shared")) - XCTAssertEqual(store.allEntries().count, 1) - - // The same content arrives from the phone. - let peer = try adapter.entry(from: linkItem(.text, reps: [LinkUTI.plainText: Data("shared".utf8)])) - store.insert(peer) - - XCTAssertEqual(store.allEntries().count, 1, "identical content must de-dup, not duplicate") - } -} diff --git a/Tests/ThreeFingerSwitcherTests/LinkOutboundAdapterTests.swift b/Tests/ThreeFingerSwitcherTests/LinkOutboundAdapterTests.swift deleted file mode 100644 index 9207cc9..0000000 --- a/Tests/ThreeFingerSwitcherTests/LinkOutboundAdapterTests.swift +++ /dev/null @@ -1,96 +0,0 @@ -import XCTest -import DeviceLinkProtocol -@testable import ThreeFingerSwitcherCore - -/// The send-side adapter (ClipboardEntry → LinkItem) and its round-trip fidelity with the inbound adapter. -final class LinkOutboundAdapterTests: XCTestCase { - - private var tempDir: URL! - private let outbound = LinkOutboundAdapter() - private let mac = DeviceIdentity(id: "mac", name: "Mac") - - override func setUpWithError() throws { - tempDir = FileManager.default.temporaryDirectory.appendingPathComponent("tfs-out-\(UUID().uuidString)") - try FileManager.default.createDirectory(at: tempDir, withIntermediateDirectories: true) - } - override func tearDownWithError() throws { try? FileManager.default.removeItem(at: tempDir) } - - private func textEntry(_ s: String) -> ClipboardEntry { - ClipboardEntry(capturedAt: Date(), kind: .text, key: s, - representations: [ClipboardUTI.plainText: .inline(Data(s.utf8))], fingerprint: "text:\(s)") - } - - func testTextMapping() throws { - let item = try outbound.linkItem(from: textEntry("hi"), origin: mac) - XCTAssertEqual(item.kind, .text) - XCTAssertEqual(item.representations[ClipboardUTI.plainText], Data("hi".utf8)) - XCTAssertEqual(item.origin, mac) - } - - func testUrlMapping() throws { - let entry = ClipboardEntry(capturedAt: Date(), kind: .url, key: "u", - representations: [ClipboardUTI.url: .inline(Data("https://x".utf8))], - fingerprint: "url:https://x") - let item = try outbound.linkItem(from: entry, origin: mac) - XCTAssertEqual(item.kind, .url) - XCTAssertEqual(item.representations[ClipboardUTI.url], Data("https://x".utf8)) - } - - func testEmptyEntryThrows() { - let empty = ClipboardEntry(capturedAt: Date(), kind: .text, key: "", representations: [:], fingerprint: "text:") - XCTAssertThrowsError(try outbound.linkItem(from: empty, origin: mac)) { - XCTAssertEqual($0 as? LinkOutboundError, .noContent) - } - } - - func testFileEntrySendsBytes() throws { - let src = tempDir.appendingPathComponent("doc.pdf") - let bytes = Data((0..<1500).map { UInt8($0 & 0xff) }) - try bytes.write(to: src) - let entry = ClipboardEntry(capturedAt: Date(), kind: .file, key: "doc.pdf", - representations: [ClipboardUTI.fileURL: .inline(Data(src.absoluteString.utf8))], - fingerprint: "file:\(src.path)") - let item = try outbound.linkItem(from: entry, origin: mac) - XCTAssertEqual(item.kind, .file) - XCTAssertEqual(item.suggestedName, "doc.pdf") - XCTAssertEqual(item.representations[LinkOutboundAdapter.fileContentUTI], bytes) - } - - func testUnreadableFileThrows() { - let entry = ClipboardEntry(capturedAt: Date(), kind: .file, key: "ghost", - representations: [ClipboardUTI.fileURL: .inline(Data("file:///nope/ghost.bin".utf8))], - fingerprint: "file:/nope/ghost.bin") - XCTAssertThrowsError(try outbound.linkItem(from: entry, origin: mac)) { - XCTAssertEqual($0 as? LinkOutboundError, .unreadableFile) - } - } - - // MARK: Round-trip with the inbound adapter - - func testTextRoundTrip() throws { - let item = try outbound.linkItem(from: textEntry("round trip"), origin: mac) - let inbound = LinkInboundAdapter(inboxDirectory: tempDir.appendingPathComponent("inbox")) - let back = try inbound.entry(from: item) - XCTAssertEqual(back.kind, .text) - XCTAssertEqual(back.data(for: ClipboardUTI.plainText), Data("round trip".utf8)) - XCTAssertEqual(back.peerDeviceName, "Mac") - } - - func testFileRoundTrip() throws { - let src = tempDir.appendingPathComponent("photo.png") - let bytes = Data((0..<3000).map { UInt8(($0 * 7) & 0xff) }) - try bytes.write(to: src) - let entry = ClipboardEntry(capturedAt: Date(), kind: .file, key: "photo.png", - representations: [ClipboardUTI.fileURL: .inline(Data(src.absoluteString.utf8))], - fingerprint: "file:\(src.path)") - - let item = try outbound.linkItem(from: entry, origin: mac) - let inbound = LinkInboundAdapter(inboxDirectory: tempDir.appendingPathComponent("inbox")) - let back = try inbound.entry(from: item) - - let urlData = try XCTUnwrap(back.data(for: ClipboardUTI.fileURL)) - let inboxURL = try XCTUnwrap(URL(string: String(decoding: urlData, as: UTF8.self))) - XCTAssertEqual(try Data(contentsOf: inboxURL), bytes, "file content survives the full Mac→wire→Mac round-trip") - XCTAssertTrue(inboxURL.lastPathComponent.hasSuffix("photo.png")) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/PairingHandshakeTests.swift b/Tests/ThreeFingerSwitcherTests/PairingHandshakeTests.swift deleted file mode 100644 index 5f7a17d..0000000 --- a/Tests/ThreeFingerSwitcherTests/PairingHandshakeTests.swift +++ /dev/null @@ -1,28 +0,0 @@ -import XCTest -import CryptoKit -@testable import ThreeFingerSwitcherCore - -/// The Mac-side pinned-peer store. (The pairing crypto — code + handshake, incl. MITM resistance — moved -/// to the shared `DeviceLinkPairing` package; its tests live in `DeviceLinkPairingTests`.) -final class PairedDeviceStoreTests: XCTestCase { - - func testStorePinReloadAndRemove() throws { - let dir = FileManager.default.temporaryDirectory.appendingPathComponent("tfs-pair-\(UUID().uuidString)") - defer { try? FileManager.default.removeItem(at: dir) } - let spki = Data(SHA256.hash(data: Data("peer-spki".utf8))) - - let store = PairedDeviceStore(directory: dir) - XCTAssertFalse(store.isPinned(spkiHash: spki)) - store.add(PairedDevice(id: "phone-1", name: "iPhone", pinnedSPKIHash: spki, pairedAt: Date())) - - // Reload from disk → pin persists. - let reloaded = PairedDeviceStore(directory: dir) - XCTAssertTrue(reloaded.isPinned(spkiHash: spki)) - XCTAssertEqual(reloaded.all().map(\.id), ["phone-1"]) - XCTAssertFalse(reloaded.isPinned(spkiHash: Data(SHA256.hash(data: Data("unknown".utf8))))) - - reloaded.remove(id: "phone-1") - XCTAssertFalse(reloaded.isPinned(spkiHash: spki)) - XCTAssertTrue(reloaded.all().isEmpty) - } -} diff --git a/Tests/ThreeFingerSwitcherTests/QRImageTests.swift b/Tests/ThreeFingerSwitcherTests/QRImageTests.swift deleted file mode 100644 index 7b07ade..0000000 --- a/Tests/ThreeFingerSwitcherTests/QRImageTests.swift +++ /dev/null @@ -1,29 +0,0 @@ -import XCTest -import DeviceLinkProtocol -import DeviceLinkPairing -@testable import ThreeFingerSwitcherCore - -/// The Mac QR image generation round-trips (generate → decode), proving the QR path the Hub Show-code -/// surface renders. The cross-device scan is user-verified. -final class QRImageTests: XCTestCase { - - func testQRRoundTripsAPairingPayload() throws { - let payload = PairingQRPayload(device: DeviceIdentity(id: "mac-1", name: "Mac"), - secret: PairingQRPayload.makeSecret(), - spkiFingerprint: Data((0..<32).map { UInt8($0) })) - let string = payload.encodedString() - - // The image is generated... - XCTAssertNotNil(QRImage.image(from: string), "a QR image is produced") - // ...and decodes back to the same string. - XCTAssertEqual(QRImage.decode(string), string, "the generated QR decodes to the original payload") - - // And the decoded string parses back to the equal payload. - let decoded = try PairingQRPayload(string: QRImage.decode(string) ?? "") - XCTAssertEqual(decoded, payload) - } - - func testQRRoundTripsAShortString() { - XCTAssertEqual(QRImage.decode("tfslink:hello"), "tfslink:hello") - } -} diff --git a/openspec/changes/remove-device-link/design.md b/openspec/changes/remove-device-link/design.md new file mode 100644 index 0000000..7b9d5bb --- /dev/null +++ b/openspec/changes/remove-device-link/design.md @@ -0,0 +1,19 @@ +# Design — remove-device-link + +Removal, not redesign. The two decisions worth recording: + +## D1 — Delete the provenance model instead of stranding it + +`ClipboardOrigin` existed only so the band could badge entries that arrived over the link. With the link gone nothing can mint a `.peer` entry, so keeping the field would be dead schema that every future reader has to reason about. It is deleted from `ClipboardEntry` (property, `CodingKeys`, initializer, `isPeer`/`peerDeviceName`) rather than deprecated. + +**Compatibility:** `Codable` ignores unknown JSON keys, so a persisted index whose entries carry `"origin"` (including `.peer` records received before the removal) decodes cleanly; those entries just render as ordinary local copies. This is codified in `clipboard-history`'s new "Legacy index compatibility" requirement — the replacement for the two deleted provenance requirements. + +## D2 — `suppressSelfWrite` goes with its only caller + +The monitor's one-shot change-count suppression existed solely so the receive path's auto-paste wouldn't be re-captured as a duplicate entry. No other writer uses it (the launcher's paste path *wants* its writes treated normally, and fires through the same pasteboard the user sees). Keeping an untriggerable seam plus its three tests would be exactly the kind of surface the cleanup is for — deleted, with `poll()` reverted to the plain change-count comparison. + +## Explicitly untouched + +- The clipboard store's `materializedEntry`/`recentWindow` full-byte materialization (the band's truncated-preview design needs it regardless of who consumes the bytes). +- The archive (`openspec/changes/archive/`) — all device-link change folders remain as design history. +- `v1` branch / `v1.0.0` tag — the preserved full-featured app. diff --git a/openspec/changes/remove-device-link/proposal.md b/openspec/changes/remove-device-link/proposal.md new file mode 100644 index 0000000..02e6167 --- /dev/null +++ b/openspec/changes/remove-device-link/proposal.md @@ -0,0 +1,28 @@ +# Proposal — Remove the device link (iPhone↔Mac pairing + clipboard/file bridge) + +## Why + +The great cleanup (`remove-local-ai`) refocused the app on the **switcher, the launcher, and clipboard history** — and explicitly kept the device link. On reflection it goes too: remote device syncing is the same kind of drift from the app's intent as the AI stack was. It carries a vendored cross-platform package (`DeviceLinkKit`), a local-network listener + Bonjour advertising (and the macOS Local Network permission that comes with them), QR-pairing crypto, a Hub page, and nine spec capabilities — for a bridge to an iOS companion app that never shipped. + +The full-featured app remains preserved on the **`v1` branch** and the **`v1.0.0` release tag**. + +## What Changes + +**Removed outright (code, tests, specs):** + +- **The link runtime:** `Sources/ThreeFingerSwitcher/DeviceLink/` — `DeviceLinkService`, the transport stack (`LinkByteTransport`/`NWByteTransport`/`SealingByteTransport`), `LinkConnection`, `LinkHandshake`, the `ReceiveHUD`, and the pairing suite (`MacPairingCoordinator`/`MacPairingChannel`/`MacLocalIdentity`/`PairedDevice`/`PairedDeviceStore`/`QRImage`). +- **The clipboard adapters:** `LinkInboundAdapter` (LinkItem → ClipboardEntry + the `inbox/` file landing) and `LinkOutboundAdapter`, plus the coordinator's receive/auto-paste/send-latest paths and the `ClipboardMonitor.suppressSelfWrite` seam (its only production caller was the receive path). +- **The vendored package:** `DeviceLinkKit/` (DeviceLinkProtocol / DeviceLinkPairing / DeviceLinkMirror) and its products from `Package.swift`. +- **The Hub Devices page:** `HubDevicesPage`, the `.devices` destination/rail button, and the `pairedDevices`/`onForgetDevice`/`onSendLatestToDevices`/`pairingCoordinator` context seams. +- **Settings:** the `enableDeviceLink` opt-in (property, default, key, observer, test). +- **Clipboard provenance:** `ClipboardOrigin` and `ClipboardEntry.origin`/`isPeer`/`peerDeviceName`, and the band's `ProvenanceChip` — nothing can produce a `.peer` entry any more. A legacy persisted index carrying an `origin` key still decodes (unknown JSON keys are ignored); old peer entries simply behave as local copies. +- **Info.plist:** `NSLocalNetworkUsageDescription` and the `NSBonjourServices` types (`_tfslink._tcp`, `_tfspair._tcp`) — the app no longer touches the local network. +- **Specs:** the 8 `device-link-*` capability folders + `mac-qr-pairing` are deleted; `tunable-settings` drops the device-link opt-in requirement; `clipboard-history` replaces the two provenance requirements with a legacy-index-compatibility requirement. The archive is untouched (design history). +- **Tests:** `LinkConnectionTests`, `LinkInboundAdapterTests`, `LinkOutboundAdapterTests`, `PairingHandshakeTests`, `QRImageTests`, the three self-write-suppression tests, and the `enableDeviceLink` settings test. + +## Impact + +- `swift build` + `swift test` green: 777 tests (was 805; the deleted suites were device-link-only). +- No behavior change to the keepers. Clipboard capture, band, paste, retention, and exclusions are untouched; the only clipboard delta is that received-from-phone entries can no longer exist. +- Stale UserDefaults keys (`enableDeviceLink`, `deviceLinkLocalID`) are left in place, harmless, per the cleanup convention. A previously granted Local Network permission simply goes unused. +- The `Sources/ThreeFingerSwitcher/KeyboardLanguage/`-style Core-only layout is preserved — everything removed was Core, so no signing/TCC implications. diff --git a/openspec/changes/remove-device-link/tasks.md b/openspec/changes/remove-device-link/tasks.md new file mode 100644 index 0000000..548cfbd --- /dev/null +++ b/openspec/changes/remove-device-link/tasks.md @@ -0,0 +1,17 @@ +# Tasks — remove-device-link + +- [x] Delete `Sources/ThreeFingerSwitcher/DeviceLink/` (service, transports, handshake, ReceiveHUD, pairing suite) +- [x] Delete `Clipboard/LinkInboundAdapter.swift` + `Clipboard/LinkOutboundAdapter.swift` +- [x] Delete `Hub/HubDevicesPage.swift` +- [x] Delete the vendored `DeviceLinkKit/` package; drop its products from `Package.swift` (Core + test target) +- [x] AppCoordinator: remove the `DeviceLinkProtocol` import, `receiveHUD`, `deviceLinkService`, `pairedDeviceStore`, `macPairingCoordinator`, the adapters, `localDeviceIdentity`, `observeDeviceLinkToggle`/`setDeviceLink`, `receiveLinkItem`, `sendLatestClipboardToDevices`, and the Hub context wiring +- [x] HubView: remove the `.devices` destination (title, sidebar, icon, rail button, detail case) and the Devices context seams +- [x] AppSettings: remove `enableDeviceLink` (property, load, default, key) +- [x] ClipboardEntry: remove `ClipboardOrigin`, `origin`, `isPeer`, `peerDeviceName`; keep legacy decode +- [x] ClipboardBandView: remove `ProvenanceChip` and its key-row use +- [x] ClipboardMonitor: remove `suppressSelfWrite`/`suppressedChangeCount`; simplify `poll()` +- [x] Info.plist: remove `NSLocalNetworkUsageDescription` + `NSBonjourServices` +- [x] Tests: delete the 5 device-link suites, the 3 suppression tests, the `enableDeviceLink` settings test +- [x] Specs: delete the 9 device-link/pairing capability folders; trim `tunable-settings`; replace `clipboard-history` provenance requirements with legacy-compat +- [x] Docs: README capability roster (35 → 26) + Hub sidebar listings; CLAUDE.md cleanup note +- [x] Verify: `swift build` + `swift test` green (777 tests) diff --git a/openspec/specs/clipboard-history/spec.md b/openspec/specs/clipboard-history/spec.md index 96ad02d..0abc1ab 100644 --- a/openspec/specs/clipboard-history/spec.md +++ b/openspec/specs/clipboard-history/spec.md @@ -134,25 +134,10 @@ To paste usefully into apps that do not accept the rich type, the system SHALL a - **WHEN** a fired file entry references a file that no longer exists - **THEN** the paste does nothing harmful and the app does not crash -### Requirement: Entry provenance -A `ClipboardEntry` SHALL carry an optional provenance describing where it came from — a device origin distinct from the existing app `sourceApp`. The provenance SHALL be additive and backward-compatible: an entry persisted before this field existed SHALL load successfully with provenance treated as local, and entries created by local pasteboard capture SHALL NOT set a peer origin. The provenance SHALL distinguish a local copy from one received from a paired device (and, for a paired device, MAY carry that device's name). +### Requirement: Legacy index compatibility +A persisted history index written by an older version MAY carry fields this version no longer models (e.g. the removed device-link `origin` provenance). Loading such an index SHALL succeed, ignoring the unknown fields (every entry is treated as a local copy). -#### Scenario: Legacy entries load as local -- **WHEN** a persisted history index written before the provenance field existed is loaded -- **THEN** every entry loads successfully and is treated as local provenance (no decode failure) - -#### Scenario: Local capture is not marked peer -- **WHEN** an entry is created from local pasteboard capture -- **THEN** its provenance is not a peer origin - -### Requirement: Clipboard band shows peer provenance -The Clipboard band SHALL visually mark an entry whose provenance is a paired device, so the user can distinguish a mirrored item from a local copy. The marker SHALL identify the source device when its name is known and SHALL be unobtrusive (it does not change the entry's key text or value preview). - -#### Scenario: Peer entry shows a source chip -- **WHEN** the Clipboard band renders an entry whose provenance is a paired device named "iPhone" -- **THEN** the entry shows a small "from iPhone" marker alongside its key - -#### Scenario: Local entry shows no chip -- **WHEN** the Clipboard band renders a locally-captured entry -- **THEN** no provenance marker is shown +#### Scenario: Legacy entries load +- **WHEN** a persisted history index written by an older version (including one with provenance fields) is loaded +- **THEN** every entry loads successfully with no decode failure diff --git a/openspec/specs/device-link-hub/spec.md b/openspec/specs/device-link-hub/spec.md deleted file mode 100644 index aad3584..0000000 --- a/openspec/specs/device-link-hub/spec.md +++ /dev/null @@ -1,34 +0,0 @@ -# device-link-hub Specification - -## Purpose -TBD - created by archiving change device-link-hub. Update Purpose after archive. -## Requirements -### Requirement: Device-link service lifecycle -The app SHALL run the device-link receive service when, and only when, the `enableDeviceLink` opt-in is on and the app is enabled, starting it on enable and stopping it on disable or quit. Received `LinkItem`s SHALL be routed through the inbound adapter into the existing clipboard store, so they appear in the Clipboard band and reuse retention and lift-to-paste. The local device identity SHALL be derived from the host name. - -#### Scenario: Service starts only when opted in -- **WHEN** the `enableDeviceLink` opt-in turns on while the app is enabled -- **THEN** the service starts advertising and accepting; when the opt-in turns off, the service stops - -#### Scenario: Received items enter the clipboard store -- **WHEN** the service surfaces a received `LinkItem` -- **THEN** it is adapted to a `ClipboardEntry` (files written to the inbox) and inserted into the clipboard store, where the Clipboard band shows it tagged with its source device - -### Requirement: Hub Devices page -The Hub SHALL present a Devices destination with: the `enableDeviceLink` opt-in toggle; the list of paired devices with a Forget action per device; an outbound trigger to send the most recent clipboard item to connected devices; and honest status copy (local-network only; pairing completed on-device; not yet encrypted until the pairing TLS follow-up). - -#### Scenario: Devices page surfaces the opt-in and paired devices -- **WHEN** the user opens the Hub Devices page -- **THEN** it shows the enable toggle and the current paired devices, each with a Forget action - -#### Scenario: Forget removes a pairing -- **WHEN** the user forgets a paired device -- **THEN** that device is removed from the paired-device store and no longer listed - -### Requirement: Local-network and Bonjour declarations -The app bundle SHALL declare `NSLocalNetworkUsageDescription` and the `NSBonjourServices` entry for the device-link service type, so the OS local-network/Bonjour APIs function and the user sees a clear purpose string on the Local Network prompt. - -#### Scenario: Info.plist declares the keys -- **WHEN** the app bundle is built -- **THEN** its Info.plist contains a local-network usage description and the device-link Bonjour service type - diff --git a/openspec/specs/device-link-inbound/spec.md b/openspec/specs/device-link-inbound/spec.md deleted file mode 100644 index 71e80fb..0000000 --- a/openspec/specs/device-link-inbound/spec.md +++ /dev/null @@ -1,57 +0,0 @@ -# device-link-inbound Specification - -## Purpose -TBD - created by archiving change device-link-inbound. Update Purpose after archive. -## Requirements -### Requirement: Map a received LinkItem to a ClipboardEntry -The system SHALL convert a `DeviceLinkProtocol.LinkItem` into a `ClipboardEntry`, mirroring the per-kind representation building used for local pasteboard capture. For each `LinkItemKind` it SHALL populate the `ClipboardEntry.representations` keyed by the same UTI strings used for local capture, derive a single-line `key` via the existing key helpers, and derive a stable `fingerprint` consistent with local capture so peer and local copies of identical content de-duplicate to one entry. Text/richText/url/color/image map to inline representations; `file` is handled by the inbox requirement below. - -#### Scenario: Text item maps to a text entry -- **WHEN** a `LinkItem` of kind `text` carrying UTF-8 plain-text bytes is adapted -- **THEN** the result is a `ClipboardEntry` of kind `text` whose plain-text representation equals those bytes, with a first-line key and a fingerprint equal to the one a local copy of the same text would produce - -#### Scenario: Identical local and peer content de-duplicates -- **WHEN** the same text is first copied locally and later received from a peer -- **THEN** both produce the same fingerprint, so inserting the peer entry bumps the existing entry's recency rather than creating a duplicate - -#### Scenario: Image item carries its bytes and dimensions key -- **WHEN** a `LinkItem` of kind `image` carrying PNG/TIFF bytes is adapted -- **THEN** the entry holds the image representation and a key describing its pixel dimensions - -### Requirement: Persist received files to a dedicated inbox directory -For a `LinkItem` of kind `file`, the system SHALL write the transferred bytes to a dedicated **inbox directory** (a sibling of the store's `blobs/` directory, e.g. `…/clipboard/inbox`), under a name derived from the message id and the item's suggested name, and SHALL produce a `.file` `ClipboardEntry` whose file-URL representation points at the written path. The inbox SHALL be outside the `blobs/` deterministic-naming scheme so the store's blob externalization never overwrites or prunes a received file. The inbox directory SHALL be injectable (for testing) and created on demand. - -#### Scenario: A received file is written and referenced -- **WHEN** a `LinkItem` of kind `file` with bytes B and suggested name N is adapted against an inbox directory D -- **THEN** B is written to a file under D whose name incorporates N, and the resulting `.file` entry's file-URL representation resolves to that file - -#### Scenario: Inbox is created on demand -- **WHEN** the inbox directory does not yet exist and a file item is adapted -- **THEN** the directory is created and the write succeeds - -#### Scenario: A received file references a real, openable path -- **WHEN** a file entry produced from a peer item is later pasted -- **THEN** its file-URL representation is a valid `file://` URL to the inbox copy (so a paste targets a real file, not a dangling reference) - -### Requirement: Stamp peer provenance on received entries -Every `ClipboardEntry` produced from a `LinkItem` SHALL carry `origin = .peer(deviceName:)` taken from the item's originating device identity (the device name when present). Entries produced from local pasteboard capture SHALL NOT be stamped peer (their origin remains unset / local). - -#### Scenario: Peer item is stamped with its device name -- **WHEN** a `LinkItem` whose origin device name is "iPhone" is adapted -- **THEN** the resulting entry's `origin` is `.peer(deviceName: "iPhone")` - -#### Scenario: Missing device name still marks peer -- **WHEN** a `LinkItem` with no origin device name is adapted -- **THEN** the resulting entry's `origin` is `.peer(deviceName: nil)`, distinct from local - -### Requirement: Insert received items through the existing store seam -The adapter's output SHALL be inserted via the existing `ClipboardStore.insert` write path, so received items are subject to the same de-duplication, retention caps (count / bytes / age), pinned-exemption, and band assembly as local copies. There SHALL be no separate peer-only storage or retention path. - -#### Scenario: Retention applies to peer items -- **WHEN** received items exceed the configured retention caps -- **THEN** they are evicted by the same rules as local items (oldest non-pinned first), with pinned entries exempt - -#### Scenario: Received item appears in the Clipboard band -- **WHEN** a peer item is inserted while the Clipboard band is shown on the next launcher open -- **THEN** it appears in the band's recent window like any other entry - diff --git a/openspec/specs/device-link-mirror-store/spec.md b/openspec/specs/device-link-mirror-store/spec.md deleted file mode 100644 index 4c9a543..0000000 --- a/openspec/specs/device-link-mirror-store/spec.md +++ /dev/null @@ -1,38 +0,0 @@ -# device-link-mirror-store Specification - -## Purpose -TBD - created by archiving change device-link-mirror-store. Update Purpose after archive. -## Requirements -### Requirement: MovedItem model and LinkItem mapping -The package SHALL define a `MovedItem` value type recording one moved thing: a stable id, a direction (`sent` or `received`), the item kind, a single-line display title, an optional peer device name, a timestamp, and the materialized representation bytes keyed by UTI. It SHALL provide a builder from a `LinkItem` that carries the representations and derives the title (first non-empty line for text/url, the suggested name for a file, a fixed label for image/color). `MovedItem` SHALL be `Codable` and `Sendable`. - -#### Scenario: Build a moved item from a text LinkItem -- **WHEN** a `text` `LinkItem` is mapped with direction `received` -- **THEN** the `MovedItem` has that direction, kind `text`, a title equal to the text's first line, and the same representation bytes - -#### Scenario: A file item's title is its suggested name -- **WHEN** a `file` `LinkItem` with a suggested name is mapped -- **THEN** the `MovedItem`'s title is that suggested name - -### Requirement: Persistent moved-item store -The package SHALL provide a `MovedItemStore` persisting to an injectable directory as a small JSON index of metadata plus per-representation blob files (so the index stays small and binary payloads are externalized). It SHALL support inserting an item (newest-first; an item with an existing id replaces the prior one), listing items newest-first with their representation bytes materialized from blobs, removing by id, and clearing. Persisted items SHALL reload across store instances with their bytes intact. - -#### Scenario: Insert then list newest-first -- **WHEN** two items are inserted at different times -- **THEN** listing returns them newest-first, each with its representation bytes - -#### Scenario: Bytes survive reload -- **WHEN** an item with binary representation bytes is inserted and a new store instance is opened on the same directory -- **THEN** the item is listed with byte-identical representations (materialized from its blobs) - -#### Scenario: Remove and clear -- **WHEN** an item is removed by id (or the store is cleared) -- **THEN** it (or everything) is gone from the listing and its blob files are deleted - -### Requirement: Count-cap eviction -The store SHALL enforce a configurable maximum item count, evicting the oldest items beyond the cap on insert and deleting their blob files, so the store does not grow without bound. - -#### Scenario: Oldest evicted beyond the cap -- **WHEN** more items than the cap are inserted -- **THEN** only the newest `cap` items remain listed, and the evicted items' blob files are removed - diff --git a/openspec/specs/device-link-outbound/spec.md b/openspec/specs/device-link-outbound/spec.md deleted file mode 100644 index f6dfc4b..0000000 --- a/openspec/specs/device-link-outbound/spec.md +++ /dev/null @@ -1,45 +0,0 @@ -# device-link-outbound Specification - -## Purpose -TBD - created by archiving change device-link-outbound. Update Purpose after archive. -## Requirements -### Requirement: Map a ClipboardEntry to a LinkItem -The system SHALL convert a materialized `ClipboardEntry` into a `LinkItem`: the kind SHALL map 1:1 between `ClipboardKind` and `LinkItemKind`; for text/richText/image/color/url the item SHALL carry the entry's inline representation bytes keyed by the same UTIs; the item SHALL be stamped with a provided **local device identity** as origin; and a fresh message id SHALL be assigned. An entry with no usable inline content SHALL produce a typed error rather than an empty item. - -#### Scenario: Text entry maps to a text item -- **WHEN** a `text` `ClipboardEntry` with inline plain-text bytes is adapted with a local identity -- **THEN** the result is a `LinkItem` of kind `text` whose plain-text representation equals those bytes and whose origin is the local identity - -#### Scenario: An empty entry is rejected -- **WHEN** an entry with no inline representation bytes is adapted -- **THEN** a typed error is thrown (no empty item is produced) - -### Requirement: File entries send their bytes -For a `file` `ClipboardEntry`, the adapter SHALL resolve the entry's `file://` reference, read the referenced file's bytes, and produce a `LinkItem` of kind `file` carrying those bytes with `suggestedName` set to the file's name. If the file cannot be read, a typed error SHALL be thrown. - -#### Scenario: A file entry carries the file's content -- **WHEN** a `file` entry referencing an existing file is adapted -- **THEN** the resulting `file` `LinkItem` carries the file's bytes and a suggested name equal to the file's last path component - -#### Scenario: An unreadable file errors -- **WHEN** a `file` entry references a path that does not exist or cannot be read -- **THEN** a typed error is thrown - -### Requirement: Round-trip fidelity with the inbound adapter -An item produced by the outbound adapter and then processed by the inbound adapter SHALL reconstruct an equivalent `ClipboardEntry` content. For text/url the representation bytes SHALL be preserved; for a file the received inbox file's bytes SHALL equal the original file's bytes. - -#### Scenario: Text round-trips -- **WHEN** a text entry is adapted outbound to a `LinkItem` and then adapted inbound -- **THEN** the resulting entry's plain-text bytes equal the original's - -#### Scenario: File content round-trips -- **WHEN** a file entry is adapted outbound and then inbound -- **THEN** the inbox file the inbound adapter writes holds bytes equal to the original file's - -### Requirement: Service send to connected peers -The transport service SHALL expose sending a `LinkItem` to its currently-connected peers, dispatched on its own serial context. (Choosing a specific target device and the user trigger are outside this capability.) - -#### Scenario: Send forwards to connections -- **WHEN** the service has connected peers and is asked to send an item -- **THEN** each connection sends the item to its peer - diff --git a/openspec/specs/device-link-pairing/spec.md b/openspec/specs/device-link-pairing/spec.md deleted file mode 100644 index 0cbb17e..0000000 --- a/openspec/specs/device-link-pairing/spec.md +++ /dev/null @@ -1,46 +0,0 @@ -# device-link-pairing Specification - -## Purpose -TBD - created by archiving change device-link-pairing. Update Purpose after archive. -## Requirements -### Requirement: High-entropy pairing code -The system SHALL generate a pairing code from a cryptographically-secure random source with at least ~27 bits of entropy (default 8 decimal digits). The code SHALL never be transmitted over the link and SHALL never be used directly as an encryption key. The system SHALL validate a code's format. - -#### Scenario: Generated code has the requested length and is numeric -- **WHEN** a pairing code is generated with the default length -- **THEN** it is 8 decimal digits - -#### Scenario: Code format validation -- **WHEN** a candidate string is checked -- **THEN** it is accepted only if it is the expected number of decimal digits - -### Requirement: Code-authenticated key agreement -Two parties, each with an ephemeral X25519 key pair, SHALL derive a shared confirmation key from the ECDH shared secret, the pairing code, and both public keys, such that both sides compute the **same** confirmation key when (and only when) they used the **same** code. The derivation SHALL be role-independent (the same key regardless of which side is initiator). Confirmation SHALL be by an exchanged HMAC over the agreed key; matching confirmations prove both sides knew the code. The implementation SHALL live in the shared `DeviceLinkPairing` package so the Mac and the iOS app use one tested copy. - -#### Scenario: Same code, both sides agree -- **WHEN** two parties run the handshake with each other's public key and the same code -- **THEN** both derive confirmation values that match, so each accepts the other - -#### Scenario: Different code defeats a man-in-the-middle -- **WHEN** the two parties used different codes (as an active MITM relaying keys would force) -- **THEN** their confirmation values do not match, so confirmation fails and pairing is rejected - -#### Scenario: Shared package hosts the crypto -- **WHEN** either the Mac or the iOS app performs pairing -- **THEN** it uses the `PairingHandshake`/`PairingCode` from the shared `DeviceLinkPairing` package (not a per-platform copy) - -### Requirement: Pinned-peer trust store -Successful pairing SHALL persist a durable trust record for the peer: its identity and a pinned public-key (SPKI) hash, so subsequent sessions authenticate the peer by pin without the code. The store SHALL support adding, removing, listing, and testing whether a given SPKI hash is pinned, and SHALL be persisted to an injectable location. - -#### Scenario: A paired peer is pinned and recognized -- **WHEN** a peer is paired and its SPKI hash recorded, then the store is reloaded -- **THEN** the store reports that SPKI hash as pinned and lists the peer - -#### Scenario: An unknown peer is not pinned -- **WHEN** an SPKI hash that was never paired is tested -- **THEN** the store reports it as not pinned - -#### Scenario: Unpairing removes the pin -- **WHEN** a paired peer is removed -- **THEN** its SPKI hash is no longer reported as pinned and it is not listed - diff --git a/openspec/specs/device-link-protocol/spec.md b/openspec/specs/device-link-protocol/spec.md deleted file mode 100644 index 3e6cf97..0000000 --- a/openspec/specs/device-link-protocol/spec.md +++ /dev/null @@ -1,162 +0,0 @@ -# device-link-protocol Specification - -## Purpose -TBD - created by archiving change device-link-protocol. Update Purpose after archive. -## Requirements -### Requirement: Versioned protocol identity -The protocol SHALL expose a single integer `protocolVersion` constant, and every session SHALL open with a `hello` control frame carrying the sender's `protocolVersion` and a device identity (a stable device id + a human-readable name). A receiver SHALL compare versions and, on an incompatible major version, SHALL refuse the session with a typed error rather than attempting to parse later frames. The wire format SHALL be self-describing enough that a receiver can reject a stream that is not this protocol before allocating for it. - -#### Scenario: Compatible hello is accepted -- **WHEN** a `hello` frame is received whose `protocolVersion` matches the receiver's -- **THEN** the session is accepted and subsequent frames are processed - -#### Scenario: Incompatible version is refused -- **WHEN** a `hello` frame is received with an incompatible `protocolVersion` -- **THEN** the receiver surfaces a typed version-mismatch error and does not process further item frames - -#### Scenario: Non-protocol bytes are rejected early -- **WHEN** a frame is decoded whose magic prefix is not the protocol's -- **THEN** decoding fails with a typed error before any payload is allocated - -### Requirement: Explicit framed message set -The protocol SHALL define a closed, `Codable` set of frames: **control** (`hello`, `ack` carrying the acknowledged message id, `error` carrying a typed code), an **item header** (`itemBegin`), streamed **payload** (`chunk`), and **terminators** (`itemEnd`, `cancel`). Every item-bearing frame SHALL carry the `messageID` it belongs to so frames for different items can be interleaved on one stream. There SHALL be no implicit or untyped frame; an unknown frame tag SHALL decode to a typed error. - -#### Scenario: Each frame carries its message id -- **WHEN** an `itemBegin`, `chunk`, `itemEnd`, or `cancel` frame is constructed -- **THEN** it carries the `messageID` of the item it belongs to - -#### Scenario: Unknown frame tag is a typed error -- **WHEN** a frame with an unrecognized type tag is decoded -- **THEN** decoding yields a typed `LinkProtocolError`, not a crash or silent drop - -### Requirement: Length-prefixed binary codec -The protocol SHALL provide pure `Data`-in/`Data`-out encode and decode functions that frame each message as: a fixed **magic** marker, a **version** byte, a **frame-type** tag, and a big-endian `UInt32` **length** prefix followed by exactly that many payload bytes. The decoder SHALL split a byte stream into frames deterministically, reassemble a frame that arrives across multiple buffer reads, reject a declared length above a configured maximum, and reject a truncated or malformed frame with a typed error. Encode→decode SHALL round-trip every frame type without loss. - -#### Scenario: Round-trip preserves every frame -- **WHEN** any frame is encoded and the bytes are decoded -- **THEN** the decoded frame equals the original - -#### Scenario: Partial buffer reassembles -- **WHEN** a frame's bytes arrive split across two or more reads -- **THEN** the decoder buffers and emits the frame once its full length is available, leaving any trailing bytes for the next frame - -#### Scenario: Oversize length is rejected -- **WHEN** a frame declares a length above the configured maximum -- **THEN** the decoder fails with a typed error and consumes no unbounded memory - -#### Scenario: Truncated frame is rejected -- **WHEN** a frame is decoded whose payload is shorter than its declared length and the stream has ended -- **THEN** the decoder reports a typed truncation error - -### Requirement: Streamed chunked item model -An item SHALL be transmitted as one `itemBegin` header, then one or more `chunk` frames, then one `itemEnd`. The `itemBegin` header SHALL declare the item kind and a **manifest** mapping each representation's UTI to its total byte length, so the receiver knows the complete size before bytes arrive and never needs the whole item buffered to begin handling it. Each `chunk` SHALL carry a representation UTI, a monotonically increasing sequence number, and a bounded slice of that representation's bytes. The sender SHALL be able to emit a small item's frames ahead of, or interleaved with, an in-flight large item's chunks (the framing SHALL not require an item to complete before another begins). - -#### Scenario: Header declares the full manifest before bytes -- **WHEN** an `itemBegin` is produced for an item with representations R -- **THEN** its manifest contains every UTI in R mapped to that representation's exact total byte length - -#### Scenario: Chunks are bounded and ordered -- **WHEN** a representation larger than the chunk bound is sent -- **THEN** it is split into multiple `chunk` frames with consecutive sequence numbers, each no larger than the bound - -#### Scenario: Interleaving a small item ahead of a large one -- **WHEN** a large item's chunks are mid-flight and a small item is enqueued -- **THEN** the small item's `itemBegin`/`chunk`/`itemEnd` frames may be emitted interleaved, distinguished by `messageID`, without waiting for the large item to finish - -### Requirement: Transport-agnostic item DTO -The protocol SHALL define a `LinkItem` value type — a `LinkItemKind` (`text`, `richText`, `image`, `color`, `url`, `file`), a `representations: [String: Data]` map keyed by UTI string, and metadata (`messageID`, optional `suggestedName`, `capturedAt`, and an `origin` device descriptor) — that is `Equatable`, `Sendable`, and carries **no AppKit/UIKit/Network dependency**. The protocol SHALL also expose shared UTI string constants so both ends name representations identically. The DTO SHALL be defined independently of any storage type (e.g. the Mac's `ClipboardEntry`); mapping the DTO to/from storage is explicitly out of this capability. - -#### Scenario: LinkItem is platform-free -- **WHEN** the `DeviceLinkProtocol` module is built -- **THEN** it imports no AppKit, UIKit, or Network framework, and builds under plain `swift build` - -#### Scenario: Shared UTI constants -- **WHEN** either end labels a representation (e.g. plain text, png, file-url) -- **THEN** it uses the protocol's shared UTI constant, so the manifest keys match across devices - -### Requirement: Inbound reassembly state machine -The protocol SHALL provide a pure `InboundAssembler` that consumes decoded frames and emits a completed `LinkItem` (or a typed error) per message. It SHALL track each in-flight `messageID`'s manifest, accumulate `chunk` bytes per representation in sequence, and on `itemEnd` validate that every representation's accumulated byte count equals its manifest total before emitting the item. It SHALL reject a `chunk` for an unknown message, a duplicate or out-of-range sequence, a total exceeding the manifest, and a second `itemBegin` for a live `messageID`; a `cancel` frame SHALL discard that message's partial state. The assembler SHALL perform no I/O and hold only the bytes of items currently in flight. - -#### Scenario: Complete item is emitted on itemEnd -- **WHEN** an item's `itemBegin`, all `chunk`s, and `itemEnd` are fed in order -- **THEN** the assembler emits one `LinkItem` whose representations match the manifest exactly - -#### Scenario: Byte-count mismatch is rejected -- **WHEN** `itemEnd` arrives but a representation's accumulated bytes do not equal its manifest total -- **THEN** the assembler emits a typed error and discards that message's state, emitting no item - -#### Scenario: Cancel discards partial state -- **WHEN** a `cancel` frame arrives for an in-flight `messageID` -- **THEN** the assembler drops that message's accumulated bytes and emits neither an item nor an error for it - -#### Scenario: Chunk for an unknown message is rejected -- **WHEN** a `chunk` arrives whose `messageID` has no live `itemBegin` -- **THEN** the assembler emits a typed error and ignores the chunk - -### Requirement: Protocol error taxonomy -The protocol SHALL define a single `LinkProtocolError` type conforming to `Error`/`LocalizedError`, with a distinct case per failure class (bad magic, unsupported version, unknown frame tag, oversize length, truncated frame, manifest mismatch, unknown message, duplicate/out-of-range sequence, cancelled), each with a clean human-readable message. Transports and feature code SHALL be able to map these at their boundary; the protocol itself SHALL never surface a raw decoding/Foundation error to callers. - -#### Scenario: Every failure maps to a typed case -- **WHEN** any decode or reassembly failure occurs -- **THEN** callers receive a `LinkProtocolError` case, never an untyped or Foundation error - -#### Scenario: Errors carry a clean message -- **WHEN** a `LinkProtocolError` is presented via `localizedDescription` -- **THEN** it yields a concise human-readable string with no raw interpolation of internal state - -### Requirement: Outbound frame-stream encoding -The protocol SHALL provide a pure `FrameStreamEncoder` that maps a `LinkItem` to an ordered sequence of frames: exactly one `itemBegin` (carrying the item's kind and a manifest of every representation's UTI → total byte length, plus the item's metadata), then, per representation, one or more `chunk` frames whose bytes concatenate to that representation, then exactly one `itemEnd`. No `chunk` SHALL exceed the encoder's configurable chunk byte bound. Representations SHALL be emitted in a deterministic order and each representation's chunks SHALL carry 0-based, consecutive sequence numbers. The encoder SHALL perform no I/O. - -#### Scenario: A small item encodes to begin, one chunk, end -- **WHEN** a `LinkItem` with a single representation smaller than the chunk bound is encoded -- **THEN** the result is exactly `[itemBegin, chunk(seq 0), itemEnd]`, the header manifest lists that representation's byte length, and the chunk's bytes equal the representation - -#### Scenario: A large representation is split into bounded chunks -- **WHEN** a representation larger than the chunk bound is encoded with bound B -- **THEN** it is emitted as ceil(size/B) `chunk` frames with consecutive sequence numbers, each at most B bytes, concatenating back to the representation - -#### Scenario: Deterministic output -- **WHEN** the same `LinkItem` is encoded twice -- **THEN** the two frame sequences are identical (stable representation order and sequence numbers) - -### Requirement: Encode/decode round-trip fidelity -A `LinkItem` encoded by `FrameStreamEncoder` and fed frame-by-frame into `InboundAssembler` SHALL reassemble to an equal `LinkItem` (same kind, representations, and metadata), including items with multiple representations of mixed sizes, multi-chunk representations, and empty representations. - -#### Scenario: Multi-representation round-trip -- **WHEN** an item with several representations of different sizes (some larger than the chunk bound) is encoded and reassembled -- **THEN** the reassembled item equals the original - -#### Scenario: Empty representation survives the round-trip -- **WHEN** an item containing a zero-byte representation is encoded and reassembled -- **THEN** the reassembled item still contains that representation (as empty), equal to the original - -### Requirement: LinkPump bridges items and channel bytes -The protocol SHALL provide a pure, synchronous `LinkPump` that composes the frame encoder, codec, decoder, and inbound assembler. It SHALL expose: encoding a `LinkItem` to the ordered byte buffers to write (one per frame); encoding a single control frame to bytes; ingesting received bytes and returning any completed inbound results (`.item` or `.control`); and a stream-end check that fails on a truncated frame. It SHALL perform no I/O and SHALL surface only typed `LinkProtocolError`s on violations. - -#### Scenario: Outbound item becomes per-frame byte buffers -- **WHEN** a `LinkItem` is passed to the pump's outbound encoding -- **THEN** it returns one encoded byte buffer per frame (itemBegin, chunks, itemEnd), each a complete encoded frame - -#### Scenario: Ingest returns completed items -- **WHEN** the encoded buffers for an item are ingested -- **THEN** the pump returns exactly one `.item` equal to the original once `itemEnd` is processed - -#### Scenario: Protocol violation surfaces a typed error -- **WHEN** malformed bytes are ingested -- **THEN** ingest throws a `LinkProtocolError`, not an untyped error - -### Requirement: Loopback fidelity under arbitrary fragmentation -A `LinkItem` encoded by a sender pump and ingested by a receiver pump SHALL reassemble to an equal item regardless of how the encoded bytes are grouped — whether each frame is delivered separately, all frames are concatenated into one buffer, or the byte stream is re-split at arbitrary boundaries. - -#### Scenario: Frame-per-buffer delivery -- **WHEN** each outbound buffer is ingested individually -- **THEN** the item reassembles equal to the original - -#### Scenario: Concatenated delivery -- **WHEN** all outbound buffers are concatenated into one buffer and ingested at once -- **THEN** the item reassembles equal to the original - -#### Scenario: Re-split at arbitrary boundaries -- **WHEN** the concatenated bytes are re-sliced into fixed-size pieces that do not align with frame boundaries and ingested in order -- **THEN** the item reassembles equal to the original - diff --git a/openspec/specs/device-link-qr-pairing/spec.md b/openspec/specs/device-link-qr-pairing/spec.md deleted file mode 100644 index 8e1ec8f..0000000 --- a/openspec/specs/device-link-qr-pairing/spec.md +++ /dev/null @@ -1,35 +0,0 @@ -# device-link-qr-pairing Specification - -## Purpose -TBD - created by archiving change device-link-qr-pairing. Update Purpose after archive. -## Requirements -### Requirement: QR pairing payload -The system SHALL define a versioned `PairingQRPayload` carrying the device identity, a high-entropy pairing secret (at least 32 bytes from a CSPRNG), and the device's long-lived public-key (SPKI) fingerprint. It SHALL encode to a compact string (a `tfslink:` scheme + base64url body) and decode back losslessly, rejecting a wrong scheme, an unsupported version, or missing fields. The secret SHALL never be derived from anything guessable. - -#### Scenario: Payload round-trips -- **WHEN** a payload is encoded to its string and decoded back -- **THEN** the decoded payload equals the original (identity, secret, fingerprint, version) - -#### Scenario: Malformed string is rejected -- **WHEN** a string with the wrong scheme or an unsupported version is decoded -- **THEN** decoding fails with a typed error rather than producing a payload - -### Requirement: Authenticated pairing exchange -The system SHALL provide a pure `PairingExchange` state machine with two roles (host = showed the QR, joiner = scanned it) that exchange `PairingMessage`s. Using the QR secret to authenticate an ephemeral X25519 agreement (confirmation key = `HKDF(ECDH shared secret, salt: secret)`), the parties SHALL exchange and verify HMAC confirmations and, on success, each SHALL pin the other's long-lived public-key fingerprint and identity. The machine SHALL perform no I/O; a transport carries the messages. - -#### Scenario: Matching secret → mutual pinning -- **WHEN** a host and a joiner run the exchange with the same QR secret -- **THEN** the joiner ends pinned to the host's identity + fingerprint, and the host ends pinned to the joiner's, each verifying the other's confirmation - -#### Scenario: Wrong secret defeats a man-in-the-middle -- **WHEN** the two run the exchange with different secrets (as an attacker who could not read the QR would force) -- **THEN** confirmation verification fails and neither side pins the other - -#### Scenario: Tampered confirmation fails -- **WHEN** a confirmation message is altered in transit -- **THEN** the receiving side reports failure and does not pin - -#### Scenario: No I/O -- **WHEN** the exchange runs -- **THEN** it produces and consumes `PairingMessage` values only, performing no networking itself - diff --git a/openspec/specs/device-link-transport/spec.md b/openspec/specs/device-link-transport/spec.md deleted file mode 100644 index 9560b34..0000000 --- a/openspec/specs/device-link-transport/spec.md +++ /dev/null @@ -1,46 +0,0 @@ -# device-link-transport Specification - -## Purpose -TBD - created by archiving change device-link-transport. Update Purpose after archive. -## Requirements -### Requirement: Byte-transport seam -The transport SHALL define an abstract bidirectional byte channel (`LinkByteTransport`) that exposes sending a `Data` buffer, a callback for received `Data`, a callback for close (with an optional error), and an explicit close. The connection logic SHALL depend only on this seam, never directly on `Network.framework`, so it is testable with a mock transport. - -#### Scenario: Connection logic is transport-agnostic -- **WHEN** a `LinkConnection` is constructed with any `LinkByteTransport` -- **THEN** it sends and receives without referencing `Network.framework`, so a mock loopback transport drives it in tests - -### Requirement: Connection handshake and item exchange -A `LinkConnection` SHALL, on start, send a `hello` carrying the local device identity and protocol version. On receiving the peer's `hello`, it SHALL validate version compatibility and, if compatible, record the peer identity and report the handshake; if the major version is incompatible, it SHALL refuse — reporting an error and closing — without processing item frames. It SHALL send a `LinkItem` by writing the pump's outbound buffers to the transport, and SHALL surface every fully-received `LinkItem` via a callback. A malformed/violating inbound stream SHALL tear the connection down with a typed error. - -#### Scenario: Hellos are exchanged and identities learned -- **WHEN** two connections over a loopback transport both start -- **THEN** each learns and reports the other's device identity - -#### Scenario: An item sent on one side arrives on the other -- **WHEN** one connection sends a `LinkItem` after the handshake -- **THEN** the other surfaces an equal `LinkItem` via its received-item callback - -#### Scenario: Incompatible version is refused -- **WHEN** a peer's `hello` carries an incompatible major protocol version -- **THEN** the connection reports an error and closes, and does not surface any item - -#### Scenario: Malformed stream tears down with a typed error -- **WHEN** malformed bytes arrive on the transport -- **THEN** the connection reports a `LinkProtocolError` and closes the transport - -### Requirement: Bonjour service discovery and acceptance -The transport SHALL provide a `DeviceLinkService` that advertises an `NWListener` over a dedicated Bonjour service type with peer-to-peer enabled (so it can use peer-to-peer Wi-Fi for high-bandwidth transfer), accepts incoming connections, wraps each accepted `NWConnection` in the byte-transport + a `LinkConnection`, and surfaces received `LinkItem`s through a single callback for the app to consume. It SHALL support an explicit start/stop lifecycle. - -#### Scenario: Service advertises and accepts -- **WHEN** the service starts -- **THEN** it advertises the device-link Bonjour service with peer-to-peer enabled and accepts an incoming connection into a managed `LinkConnection` - -#### Scenario: Received items are surfaced to the app -- **WHEN** an accepted peer sends an item -- **THEN** the service invokes its received-item callback with the reassembled `LinkItem` - -#### Scenario: Stop tears down -- **WHEN** the service is stopped -- **THEN** the listener is cancelled and all managed connections are closed - diff --git a/openspec/specs/mac-qr-pairing/spec.md b/openspec/specs/mac-qr-pairing/spec.md deleted file mode 100644 index d883d67..0000000 --- a/openspec/specs/mac-qr-pairing/spec.md +++ /dev/null @@ -1,27 +0,0 @@ -# mac-qr-pairing Specification - -## Purpose -TBD - created by archiving change mac-qr-pairing. Update Purpose after archive. -## Requirements -### Requirement: Mac long-lived identity and QR image -The Mac SHALL maintain a long-lived Curve25519 key in the Keychain + its SHA-256 fingerprint, and SHALL generate a scannable QR `NSImage` from a string. QR generation SHALL round-trip (a generated QR decodes back to the original string). - -#### Scenario: QR round-trips -- **WHEN** a string is rendered to a QR image and that image is decoded -- **THEN** the decoded string equals the original - -#### Scenario: Identity persists -- **WHEN** the Mac identity is requested across launches -- **THEN** the same key + fingerprint are returned - -### Requirement: Mac shows a code and accepts a scanner -The Mac SHALL present its pairing QR (its `PairingQRPayload`) on the Hub Devices page and, while shown, advertise a dedicated pairing service and accept a connection, running the host side of the `PairingExchange`. On success it SHALL pin the scanner into `PairedDeviceStore`; on failure it SHALL pin nothing. - -#### Scenario: A scan pairs the new device -- **WHEN** a device scans the Mac's code and the exchange succeeds -- **THEN** the device is added to `PairedDeviceStore` and appears in the Hub's paired list - -#### Scenario: A failed exchange pins nothing -- **WHEN** the exchange fails -- **THEN** no device is added - diff --git a/openspec/specs/tunable-settings/spec.md b/openspec/specs/tunable-settings/spec.md index 801e6c8..4eac37c 100644 --- a/openspec/specs/tunable-settings/spec.md +++ b/openspec/specs/tunable-settings/spec.md @@ -163,17 +163,6 @@ The settings SHALL expose a "Keep clipboard history" opt-in that defaults to OFF - **WHEN** settings saved before this feature are loaded - **THEN** they decode successfully with the opt-in OFF and no clipboard history, and existing settings are not reset -### Requirement: Device-link opt-in -Settings SHALL expose an `enableDeviceLink` opt-in (default OFF) that gates the device-link receive/send service. Like the clipboard-history opt-in, it relocates no native gesture, needs no re-login, and has no `is…Effective` gate — it takes effect immediately when toggled. It SHALL persist across launches, and settings written before it existed SHALL load with it OFF. - -#### Scenario: Default off and persists -- **WHEN** a fresh settings store is read -- **THEN** `enableDeviceLink` is false; setting it true and reloading reads back true - -#### Scenario: Legacy settings load with it off -- **WHEN** settings written before this opt-in existed are loaded -- **THEN** `enableDeviceLink` reads as false (no key present) - ### Requirement: Include-minimized-windows opt-in The system SHALL expose an "include minimized windows in the switcher" opt-in, **off by default**, that makes minimized windows appear in the three-finger switcher and the ⌘-Tab reel — each flagged and badged as minimized — with selection **un-minimizing the window in place** and raising it. It SHALL be independent of the include-non-standard-windows setting (either may be on without the other). It SHALL persist across launches, take effect on the next gesture without a restart, and appear in the Settings UI. "Reset to defaults" SHALL restore it to off (it has no system side effect, permission, or download to preserve). From 2ae9b1537b345f3d6a01bc30ab9bb4a7a15d7421 Mon Sep 17 00:00:00 2001 From: Amitay Keisar Date: Thu, 20 Aug 2026 08:31:02 +0300 Subject: [PATCH 3/5] fix(perf): stop the app getting slower the longer it runs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The app degraded with use — CPU creeping up, switcher previews refreshing slower, the gesture trigger developing a "needs to wake up" lag — clearing only on restart. A four-way audit (repeating-work leaks, event-source lifecycle, unbounded growth, thread/QoS) found independent accumulation bugs plus a scheduling posture that let macOS starve the app under load. Accumulation (the "more use = slower" half): - Vendored OpenMTManager registers its OWN sleep/wake observers; its wake handler double-starts multitouch devices, orphaning one running, callback-registered device per sleep/wake cycle — N wakes meant every touch frame processed N+1 times through one serial queue. TouchEngine now strips those observers at init via the ObjC runtime (fails safe to a no-op); AppCoordinator is the sole owner of sleep/wake restart. - MRUTracker.start()/KeyboardLanguageService.start() were not idempotent and the $enabled sink could ping-pong unbounded in the no-trackpad case (@Published emits on every WRITE; enable() re-writes settings.enabled) — each lap stacked a permanent NSWorkspace activation observer. Guards + .removeDuplicates() on both sinks. - Closing the first-run wizard only suspend()ed its model, leaving the retained NSHostingView's ungated 15-20 Hz TimelineView breathers ticking forever (the postmortem-idle-cpu-spin loop, reachable from the red button). willClose now destroys window + model + hosting tree; attract timer drops its strong self. - Preview sweeps stacked: each 0.8 s tick spawned an uncancelled Task, and once a sweep overran the tick each overlap paid a full SCShareableContent enumeration. Sweeps are now single-slot (skip-if-busy) and cancelled on overlay hide. - ThumbnailService cache was FIFO and never dropped closed windows (~1 MB pinned per entry, two 64-slot instances): now true LRU + pruned to the live cross-Space id set each switcher open. - The 1 Hz permissions poll (six TCC/XPC round-trips per tick) could strand forever when the Hub closed on the Setup page (.onDisappear never fires in a retained window): the tick now no-ops with no regular window visible. Load sensitivity (the "can't compete with other apps" half): - Process-global AXUIElementSetMessagingTimeout(0.5 s): AX calls default to a 6 s per-call timeout serviced by the TARGET app's main thread, and the snapshot makes several per window synchronously inline in the gesture. - App Nap opt-out (NSAppSleepDisabled + beginActivity with .userInitiatedAllowingIdleSystemSleep — deliberately NOT .userInitiated, which would block system idle sleep). - Event taps get a 2 s tapIsEnabled watchdog (the in-band re-enable only runs when the NEXT event limps through, dropping the first post-stall gesture) + CFMachPortInvalidate on stop. - snapshot()'s per-app off-Space brute-force sweeps get one 250 ms aggregate deadline (was: unbounded 100 ms x app count, inline in the trigger). - Dock AX walk throttled to an 80 ms TTL cache (was: full Dock.app AX tree walk per mouse-move at 60-125 Hz near any screen edge); the dock defaults handle is reused; the snap monitor stops installing per-move monitors it never consumed. - The launcher rebuilt its whole SwiftUI graph on EVERY open: the panel stays disposable (ghost-on-Space-switch fix untouched) but the NSHostingView is now built once and re-parented. Spec-first: openspec/changes/fix-progressive-cpu-degradation/ (proposal, design with rejected alternatives + deferred follow-ups, spec deltas for touch-input and menubar-app-shell). CLAUDE.md gains a guardrails section so the odd-looking-but-load-bearing bits (runtime removeObserver, the activity option choice) don't get "cleaned up". Verification: swift build clean, swift test 777/777 green. Real-build checks that need the user's stable-signed install (single-processing across a real sleep/wake, trigger latency under CPU load) are listed as task 3.3. Story-origin: the app aged like milk and the villain was plural — five separate leaks all compounding; the smoking gun was a vendored framework double-starting trackpad devices on every wake. Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 10 ++++ Resources/Info.plist | 6 +++ .../App/AppCoordinator.swift | 52 +++++++++++++++--- .../ThreeFingerSwitcher/App/AppDelegate.swift | 21 ++++++++ .../Dock/AXDockReader.swift | 6 ++- .../Dock/DockPreviewController.swift | 18 ++++++- .../Dock/GlobalCursorMonitor.swift | 9 +++- .../KeyboardLanguageService.swift | 4 ++ .../Onboarding/FirstTouchWizardModel.swift | 6 ++- .../Overlay/LauncherOverlayController.swift | 16 +++++- .../Permissions/PermissionsService.swift | 18 ++++++- .../TouchInput/KeyboardSwitcherTap.swift | 24 ++++++++- .../TouchInput/ScrollEventTap.swift | 26 ++++++++- .../TouchInput/TouchEngine.swift | 31 +++++++++++ .../Windows/MRUTracker.swift | 9 ++++ .../Windows/ThumbnailService.swift | 44 ++++++++++++++- .../Windows/WindowService.swift | 17 +++++- .../fix-progressive-cpu-degradation/design.md | 54 +++++++++++++++++++ .../proposal.md | 39 ++++++++++++++ .../specs/menubar-app-shell/spec.md | 17 ++++++ .../specs/touch-input/spec.md | 21 ++++++++ .../fix-progressive-cpu-degradation/tasks.md | 37 +++++++++++++ 22 files changed, 466 insertions(+), 19 deletions(-) create mode 100644 openspec/changes/fix-progressive-cpu-degradation/design.md create mode 100644 openspec/changes/fix-progressive-cpu-degradation/proposal.md create mode 100644 openspec/changes/fix-progressive-cpu-degradation/specs/menubar-app-shell/spec.md create mode 100644 openspec/changes/fix-progressive-cpu-degradation/specs/touch-input/spec.md create mode 100644 openspec/changes/fix-progressive-cpu-degradation/tasks.md diff --git a/CLAUDE.md b/CLAUDE.md index 41870aa..bb08489 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -30,6 +30,16 @@ New failure-surfacing code inherits this convention (established by the archived - **Failure is observable state, never silence:** a failure transitions to a `.failed` state carrying the clean headline (never leaves a state stuck mid-flight; cancellation is not a failure). A side effect that didn't land becomes `.failed`, never a false "Done." - **Bounded + non-blocking UI:** never surface a background failure via app-modal `NSAlert.runModal()` (it freezes the window) — use an in-window `.failed` row/card + Retry. Cap long messages (`.lineLimit` + `.truncationMode(.middle)`), put raw text behind a "Show details / Copy" disclosure, and keep layouts scroll-safe. +## Progressive-degradation guardrails (change `fix-progressive-cpu-degradation`) + +The app once got slower the longer it ran (growing CPU, stale previews, a gesture trigger that "needed to wake up"). The fixes are small and easy to delete by accident — don't: + +- **`TouchEngine.neutralizeFrameworkSleepWakeObservers()` is load-bearing.** The vendored `OpenMTManager` (binary XCFramework) registers its own sleep/wake observers whose wake handler double-starts multitouch devices — one orphaned, still-running device per sleep/wake cycle, each re-processing every touch frame (N wakes → N+1× processing). The odd-looking ObjC-runtime `removeObserver` at TouchEngine init is the fix; `AppCoordinator` is the sole owner of sleep/wake restart. It fails safe (no-op) if the framework changes. +- **The `AXUIElementSetMessagingTimeout(systemWide, 0.5)` + App Nap opt-out in `AppDelegate` are why the trigger stays instant under load.** AX calls default to a 6 s per-call timeout serviced by the *target* app's main thread, and a napped LSUIElement accessory is demoted exactly when other apps are busy. Keep the activity option `.userInitiatedAllowingIdleSystemSleep` — plain `.userInitiated` would keep the Mac from ever sleeping. +- **Idempotent `start()`s + `.removeDuplicates()` on settings sinks.** `@Published` emits on every WRITE; `enable()` re-writes `settings.enabled` internally, and in the no-trackpad case that ping-ponged forever, stacking `NSWorkspace` activation observers (`MRUTracker`, `KeyboardLanguageService`). New observer-owning services must guard `start()` like `WindowFocusTracker` does. +- **One sweep, one launcher graph, LRU + prune.** `ThumbnailService.prefetch` is single-slot (skip-if-busy, cancelled on overlay hide) — don't re-add fire-and-forget sweep Tasks; its cache is LRU and pruned to the snapshot's live ids on each open. `LauncherOverlayController` reuses ONE `NSHostingView` across its deliberately-disposable panels (the panel destruction is the ghost-on-Space-switch fix; the graph reuse avoids per-gesture SwiftUI construction) — don't move the hosting view back into `makePanel`'s per-call scope. +- **The wizard is destroyed on close, not suspended** (`releaseWizardReferences` + `contentViewController = nil` in the willClose observer) — its `TimelineView` breathers are ungated, so a retained tree is the postmortem spin. If you make the wizard window reusable, gate the breathers like the Hub first. + ## SwiftUI "liveness" animations — the idle-CPU-spin landmine A repeating `TimelineView(.periodic)` / "breathing" animation hosted in an `NSHostingView` **keeps ticking after its window is hidden** — `orderOut` does NOT stop the SwiftUI animation clock, and neither does wrapping it in `.opacity(0)` or `if isActive { … }`. In a window kept alive by **`isReleasedWhenClosed = false`** (the Hub, the first-touch wizard), each tick drives a **non-converging Auto-Layout ⇄ render ⇄ Observation loop** that **pins the main thread at ~100% forever** — which starves the main-thread gesture→switcher path and reads as "the switcher is slow after a break," clearing only on restart. Full repro + stack: **`docs/postmortem-idle-cpu-spin.md`**. **Rule:** gate every repeating animation on **real window visibility** (`NSWindow.occlusionState` / `didChangeOcclusionStateNotification`, or an explicit active flag the controller sets on show/hide) — **never** on `onAppear`/`onDisappear` (they don't fire for a hidden-but-retained window) — or release the hosting controller on close. The Hub gesture preview is a **visibility-gated autoplay** (live finger-tracking + the free-running model driver deleted — the Switcher AND Launcher/band miniatures follow the ghost hand only via the preview's **clockless sync seam**, `GhostSyncPose` frames emitted from inside the gated `TimelineView`, so it owns no timer and a hidden Hub drives nothing; don't "upgrade" it back to a self-ticking driver); the onboarding wizard (`Onboarding/WizardMotion.swift`) uses the same breathers — first-run-only, but the same pattern. diff --git a/Resources/Info.plist b/Resources/Info.plist index 2e225fb..2554239 100644 --- a/Resources/Info.plist +++ b/Resources/Info.plist @@ -23,6 +23,12 @@ LSUIElement + + NSAppSleepDisabled + NSHumanReadableCopyright GPL-3.0. See LICENSE. diff --git a/Sources/ThreeFingerSwitcher/App/AppCoordinator.swift b/Sources/ThreeFingerSwitcher/App/AppCoordinator.swift index 4e2e6ab..af814ea 100644 --- a/Sources/ThreeFingerSwitcher/App/AppCoordinator.swift +++ b/Sources/ThreeFingerSwitcher/App/AppCoordinator.swift @@ -394,6 +394,14 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate private func observeEnabledToggle() { settings.$enabled .dropFirst() + // `@Published` fires on every WRITE (willSet), not every change — and `enable()`/`disable()` + // re-write `settings.enabled` internally. Because the body below is deferred with + // `DispatchQueue.main.async`, the `applyingEnabledToggle` flag is already false again by the + // time the deferred block runs, so a same-value re-emission slipped past the guard. In the + // no-trackpad case (`enable()` can never set `isEnabled = true`) that produced an UNBOUNDED + // main-queue ping-pong — enable → write → emission → enable … — each lap re-running the full + // enable path forever. Dropping same-value emissions ends the cycle after one lap. + .removeDuplicates() .sink { [weak self] on in // Defer off the `willSet` emission so the master toggle updates in place immediately; // enabling/disabling starts or tears down the touch engine + taps, which otherwise stalled @@ -545,6 +553,10 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate windows.append(hub) } guard !windows.isEmpty else { return false } + // Prune the last-good-frame cache to windows that still exist somewhere (the snapshot is the + // full cross-Space enumeration): closed windows' frames otherwise pin image memory and, under + // the LRU cap, evict frames of windows that are still alive. + thumbnails.retain(only: Set(windows.map(\.id))) let grid = SpaceGrouping.group(windows) // When the app has Mission Control open, float the overlay above it (otherwise it renders // behind the MC windows). The elevated config is scoped to this case in `OverlayController`. @@ -705,10 +717,12 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate /// Stop the periodic preview refresh. Idempotent — safe when already stopped (the timer is nil) — so it /// can be paired with every overlay teardown site unconditionally. The thumbnail cache persists as the - /// last-good-frame store. + /// last-good-frame store; only the in-flight SWEEP is cancelled (it would otherwise keep enumerating + + /// capturing into the now-hidden overlay until it drained). private func stopPreviewRefresh() { previewRefreshTimer?.invalidate() previewRefreshTimer = nil + thumbnails.cancelSweeps() } func gestureDidCommit() { @@ -1060,6 +1074,10 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate private func observeKeyboardLanguageToggle() { settings.$keyboardLanguageEnabled .dropFirst() + // Same-value writes re-emit (`@Published` fires on willSet); without de-duplication each + // redundant `true` write re-ran `start()` — which used to stack a permanent extra + // activation observer per lap (now also guarded in the service itself). + .removeDuplicates() .sink { [weak self] on in MainActor.assumeIsolated { guard let self else { return } @@ -1781,12 +1799,21 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate window.setContentSize(NSSize(width: 960, height: 640)) window.center() wizardWindow = window - // Closing the window mid-flow is "later": progress is already persisted, but the - // model's machinery (attract loop, permission polling, the touch feed) must stop. + // Closing the window mid-flow is "later": progress is already persisted, so tear the + // wizard down COMPLETELY — window, model, and hosting tree. suspend() alone left the + // retained NSHostingView's TimelineView breathers (PulseHalo / BreathingGlowBackdrop, + // 15–20 Hz, ungated) ticking in the ordered-out window for the rest of the process — + // the docs/postmortem-idle-cpu-spin.md main-thread spin, reachable from the red button. + // Resume/Replay rebuilds the wizard from scratch (state is persisted in `firstRun`). wizardCloseObserver = NotificationCenter.default.addObserver( forName: NSWindow.willCloseNotification, object: window, queue: .main ) { [weak self] _ in - MainActor.assumeIsolated { self?.wizardModel?.suspend() } + MainActor.assumeIsolated { + let window = self?.releaseWizardReferences() + // Releasing the hosting tree is what actually stops its animation clocks; the + // window is already closing, so no close() here (it would re-enter willClose). + window?.contentViewController = nil + } } } let firstPresentation = !(wizardWindow?.isVisible ?? true) @@ -1823,7 +1850,11 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate showFirstTouchWizard() } - private func closeWizard() { + /// Shared teardown of the wizard's model / observer / window references. Does NOT close or + /// blank the window — callers own that: from willClose the window is already closing; from + /// `closeWizard` the exhale animation still needs the content on screen. Removing the + /// willClose observer here also means a subsequent `close()` cannot re-enter the teardown. + private func releaseWizardReferences() -> NSWindow? { wizardModel?.suspend() onWizardTouchFrame = nil if let wizardCloseObserver { @@ -1833,9 +1864,17 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate let window = wizardWindow wizardWindow = nil wizardModel = nil - guard let window else { return } + return window + } + + private func closeWizard() { + guard let window = releaseWizardReferences() else { return } + // Both exits release the SwiftUI tree with the window (`contentViewController = nil`) — + // the hosting view's TimelineView clocks only stop when the tree is destroyed (see + // docs/postmortem-idle-cpu-spin.md). guard window.isVisible else { window.orderOut(nil) + window.contentViewController = nil window.close() return } @@ -1848,6 +1887,7 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate window.animator().setFrame(window.frame.offsetBy(dx: 0, dy: 10), display: true) }, completionHandler: { window.orderOut(nil) + window.contentViewController = nil window.close() }) } diff --git a/Sources/ThreeFingerSwitcher/App/AppDelegate.swift b/Sources/ThreeFingerSwitcher/App/AppDelegate.swift index 9b9578a..f67d742 100644 --- a/Sources/ThreeFingerSwitcher/App/AppDelegate.swift +++ b/Sources/ThreeFingerSwitcher/App/AppDelegate.swift @@ -1,14 +1,35 @@ import AppKit +import ApplicationServices @MainActor final class AppDelegate: NSObject, NSApplicationDelegate { private var coordinator: AppCoordinator? private var statusItem: StatusItemController? + /// Process-lifetime App Nap opt-out (see `applicationDidFinishLaunching`). Held, never ended. + private var appNapActivity: NSObjectProtocol? func applicationDidFinishLaunching(_ notification: Notification) { NSApp.setActivationPolicy(.accessory) // menu-bar agent: no Dock icon, no main window installMainMenu() + // Bound every Accessibility round-trip this process makes. AX calls are serviced by the + // TARGET app's main thread with a 6-SECOND default timeout — and the switcher's snapshot + // makes several per window, synchronously, inline in the gesture. One busy/hung peer app + // otherwise stalls the gesture for seconds ("the trigger needs to wake up"). Setting the + // timeout on the system-wide element makes it the process-global default: a peer that + // can't answer in 0.5 s is dropped from that snapshot and picked up on the next one. + AXUIElementSetMessagingTimeout(AXUIElementCreateSystemWide(), 0.5) + + // Opt out of App Nap for the process's lifetime. An LSUIElement accessory that is never + // frontmost is the textbook nap target: macOS demotes its threads and coalesces its timers + // exactly when other apps are busy — which starved the touch-frame consumer and both event + // taps ("slow after a break", "can't compete when the Mac is loaded"). The variant that + // still allows idle SYSTEM sleep is used, so the Mac's own sleep is unaffected. + appNapActivity = ProcessInfo.processInfo.beginActivity( + options: [.userInitiatedAllowingIdleSystemSleep], + reason: "Realtime trackpad gesture recognition" + ) + let coordinator = AppCoordinator() self.coordinator = coordinator diff --git a/Sources/ThreeFingerSwitcher/Dock/AXDockReader.swift b/Sources/ThreeFingerSwitcher/Dock/AXDockReader.swift index 9b48f17..7188dd7 100644 --- a/Sources/ThreeFingerSwitcher/Dock/AXDockReader.swift +++ b/Sources/ThreeFingerSwitcher/Dock/AXDockReader.swift @@ -98,9 +98,13 @@ final class AXDockReader: DockReader { width: rect.width, height: rect.height) } + /// One reused defaults handle — constructing a `UserDefaults(suiteName:)` per read is real + /// per-call cost on a path invoked at cursor rates. The handle still reads live values. + private static let dockDefaults = UserDefaults(suiteName: "com.apple.dock") + /// The Dock's orientation, read from the `com.apple.dock` defaults (`bottom`/`left`/`right`). static func orientation() -> DockOrientation { - switch UserDefaults(suiteName: "com.apple.dock")?.string(forKey: "orientation") { + switch dockDefaults?.string(forKey: "orientation") { case "left": return .left case "right": return .right default: return .bottom diff --git a/Sources/ThreeFingerSwitcher/Dock/DockPreviewController.swift b/Sources/ThreeFingerSwitcher/Dock/DockPreviewController.swift index 9e37a06..5136b7e 100644 --- a/Sources/ThreeFingerSwitcher/Dock/DockPreviewController.swift +++ b/Sources/ThreeFingerSwitcher/Dock/DockPreviewController.swift @@ -33,6 +33,10 @@ final class DockPreviewController { private var enabled = false private var snapshot: DockSnapshot? + /// Short-lived cache for `reader.read()` (see `handleCursor`). `nil` snap is cached too — an + /// auto-hidden Dock reads empty, and re-walking it per mouse move is the same waste. + private var cachedDockRead: (at: TimeInterval, snap: DockSnapshot?)? + private static let dockReadTTL: TimeInterval = 0.08 private var shownPID: pid_t? private var emptyPID: pid_t? /// The tile whose native action menu we just opened with a right-click. While the cursor lingers on @@ -102,7 +106,19 @@ final class DockPreviewController { // Edge-gate: only read the Dock when something is shown or the cursor is near a screen edge. guard overlay.isVisible || nearDockEdge(point) else { return } - let snap = reader.read() + // Throttle the AX walk, not the hit-test: `read()` is a full cross-process AX traversal of + // Dock.app plus a running-apps enumeration, and this handler fires per mouse-move event + // (60–125 Hz near an edge) — a large steady main-thread tax competing with the gesture + // path. Tile frames only matter at ~UI rates, so a snapshot ≤80 ms old is reused; the + // hover model still gets every cursor sample at full rate for grace/anchor timing. + let tick = now() + let snap: DockSnapshot? + if let cached = cachedDockRead, tick - cached.at < Self.dockReadTTL { + snap = cached.snap + } else { + snap = reader.read() + cachedDockRead = (at: tick, snap: snap) + } snapshot = snap let tiles = snap?.tiles ?? [] diff --git a/Sources/ThreeFingerSwitcher/Dock/GlobalCursorMonitor.swift b/Sources/ThreeFingerSwitcher/Dock/GlobalCursorMonitor.swift index 9713b46..f1fe526 100644 --- a/Sources/ThreeFingerSwitcher/Dock/GlobalCursorMonitor.swift +++ b/Sources/ThreeFingerSwitcher/Dock/GlobalCursorMonitor.swift @@ -23,12 +23,17 @@ final class GlobalCursorMonitor: CursorMonitor { func start() { guard monitors.isEmpty else { return } - let mask: NSEvent.EventTypeMask = [.mouseMoved, .leftMouseDragged] // Right-click: PASSIVE only. The global monitor observes the right-click delivered to Dock.app // (it cannot consume it → the native Dock action menu still opens unmodified); the local one // covers a right-click into our own panel. Both just report the location. Left down/up follow // the same report-only contract (drag-end detection for snap-to-bind). - install(mask) { [weak self] in self?.onMove?($0) } + // + // The move-monitor pair (the only high-frequency one — it fires per cursor sample) is + // installed only when a consumer actually wired `onMove`: the window-groups snap monitor + // uses just down/up, and 4 per-move monitors firing into a nil closure is pure overhead. + if onMove != nil { + install([.mouseMoved, .leftMouseDragged]) { [weak self] in self?.onMove?($0) } + } install([.rightMouseDown]) { [weak self] in self?.onRightClick?($0) } install([.leftMouseDown]) { [weak self] in self?.onLeftDown?($0) } install([.leftMouseUp]) { [weak self] in self?.onLeftUp?($0) } diff --git a/Sources/ThreeFingerSwitcher/KeyboardLanguage/KeyboardLanguageService.swift b/Sources/ThreeFingerSwitcher/KeyboardLanguage/KeyboardLanguageService.swift index 2ee460a..e326dcf 100644 --- a/Sources/ThreeFingerSwitcher/KeyboardLanguage/KeyboardLanguageService.swift +++ b/Sources/ThreeFingerSwitcher/KeyboardLanguage/KeyboardLanguageService.swift @@ -68,6 +68,10 @@ final class KeyboardLanguageService { /// the baseline context (so the next change can learn it); the first real apply happens on the next /// context change, not on the current source. func start() { + // Idempotent: a second start() without an intervening stop() would overwrite (and thereby + // permanently leak) the activation observer — every duplicate re-runs the full context + // resolution on each app switch. The settings sink can re-emit `true` without a `false` between. + guard activationObserver == nil else { return } activationObserver = NSWorkspace.shared.notificationCenter.addObserver( forName: NSWorkspace.didActivateApplicationNotification, object: nil, diff --git a/Sources/ThreeFingerSwitcher/Onboarding/FirstTouchWizardModel.swift b/Sources/ThreeFingerSwitcher/Onboarding/FirstTouchWizardModel.swift index 0964385..9533676 100644 --- a/Sources/ThreeFingerSwitcher/Onboarding/FirstTouchWizardModel.swift +++ b/Sources/ThreeFingerSwitcher/Onboarding/FirstTouchWizardModel.swift @@ -357,8 +357,10 @@ final class FirstTouchWizardModel: ObservableObject { /// and strip move as one body, demonstrating the exact gesture they invite. private func startAttract() { guard attractTimer == nil, !liveTouchActive else { return } - attractTimer = Timer.scheduledTimer(withTimeInterval: 1.0 / 30.0, repeats: true) { _ in - Task { @MainActor in self.attractTick() } + // [weak self]: a strong capture would make the timer retain the model (and its demo + // SwitcherModel + thumbnails) past teardown if the timer is ever left running. + attractTimer = Timer.scheduledTimer(withTimeInterval: 1.0 / 30.0, repeats: true) { [weak self] _ in + Task { @MainActor in self?.attractTick() } } } diff --git a/Sources/ThreeFingerSwitcher/Overlay/LauncherOverlayController.swift b/Sources/ThreeFingerSwitcher/Overlay/LauncherOverlayController.swift index 5f4e506..b93cc16 100644 --- a/Sources/ThreeFingerSwitcher/Overlay/LauncherOverlayController.swift +++ b/Sources/ThreeFingerSwitcher/Overlay/LauncherOverlayController.swift @@ -14,6 +14,13 @@ final class LauncherOverlayController { /// Called when a RIGHT step pins/unpins the selected clipboard entry. Wired to `ClipboardStore`. var onTogglePin: ((LaunchItem) -> Void)? private var panel: SwitcherPanel? + /// The ONE SwiftUI hosting view, reused across panels. The panel is destroyed per-hide (the + /// ghost-on-Space-switch fix — see `hide()`), but rebuilding the hosting view with it meant a + /// full SwiftUI graph construction on the main thread at trigger time on EVERY launcher open, + /// and left each dead graph's `model` subscription to die with its panel — which any stray + /// AppKit retention (in-flight animation, autorelease) turned into a permanent extra observer + /// fanning out per gesture tick. One graph, one subscription, for the process lifetime. + private var hosting: NSHostingView? private var bands: [ContextBand] = [] private var dwell: Double = 0.5 /// Shared with the wizard's hold-to-continue (Overlay/DwellArmDriver.swift) so the taught @@ -118,7 +125,10 @@ final class LauncherOverlayController { // Space you switch to (verified: a Space-switch action left the launcher visible on the // destination even though isVisible was already false). Closing the window removes it from the // WindowServer entirely; `show()` recreates it fresh on the current Space. + // The hosting view is detached FIRST so the reusable SwiftUI graph deterministically survives + // the window's death (see `hosting`). panel?.orderOut(nil) + panel?.contentView = nil panel?.close() panel = nil } @@ -238,7 +248,11 @@ final class LauncherOverlayController { panel.collectionBehavior = [.fullScreenAuxiliary, .ignoresCycle] panel.isFloatingPanel = true panel.hidesOnDeactivate = false - panel.contentView = NSHostingView(rootView: LauncherView(model: model)) + // Reuse the one SwiftUI graph across panel rebuilds (see `hosting`): the panel is disposable + // (Space-binding), the graph is not (construction cost + model subscription). + let hosting = self.hosting ?? NSHostingView(rootView: LauncherView(model: model)) + self.hosting = hosting + panel.contentView = hosting return panel } diff --git a/Sources/ThreeFingerSwitcher/Permissions/PermissionsService.swift b/Sources/ThreeFingerSwitcher/Permissions/PermissionsService.swift index a678ffd..d0d4187 100644 --- a/Sources/ThreeFingerSwitcher/Permissions/PermissionsService.swift +++ b/Sources/ThreeFingerSwitcher/Permissions/PermissionsService.swift @@ -65,7 +65,7 @@ final class PermissionsService: ObservableObject { pollCount += 1 guard pollTimer == nil else { return } refresh() - pollTimer = pollTimerFactory(interval) { [weak self] in self?.refresh() } + pollTimer = pollTimerFactory(interval) { [weak self] in self?.pollTick() } activationObserver = NotificationCenter.default.addObserver( forName: NSApplication.didBecomeActiveNotification, object: nil, queue: .main ) { [weak self] _ in @@ -73,6 +73,22 @@ final class PermissionsService: ObservableObject { } } + /// One poll tick. The (six cross-process TCC/XPC round-trips) refresh runs only while a regular + /// window is actually visible: the pollers are the wizard and the Hub Setup page, and SwiftUI's + /// `.onDisappear` never fires when their RETAINED window is closed directly (the postmortem's + /// guardrail) — a stranded unbalanced `startPolling()` would otherwise tax the main thread with + /// permission probes once a second for the rest of the process. Overlay panels are excluded on + /// purpose: gesture surfaces poll nothing, and the switcher being open must not re-enable a + /// stranded poll. The `didBecomeActive` refresh still covers returning from System Settings. + private func pollTick() { + // `NSApp` is nil under `swift test` (no NSApplication is ever created there) — poll + // unconditionally in that case so the timer's contract stays testable. + if let app = NSApp, !app.windows.contains(where: { $0.isVisible && !($0 is NSPanel) }) { + return + } + refresh() + } + func stopPolling() { pollCount = max(0, pollCount - 1) guard pollCount == 0 else { return } diff --git a/Sources/ThreeFingerSwitcher/TouchInput/KeyboardSwitcherTap.swift b/Sources/ThreeFingerSwitcher/TouchInput/KeyboardSwitcherTap.swift index a8c248c..1c8e160 100644 --- a/Sources/ThreeFingerSwitcher/TouchInput/KeyboardSwitcherTap.swift +++ b/Sources/ThreeFingerSwitcher/TouchInput/KeyboardSwitcherTap.swift @@ -19,6 +19,11 @@ final class KeyboardSwitcherTap { private var tap: CFMachPort? private var source: CFRunLoopSource? + /// Health watchdog — same rationale as ScrollEventTap's: the in-band `tapDisabledByTimeout` + /// self-heal only runs when the next event arrives, so a system-disabled tap otherwise drops + /// the first post-stall ⌘-Tab. Re-enables independently of event delivery. + private var watchdog: Timer? + private static let watchdogInterval: TimeInterval = 2.0 private(set) var isRunning = false /// Whether ⌘ is currently held (tracked from `flagsChanged`); gates Tab consumption. @@ -70,14 +75,26 @@ final class KeyboardSwitcherTap { source = src CFRunLoopAddSource(CFRunLoopGetMain(), src, .commonModes) CGEvent.tapEnable(tap: tap, enable: true) + watchdog = Timer.scheduledTimer(withTimeInterval: Self.watchdogInterval, repeats: true) { [weak self] _ in + MainActor.assumeIsolated { self?.reviveIfDisabled() } + } + watchdog?.tolerance = Self.watchdogInterval / 2 // cheap check; let the OS coalesce it isRunning = true return true } func stop() { guard isRunning else { return } + watchdog?.invalidate() + watchdog = nil if let source { CFRunLoopRemoveSource(CFRunLoopGetMain(), source, .commonModes) } - if let tap { CGEvent.tapEnable(tap: tap, enable: false) } + if let tap { + CGEvent.tapEnable(tap: tap, enable: false) + // Destroy the mach receive right deterministically — the gate refreshes cycle + // start/stop on every toggle flip, and relying on CF dealloc leaves port teardown + // timing to autorelease. + CFMachPortInvalidate(tap) + } source = nil tap = nil isRunning = false @@ -85,6 +102,11 @@ final class KeyboardSwitcherTap { consumedKeyDowns.removeAll() } + private func reviveIfDisabled() { + guard let tap, !CGEvent.tapIsEnabled(tap: tap) else { return } + CGEvent.tapEnable(tap: tap, enable: true) + } + private func handle(type: CGEventType, event: CGEvent) -> Unmanaged? { let pass = Unmanaged.passUnretained(event) // The system disables the tap if our callback stalls or input is interrupted; re-enable it. diff --git a/Sources/ThreeFingerSwitcher/TouchInput/ScrollEventTap.swift b/Sources/ThreeFingerSwitcher/TouchInput/ScrollEventTap.swift index 4abbe22..0652dae 100644 --- a/Sources/ThreeFingerSwitcher/TouchInput/ScrollEventTap.swift +++ b/Sources/ThreeFingerSwitcher/TouchInput/ScrollEventTap.swift @@ -20,6 +20,13 @@ final class ScrollEventTap { private var tap: CFMachPort? private var source: CFRunLoopSource? + /// Low-frequency health watchdog. The `tapDisabledByTimeout` self-heal in `handle` is delivered + /// through the tap itself, so once the system disables the tap under main-thread congestion the + /// re-enable only runs when the NEXT event limps through — the first post-stall gesture is + /// silently dropped ("the trigger needs to wake up"). This timer re-enables independently of + /// event delivery, bounding a dead tap to one watchdog interval. + private var watchdog: Timer? + private static let watchdogInterval: TimeInterval = 2.0 private(set) var isRunning = false @@ -49,19 +56,36 @@ final class ScrollEventTap { source = src CFRunLoopAddSource(CFRunLoopGetMain(), src, .commonModes) CGEvent.tapEnable(tap: tap, enable: true) + watchdog = Timer.scheduledTimer(withTimeInterval: Self.watchdogInterval, repeats: true) { [weak self] _ in + MainActor.assumeIsolated { self?.reviveIfDisabled() } + } + watchdog?.tolerance = Self.watchdogInterval / 2 // cheap check; let the OS coalesce it isRunning = true return true } func stop() { guard isRunning else { return } + watchdog?.invalidate() + watchdog = nil if let source { CFRunLoopRemoveSource(CFRunLoopGetMain(), source, .commonModes) } - if let tap { CGEvent.tapEnable(tap: tap, enable: false) } + if let tap { + CGEvent.tapEnable(tap: tap, enable: false) + // Destroy the mach receive right deterministically — the gate refreshes cycle + // start/stop on every toggle flip, and relying on CF dealloc leaves port teardown + // timing to autorelease. + CFMachPortInvalidate(tap) + } source = nil tap = nil isRunning = false } + private func reviveIfDisabled() { + guard let tap, !CGEvent.tapIsEnabled(tap: tap) else { return } + CGEvent.tapEnable(tap: tap, enable: true) + } + private func handle(type: CGEventType, event: CGEvent) -> Unmanaged? { // The system disables the tap if our callback stalls or input is interrupted; re-enable it. if type == .tapDisabledByTimeout || type == .tapDisabledByUserInput { diff --git a/Sources/ThreeFingerSwitcher/TouchInput/TouchEngine.swift b/Sources/ThreeFingerSwitcher/TouchInput/TouchEngine.swift index af5ef28..d0379bc 100644 --- a/Sources/ThreeFingerSwitcher/TouchInput/TouchEngine.swift +++ b/Sources/ThreeFingerSwitcher/TouchInput/TouchEngine.swift @@ -1,3 +1,4 @@ +import AppKit import Foundation import QuartzCore import OpenMultitouchSupport @@ -52,6 +53,36 @@ final class TouchEngine { private let manager = OMSManager.shared private var consumer: Task? + init() { + // `manager` (the property initializer above) has already forced the framework's ObjC + // singleton into existence — its init registered the observers we now strip. + Self.neutralizeFrameworkSleepWakeObservers() + } + + /// Strip the vendored framework's OWN sleep/wake observers — AppCoordinator owns that policy. + /// + /// OpenMTManager (the binary XCFramework under OpenMultitouchSupport) registers for + /// `NSWorkspaceWillSleepNotification` / `NSWorkspaceDidWakeNotification` in its init, and its + /// wake handler calls `startHandlingMultitouchEvents` UNCONDITIONALLY: `makeDevice` overwrites + /// its device pointer without stopping or releasing the previous device. Combined with our own + /// coordinator-driven stop/start on the same notifications, every sleep/wake cycle ORPHANED one + /// still-running, callback-registered MTDevice. After N wakes each physical touch frame was + /// processed N+1 times through one serial queue — the progressive "the longer I run, the slower + /// gestures get" degradation, plus a use-after-free window in the framework's willSleep handler + /// (`MTDeviceIsRunning` on the freed device — the EXC_BREAKPOINT the pre-sleep stop in + /// `AppCoordinator.handleWillSleep` works around). + /// + /// The class is reached via the ObjC runtime (it ships as a prebuilt binary; the Swift package + /// doesn't export it), so this degrades to a no-op if a future framework version renames things — + /// worst case we're back to the old behavior, never worse. + private static func neutralizeFrameworkSleepWakeObservers() { + guard let cls = NSClassFromString("OpenMTManager") as? NSObject.Type else { return } + let shared = NSSelectorFromString("sharedManager") + guard cls.responds(to: shared), + let frameworkManager = cls.perform(shared)?.takeUnretainedValue() else { return } + NSWorkspace.shared.notificationCenter.removeObserver(frameworkManager) + } + private var lastCentroid: CGPoint? private var lastTime: CFTimeInterval? private var smoothedVelocity = CGVector.zero diff --git a/Sources/ThreeFingerSwitcher/Windows/MRUTracker.swift b/Sources/ThreeFingerSwitcher/Windows/MRUTracker.swift index 95b2cd3..3b52c41 100644 --- a/Sources/ThreeFingerSwitcher/Windows/MRUTracker.swift +++ b/Sources/ThreeFingerSwitcher/Windows/MRUTracker.swift @@ -10,6 +10,9 @@ final class MRUTracker { private var observer: NSObjectProtocol? func start() { + // Idempotent: a second start() without an intervening stop() would leak the activation + // observer — the no-trackpad toggle path can re-enter enable() without disable() tearing down. + guard observer == nil else { return } // Seed with the current frontmost app. if let front = NSWorkspace.shared.frontmostApplication { promote(front.processIdentifier) @@ -40,4 +43,10 @@ final class MRUTracker { func rank(_ pid: pid_t) -> Int { order.firstIndex(of: pid) ?? Int.max } + + /// Drop pids that no longer correspond to running apps so `order` (scanned linearly by + /// `rank`) can't grow without bound over a long session. Mirrors WindowFocusTracker.evict. + func evict(keepingLive live: Set) { + order.removeAll { !live.contains($0) } + } } diff --git a/Sources/ThreeFingerSwitcher/Windows/ThumbnailService.swift b/Sources/ThreeFingerSwitcher/Windows/ThumbnailService.swift index 85ca46d..9755aff 100644 --- a/Sources/ThreeFingerSwitcher/Windows/ThumbnailService.swift +++ b/Sources/ThreeFingerSwitcher/Windows/ThumbnailService.swift @@ -23,6 +23,15 @@ final class ThumbnailService { private var inFlight: Set = [] + /// The one in-flight refresh sweep (see `prefetch`). Single-slot: a tick that arrives while a + /// sweep is still running is SKIPPED, not queued — without this, every 0.8 s timer tick spawned + /// a fresh unstructured Task, and once a sweep overran the interval (enumeration cost scales + /// with the system's total window count) sweeps stacked without bound: each paid a full + /// `SCShareableContent` enumeration before discovering every window was already in flight. + private var sweepTask: Task? + /// Monotonic sweep token so a finished sweep never clears a slot a newer sweep now owns. + private var sweepGeneration = 0 + /// When set (env var `TFS_THUMB_LOG`), each capture logs its ScreenCaptureKit frame next to the /// window's logical frame so the set-aside/off-screen "degraded" signal can be confirmed and the /// thresholds in `isDegradedCapture` tuned against real data (see the change's task 1.2 / 1.3). @@ -76,12 +85,28 @@ final class ThumbnailService { /// windows still capture; minimized windows never reach here (`snapshot()`'s `isSwitchable` excludes them). func prefetch(_ windows: [WindowInfo]) { guard CGPreflightScreenCaptureAccess() else { return } + // Per-sweep back-pressure: while the previous sweep still runs, this tick is skipped (the + // next one re-captures everything anyway — sweeps are idempotent refreshes, never a queue). + guard sweepTask == nil else { return } // Skip minimized windows: macOS renders no fresh pixels for a minimized window, so a live capture // would be wasted/degraded. They keep their seeded last-good frame or icon and surface live only on // commit (which un-minimizes). Load-bearing now that the include-minimized-windows opt-in can list them. let targets = windows.filter { !inFlight.contains($0.id) && !$0.isMinimized } guard !targets.isEmpty else { return } - Task { await self.refreshBatch(targets) } + sweepGeneration += 1 + let generation = sweepGeneration + sweepTask = Task { + await self.refreshBatch(targets) + if self.sweepGeneration == generation { self.sweepTask = nil } + } + } + + /// Cancel the in-flight refresh sweep (if any). Called when the switcher session ends so a + /// long-running sweep doesn't keep enumerating + capturing into a hidden overlay. + func cancelSweeps() { + sweepGeneration += 1 + sweepTask?.cancel() + sweepTask = nil } /// Capture the given windows NOW and AWAIT completion — used to grab each window's live frame right @@ -108,6 +133,7 @@ final class ThumbnailService { } let byID = Dictionary(content.windows.map { ($0.windowID, $0) }, uniquingKeysWith: { _, new in new }) for w in windows { + if Task.isCancelled { return } // session ended: stop capturing into a hidden overlay guard !inFlight.contains(w.id) else { continue } // Skip a not-cleanly-presented window (parked off every display, or a Stage-Manager strip // proxy); `seed` already shows its cached/icon. Pass the real (AX) frame as the logical frame @@ -350,11 +376,25 @@ final class ThumbnailService { } private func store(_ id: CGWindowID, _ image: NSImage) { - if cache[id] == nil { cacheOrder.append(id) } + // True LRU: a re-store moves the id to the back. The previous insert-order-only (FIFO) + // eviction let long-dead windows outlive actively-refreshed ones, degrading the hit rate + // over a long session (each entry pins up to ~1 MB of image backing store). + cacheOrder.removeAll { $0 == id } + cacheOrder.append(id) cache[id] = image while cacheOrder.count > cacheLimit { let evicted = cacheOrder.removeFirst() cache.removeValue(forKey: evicted) } } + + /// Drop cached frames whose windows no longer exist anywhere (any Space). Closed windows' + /// frames otherwise sit in the cache — pinning image memory — until 64 newer insertions push + /// them out. Callers pass the full cross-Space enumeration (e.g. `snapshot()`'s id set); an + /// empty set is ignored (an enumeration hiccup must not wipe the last-good-frame store). + func retain(only live: Set) { + guard !live.isEmpty else { return } + cacheOrder.removeAll { !live.contains($0) } + cache = cache.filter { live.contains($0.key) } + } } diff --git a/Sources/ThreeFingerSwitcher/Windows/WindowService.swift b/Sources/ThreeFingerSwitcher/Windows/WindowService.swift index 686e3ee..a91c75f 100644 --- a/Sources/ThreeFingerSwitcher/Windows/WindowService.swift +++ b/Sources/ThreeFingerSwitcher/Windows/WindowService.swift @@ -181,6 +181,10 @@ final class WindowService { return legacySnapshot() } let selfPid = getpid() + // One deadline for ALL per-app brute-force sweeps in this snapshot (see the off-Space + // branch below) — bounds the gesture-open worst case regardless of how many apps hold + // off-Space windows. + let bruteForceDeadline = DispatchTime.now() + .milliseconds(250) // Backstop: re-assert the current frontmost app's focused window as most-recent before // ordering, so the current window is index 0 even if an earlier focus event did not resolve a @@ -211,6 +215,7 @@ final class WindowService { // Prune the focus history to currently-enumerated window ids so closed windows don't linger // (ids are unique per window lifetime, so a stale id can never mis-rank a new window). focus.evict(keepingLive: Set(spaceForWindow.keys)) + mru.evict(keepingLive: Set(appsByPid.keys)) let meta = metadata(for: Array(spaceForWindow.keys)) @@ -236,7 +241,17 @@ final class WindowService { element = axCurrentByPid[m.pid]?[wid] } else { if axBruteByPid[m.pid] == nil { - axBruteByPid[m.pid] = Dictionary(bruteForceWindows(pid: m.pid, includeNonStandard: bruteIncludesNonStandard(pid: m.pid)), uniquingKeysWith: { a, _ in a }) + // Aggregate brute-force budget across the WHOLE snapshot: each per-app sweep is + // budgeted (~100 ms of synchronous AX IPC worst case), but the number of apps + // holding off-Space windows grows over a long session, and N × 100 ms sits + // inline in the gesture-open path. Past the aggregate cap, remaining apps skip + // the sweep and resolve via `elementCache` (windows seen reachable before) — + // the next snapshot retries, so coverage self-heals across opens. + if DispatchTime.now() < bruteForceDeadline { + axBruteByPid[m.pid] = Dictionary(bruteForceWindows(pid: m.pid, includeNonStandard: bruteIncludesNonStandard(pid: m.pid)), uniquingKeysWith: { a, _ in a }) + } else { + axBruteByPid[m.pid] = [:] + } } element = axBruteByPid[m.pid]?[wid] } diff --git a/openspec/changes/fix-progressive-cpu-degradation/design.md b/openspec/changes/fix-progressive-cpu-degradation/design.md new file mode 100644 index 0000000..a398c5d --- /dev/null +++ b/openspec/changes/fix-progressive-cpu-degradation/design.md @@ -0,0 +1,54 @@ +# Design — fix progressive CPU / latency degradation + +## Context + +Live forensics on the installed build plus a four-way code audit identified two failure families: + +1. **Accumulation with use** — work that is created per event/session/wake and never fully torn down, so the per-event cost grows monotonically over a session (the user's "the more I use it the slower it gets"). +2. **Priority starvation under load** — the latency-critical gesture path runs entirely on the main thread at default priority, makes unbounded synchronous cross-process Accessibility calls, and the process carried no App Nap protection — so *other* apps' load became *our* latency (the user's "it can't fight for the Mac's resources"). + +## Decisions + +### D1 — Neutralize the framework's sleep/wake observers via the ObjC runtime (not a fork) + +`OpenMTManager` (prebuilt binary XCFramework) registers `willSleep`/`didWakeUp` observers in its `init`; `didWakeUp` unconditionally re-creates + re-registers + re-starts a multitouch device, and `makeDevice` overwrites its device pointer without stopping/releasing the old one. `AppCoordinator` ALSO restarts the touch engine on the same notifications (it must — the device source dies across sleep), so each full sleep/wake cycle orphaned one running device: N wakes → every touch frame processed N+1 times through the framework's single `dispatch_sync` serial queue. + +Chosen fix: at `TouchEngine` init (after the singleton exists), `NSWorkspace.shared.notificationCenter.removeObserver()`. Rationale over forking/vendoring a patched framework: zero build-pipeline change, degrades to a no-op if the framework renames anything (worst case = old behavior), and the coordinator already owns the sleep/wake policy (its pre-sleep `touchEngine.stop()` also dodges the framework's use-after-free `willSleep` probe, which this removal eliminates outright). + +### D2 — Bound AX with a process-global messaging timeout, not per-call plumbing + +`AXUIElementSetMessagingTimeout(AXUIElementCreateSystemWide(), 0.5)` at launch sets the default for every AX message the process sends. 0.5 s: generous for a merely-busy app (normal replies are sub-millisecond), 12× better worst case than the 6 s default, and a hung app costs at most ~0.5 s per call instead of stalling the gesture indefinitely. A peer that times out drops from that snapshot only; the next open retries. + +### D3 — App Nap opt-out that never blocks system sleep + +`NSAppSleepDisabled` (plist) + `beginActivity(options: [.userInitiatedAllowingIdleSystemSleep])` held for the process lifetime. Explicitly NOT `.userInitiated` (it includes `idleSystemSleepDisabled` — a menu-bar utility must never keep the Mac awake) and NOT `.latencyCritical` (power cost; unnecessary once napping is off). + +### D4 — Single-slot, cancellable thumbnail sweeps (skip, never queue) + +The per-window `inFlight` guard gave per-WINDOW back-pressure but not per-SWEEP: overlapping sweeps each paid a full `SCShareableContent` enumeration. One `sweepTask` slot + a generation token; `prefetch` skips while a sweep runs (sweeps are idempotent refreshes), `stopPreviewRefresh()` cancels the in-flight sweep, `refreshBatch` checks `Task.isCancelled` between windows. `captureNow` (capture-before-minimize) stays await-based and unaffected. + +### D5 — Wizard: destroy on close; the Hub's flag-gating stays the Hub's + +The Hub fixed the postmortem loop with a visibility flag because its window is REUSED. The wizard is rebuilt from persisted state on every entry (`showFirstTouchWizard` checks `wizardWindow == nil`), so the cheaper, stronger fix is teardown: willClose releases window + model + hosting tree (`contentViewController = nil` — the only reliable way to stop a `TimelineView` in a retained host). The shared `releaseWizardReferences()` removes the close observer first so `closeWizard()`'s own `close()` can't re-enter. + +### D6 — Launcher: disposable panel, persistent SwiftUI graph + +The per-hide panel destruction is a verified ghost-on-Space-switch fix and stays. The expensive/leak-prone part was never the `NSPanel` — it was the `NSHostingView` graph (main-thread construction per open; each dead graph's `model` subscription lives until its panel truly deallocates). One hosting view is created lazily and re-parented into each fresh panel; `hide()` detaches it (`contentView = nil`) before `close()` so it deterministically survives the window. + +### D7 — Permissions poll: gate the tick, don't re-plumb the refcount + +The refcount design is fine; the failure mode is a stranded unbalanced `startPolling()` when a retained window closes under a SwiftUI `.onDisappear` that never fires. Rather than teaching every window owner to balance counts, the tick itself no-ops unless a regular (non-`NSPanel`) window is visible — a stranded poll costs an empty 1 Hz timer instead of six cross-process probes per second. `NSApp` is nil under `swift test`; the tick polls unconditionally there to stay testable. + +### D8 — Snapshot brute-force: aggregate deadline over per-app budgets + +Per-app 100 ms budgets bound each sweep but not the sum. A single `DispatchTime` deadline (250 ms) is captured at snapshot start; past it, remaining apps get an empty brute-force map and resolve through `elementCache` (windows seen reachable before stay raisable). Coverage self-heals across opens. 250 ms ≈ 2–3 slow apps' worth — the previous common case — while capping the pathological many-app case. + +### D9 — Dock cursor path: cache the AX walk for 80 ms, cache nothing else + +The "reader caches nothing" contract exists so magnification/auto-hide reveal read fresh — but per-EVENT freshness (60–125 Hz) was never needed for tile frames; the popup's own 0.12 s timer defines the feature's real cadence. An 80 ms TTL snapshot cache (nil results included — an auto-hidden Dock reads empty) keeps every behavior at its designed cadence while cutting the AX walk rate ~10×. The hover model still receives every cursor sample (grace/anchor timing unchanged). + +## Rejected + +- **Forking OpenMultitouchSupport** — heavier to maintain than runtime observer removal; revisit only if the framework's internals change enough to break the removal (which fails safe). +- **`.latencyCritical` activity / dedicated event-tap thread / ThumbnailService off MainActor / warm window-list cache for `snapshot()`** — real candidates, deliberately deferred: each is an architecture change with its own landmines, and the accumulation fixes + timeout + nap opt-out address the reported symptoms directly. Documented as follow-ups in tasks.md §Deferred. +- **Reusing the launcher PANEL** (à la `OverlayController.prewarm`) — conflicts with the verified ghost-on-Space-switch fix; the graph reuse captures the win without the risk. diff --git a/openspec/changes/fix-progressive-cpu-degradation/proposal.md b/openspec/changes/fix-progressive-cpu-degradation/proposal.md new file mode 100644 index 0000000..862399e --- /dev/null +++ b/openspec/changes/fix-progressive-cpu-degradation/proposal.md @@ -0,0 +1,39 @@ +# Fix progressive CPU / latency degradation ("the longer it runs, the slower it gets") + +## Why + +The app degrades over hours of use: CPU creeps up, switcher previews refresh slower and slower, and the gesture trigger develops a "needs to wake up" lag — clearing only on restart. A four-way audit (repeating-work leaks, event-source lifecycle, unbounded growth, thread/QoS topology) traced it to a set of independent accumulation bugs plus a scheduling posture that let macOS starve the app whenever other apps were busy. None of these is behavior the user chose; all are lifecycle/priority defects. + +## What Changes + +**Progressive (accumulating) causes — removed:** + +- **One multitouch device, ever.** The vendored `OpenMultitouchSupport` framework registers its OWN sleep/wake observers; its wake handler re-creates + re-registers a multitouch device WITHOUT stopping the previous one, while `AppCoordinator` independently restarts the touch engine on the same notifications — orphaning one running, callback-registered device per sleep/wake cycle. After N wakes every touch frame was processed N+1 times through one serial queue (growing CPU + touch latency). The app now strips the framework's observers at startup (ObjC-runtime, no-op if the framework changes) — the coordinator is the sole owner of sleep/wake policy. +- **Idempotent observer starts.** `MRUTracker.start()` and `KeyboardLanguageService.start()` guard against double-registration (each duplicate re-ran its work on every app activation, forever), the settings toggle sinks gain `.removeDuplicates()` (`@Published` fires on every write), and the master-enable sink's deferred body no longer ping-pongs unbounded in the no-trackpad case. +- **Wizard teardown destroys the tree.** Closing the first-run wizard window (red button) previously only `suspend()`ed the model, leaving the retained `NSHostingView`'s ungated 15–20 Hz `TimelineView` breathers ticking for the rest of the process — the documented postmortem-idle-cpu-spin loop. willClose now releases the window, model, AND hosting tree; the attract timer also drops its strong `self`. +- **Thumbnail sweeps are single-slot.** Each 0.8 s preview tick spawned an uncancelled sweep task; once a sweep overran the tick (enumeration scales with system window count) they stacked without bound. Sweeps now skip-if-busy, and the session's in-flight sweep is cancelled on overlay hide. +- **Thumbnail cache prunes + true LRU.** Closed windows' frames stayed cached until 64 newer inserts, and re-captures didn't refresh eviction order (FIFO). The cache is now pruned to the live cross-Space window set on each switcher open and evicts least-recently-STORED. +- **Stranded permission polls go quiet.** The 1 Hz permissions poll (six TCC/XPC round-trips per tick) could be pinned forever when the Hub was closed on the Setup page (`onDisappear` never fires for a retained window). The tick now no-ops unless a regular window is actually visible. +- **MRU order is pruned** to running apps on each snapshot (was append-only for the session). + +**Load-sensitive causes ("can't compete with other apps") — bounded:** + +- **AX messaging timeout.** Every Accessibility round-trip (several per window, inline in the gesture) carried the 6-SECOND system default; one busy peer app stalled the trigger for seconds. A process-global 0.5 s timeout is set at launch. +- **App Nap opt-out.** An LSUIElement accessory that is never frontmost is the textbook nap target — macOS demoted the touch consumer + event taps exactly when other apps were busy. `NSAppSleepDisabled` + a process-lifetime `beginActivity(.userInitiatedAllowingIdleSystemSleep)` (system idle sleep unaffected). +- **Event-tap watchdog.** The `tapDisabledByTimeout` self-heal is delivered in-band, so a system-disabled tap dropped the first post-stall gesture. A 2 s watchdog re-enables out-of-band; `stop()` also invalidates the mach port deterministically. +- **Aggregate brute-force budget.** `snapshot()`'s per-app off-Space AX sweeps (≤100 ms each) had no cross-app cap, so gesture-open cost grew with app count; a 250 ms aggregate deadline now bounds the whole snapshot (skipped apps resolve via `elementCache` and self-heal next open). +- **Dock reads throttled.** The per-mouse-move full AX walk of Dock.app (60–125 Hz near any screen edge) is cached for 80 ms; the hover model still sees every cursor sample. The `com.apple.dock` defaults handle is reused. The window-snap monitor no longer installs the per-move monitor pair it never consumed. +- **Launcher graph reused.** The launcher rebuilt its entire SwiftUI hosting view on EVERY open (main-thread graph construction at trigger time, plus a dead-graph subscription leak risk). The panel stays disposable (the ghost-on-Space-switch fix is untouched); the hosting view is now built once and re-parented. + +## Capabilities + +### Modified Capabilities + +- `touch-input`: single-device guarantee across sleep/wake (the app owns sleep/wake policy; the framework's own observers are neutralized); consuming event taps self-heal within a bounded interval independent of event delivery. +- `menubar-app-shell`: the process opts out of App Nap (without preventing system idle sleep) and bounds every Accessibility round-trip with a process-global messaging timeout. + +## Impact + +- **Code:** `TouchInput/TouchEngine.swift`, `TouchInput/ScrollEventTap.swift`, `TouchInput/KeyboardSwitcherTap.swift`, `App/AppDelegate.swift`, `App/AppCoordinator.swift`, `Windows/MRUTracker.swift`, `Windows/WindowService.swift`, `Windows/ThumbnailService.swift`, `Permissions/PermissionsService.swift`, `Overlay/LauncherOverlayController.swift`, `Dock/DockPreviewController.swift`, `Dock/GlobalCursorMonitor.swift`, `Dock/AXDockReader.swift`, `KeyboardLanguage/KeyboardLanguageService.swift`, `Onboarding/FirstTouchWizardModel.swift`, `Resources/Info.plist`. +- **No new permissions, no gesture relocation, no re-login.** All Core → verifies under `swift build` / `swift test`. +- **Behavior:** feature-invisible except (a) a peer app that can't answer AX within 0.5 s drops out of that one snapshot (picked up next open) and (b) past the 250 ms aggregate brute-force budget, additional off-Space apps resolve via the element cache for that open. Both trade rare partial listings for a bounded trigger. diff --git a/openspec/changes/fix-progressive-cpu-degradation/specs/menubar-app-shell/spec.md b/openspec/changes/fix-progressive-cpu-degradation/specs/menubar-app-shell/spec.md new file mode 100644 index 0000000..b8a1ce3 --- /dev/null +++ b/openspec/changes/fix-progressive-cpu-degradation/specs/menubar-app-shell/spec.md @@ -0,0 +1,17 @@ +## ADDED Requirements + +### Requirement: Realtime scheduling posture + +The process SHALL opt out of App Nap for its whole lifetime (a never-frontmost `LSUIElement` accessory is otherwise demoted exactly when other apps are busy — starving the touch consumer and the event taps), and this opt-out SHALL NOT prevent the system's own idle sleep. The process SHALL also bound every Accessibility round-trip it makes with a process-global messaging timeout well under one second (the system default is 6 seconds per call, and the gesture path makes several calls per window synchronously) — a peer app that cannot answer within the timeout drops out of that one enumeration rather than stalling the gesture. + +#### Scenario: Background load does not add trigger latency +- **WHEN** other applications are consuming heavy CPU and the app has been idle in the background +- **THEN** the gesture trigger still fires without a "wake up" delay attributable to App Nap demotion or timer coalescing + +#### Scenario: A hung peer app cannot stall the switcher +- **WHEN** the switcher opens while one running app's main thread is hung +- **THEN** window enumeration completes with that app's windows possibly absent from this snapshot, instead of the trigger blocking for multiple seconds + +#### Scenario: The Mac still sleeps +- **WHEN** the machine reaches its idle-sleep timeout with the app running +- **THEN** system sleep proceeds normally (the opt-out covers App Nap only) diff --git a/openspec/changes/fix-progressive-cpu-degradation/specs/touch-input/spec.md b/openspec/changes/fix-progressive-cpu-degradation/specs/touch-input/spec.md new file mode 100644 index 0000000..4e4b842 --- /dev/null +++ b/openspec/changes/fix-progressive-cpu-degradation/specs/touch-input/spec.md @@ -0,0 +1,21 @@ +## ADDED Requirements + +### Requirement: Single multitouch device across sleep/wake + +The app SHALL own the multitouch listener's sleep/wake policy exclusively: at most ONE registered, running multitouch device SHALL exist at any time, across any number of sleep/wake cycles. The vendored package's own sleep/wake reactions SHALL be neutralized at startup (they double-start devices, orphaning one running, callback-registered device per wake — each orphan re-processes every subsequent touch frame). Neutralization SHALL degrade to a no-op (never a crash) if the package's internals change. + +#### Scenario: Sleep/wake cycles do not multiply touch processing +- **WHEN** the machine sleeps and wakes N times with the switcher enabled +- **THEN** each subsequent touch frame is processed exactly once (not N+1 times), and the app's idle CPU does not grow with wake count + +#### Scenario: Wake restart is the coordinator's alone +- **WHEN** the system posts its wake notifications +- **THEN** only the app's own restart path re-attaches the listener; the package's internal wake handler does not start an additional device + +### Requirement: Consuming event taps self-heal + +Each consuming `CGEventTap` the app installs (scroll consume, ⌘-Tab) SHALL be re-enabled within a bounded interval after the system disables it (`tapDisabledByTimeout` / `tapDisabledByUserInput`), independently of event delivery — the in-band re-enable alone drops the first post-stall event, which presents as a gesture that "needs to wake up". Tap teardown SHALL destroy the underlying mach port deterministically. + +#### Scenario: A system-disabled tap recovers without user input +- **WHEN** the system disables a tap because the main thread stalled past the tap timeout +- **THEN** the tap is re-enabled within the watchdog interval (≤ ~2 s) even if no further event of its kind arrives in between diff --git a/openspec/changes/fix-progressive-cpu-degradation/tasks.md b/openspec/changes/fix-progressive-cpu-degradation/tasks.md new file mode 100644 index 0000000..6770764 --- /dev/null +++ b/openspec/changes/fix-progressive-cpu-degradation/tasks.md @@ -0,0 +1,37 @@ +# Tasks — fix progressive CPU / latency degradation + +## 1. Accumulation fixes + +- [x] 1.1 `TouchEngine`: strip `OpenMTManager`'s own `NSWorkspace` sleep/wake observers at init via the ObjC runtime (`NSClassFromString("OpenMTManager")` → `sharedManager` → `removeObserver`); no-op if the framework renames anything. +- [x] 1.2 `MRUTracker.start()`: idempotency guard (`observer == nil`), mirroring `WindowFocusTracker`; add `evict(keepingLive:)` and call it from `WindowService.snapshot()` next to `focus.evict`. +- [x] 1.3 `KeyboardLanguageService.start()`: idempotency guard (`activationObserver == nil`). +- [x] 1.4 `AppCoordinator`: `.removeDuplicates()` on the `$enabled` and `$keyboardLanguageEnabled` sinks (ends the no-trackpad enable→write→emission ping-pong after one lap; kills same-value re-starts). +- [x] 1.5 Wizard: willClose now fully dismantles (shared `releaseWizardReferences()` + `contentViewController = nil`); `closeWizard()` reuses it and releases the tree in both exit paths; `[weak self]` on the attract timer. +- [x] 1.6 `ThumbnailService`: single-slot `sweepTask` with generation token (`prefetch` skips while busy), `cancelSweeps()` wired into `stopPreviewRefresh()`, `Task.isCancelled` check per window in `refreshBatch`. +- [x] 1.7 `ThumbnailService.store`: true LRU (re-store moves to back); `retain(only:)` pruning wired to the switcher open's full cross-Space id set (empty set ignored — an enumeration hiccup must not wipe the last-good-frame store). +- [x] 1.8 `PermissionsService`: poll tick no-ops unless a regular (non-panel) window is visible; unconditional under `swift test` (NSApp is nil there). + +## 2. Load-sensitivity fixes + +- [x] 2.1 `AppDelegate`: process-global `AXUIElementSetMessagingTimeout(systemWide, 0.5)`. +- [x] 2.2 `AppDelegate` + `Resources/Info.plist`: App Nap opt-out (`beginActivity(.userInitiatedAllowingIdleSystemSleep)` held for the process lifetime; `NSAppSleepDisabled` = true). System idle sleep unaffected. +- [x] 2.3 `ScrollEventTap` / `KeyboardSwitcherTap`: 2 s `tapIsEnabled` watchdog (tolerance 1 s) re-enabling out-of-band; `CFMachPortInvalidate` on `stop()`. +- [x] 2.4 `WindowService.snapshot()`: 250 ms aggregate brute-force deadline across all per-app off-Space sweeps (past it, apps resolve via `elementCache`; next snapshot retries). +- [x] 2.5 `DockPreviewController`: 80 ms TTL cache for `reader.read()` in `handleCursor` (nil cached too); `AXDockReader`: reuse one `UserDefaults(suiteName: "com.apple.dock")` handle. +- [x] 2.6 `GlobalCursorMonitor.start()`: install the per-move monitor pair only when `onMove` is wired (the window-snap consumer uses only down/up). +- [x] 2.7 `LauncherOverlayController`: one lazily-built `NSHostingView` re-parented across disposable panels; `hide()` detaches it before `close()`. + +## 3. Verification + +- [x] 3.1 `swift build` clean (no new warnings in touched files). +- [x] 3.2 `swift test` — full suite green (777 tests, 0 failures; the poll-tick NSApp guard keeps `PermissionsPollingTests` deterministic). +- [ ] 3.3 On the user's stable-signed build: after a sleep/wake cycle, confirm gestures stay single-processed (idle CPU ~0%, no growth across wakes); confirm previews stay fresh during a long switcher dwell; confirm the trigger stays instant with a CPU-loaded background app. + +## Deferred (follow-ups, deliberately out of scope — see design.md Rejected) + +- [ ] Move `WindowService.snapshot()` off the synchronous trigger path (warm window-list cache reconciled off-gesture; show-from-cache instantly). +- [ ] Move `ThumbnailService` capture machinery off the MainActor (actor + main-hop delivery). +- [ ] Dedicated run-loop thread (`.userInteractive`) for both CGEvent taps. +- [ ] Replace the AsyncStream + `.share()` touch delivery with a direct callback hop (removes two 1-deep frame-dropping buffers + the compiled-out priority-escalation inversion in the vendored wrapper). +- [ ] `BrowserContextMonitor`: the synchronous AppleEvents host read can block the main thread on a loaded browser; consider an async seam or timeout. +- [ ] `ClipboardStore.persist`: memoize blob names so a copy stops re-hashing every large inline payload. From b8ea550062945200a03aca4d22824042c8ef98ea Mon Sep 17 00:00:00 2001 From: Amitay Keisar Date: Thu, 20 Aug 2026 21:44:04 +0300 Subject: [PATCH 4/5] =?UTF-8?q?fix(perf):=20second=20sweep=20=E2=80=94=20h?= =?UTF-8?q?ot-path=20subprocess,=20two=20latent=20crashes,=20a=20system-wi?= =?UTF-8?q?de=20stuck=20state?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Five fresh-lens audits (retain cycles, main-thread I/O, SwiftUI render storms, retry/poll correctness, crash/safety) over the post-first-sweep tree. Everything confirmed is fixed; the full itemized list is in openspec/changes/fix-progressive-cpu-degradation/tasks.md §3. The headline finds: - With Space-row switching on, EVERY switcher open shelled out to /usr/bin/defaults on the main thread (fork + waitpid, 30-100 ms) via isSpaceRowSwitchingEffective — at exactly the moment the overlay should appear. The `&&` short-circuited it away on default installs, which is why it hid. openSwitcher now reads the recognizer's cached gate. - CGWindowID(NSWindow.windowNumber) traps on the <= 0 number of the retained-but-closed Hub — on every switcher commit and preview tick after the Hub had been opened once. One hubWindowID chokepoint. - Unguarded `as! AXUIElement` / `as! AXValue` on cross-process AX data (traps when a misbehaving app's AX server answers with another CF type) → typed axElement/axValue helpers. - currentFingerCount was never reset when the touch stream stopped: a sleep with three fingers down left the scroll tap swallowing EVERY scroll in every app until quit. Reset on disable/sleep/wake-restart plus a 0.5 s staleness guard in the consume predicate. - missionControlOpen latched true when MC was closed any way but ours, so every later commit posted a stray Escape into the user's app and every open floated at screen-saver level. Cleared on regular-app activation, Space change, sleep, hideOverlay. Main-thread cost: clipboard capture no longer walks the whole payload through Data's iterator nor fully decodes every image (bounded sample hash + ImageIO header dims); the browser AX tree-walk re-validates the remembered address field instead of re-walking at 2 Hz; StageManager's cfprefsd round-trip is TTL-cached; launcher/Hub icon lookups go through a process-wide IconCache (stable NSImage identity also stops SwiftUI re-rasterizing every cell per step); FavoritesStore coalesces its per-keystroke full-tree JSON save; blob-backed clipboard previews are memoized; FirstRunStore mirrors its stage in memory (it was a defaults read per touch frame); emptyTrash and script runs no longer hold a thread (terminationHandler, not waitUntilExit). Deferred actions are tokened (Space-settle poll, de-minimize raise, focus recovery, MC-dismiss commit, new-window single-flight, seed retries); WindowFocusTracker's AX source is in .commonModes with add, teardown and deinit agreeing (a mode mismatch there leaves a source pointing at freed memory); the TouchEngine consumer is generation- tagged; observer tokens are held and removed in deinit; the two Swift-6 isolation warnings (KeepAwakeController statics, CursorMonitor conformance) are fixed at the root. Verification: swift build clean; swift test 783/783 (6 new ResourceBoundsTests pin the LRU/prune + MRU eviction). README and CLAUDE.md guardrails updated; spec deltas extended (scroll consumption never outlives the touch stream; no subprocess/disk I/O on the gesture path). Story-origin: asked "what did we miss?" and the answer was a fork() in the gesture path, a crash that waited for you to open Settings once, and a sleep that could eat every scroll on the Mac. Story-thread: continues the progressive-degradation arc — the first sweep found the leaks, this one found the cliffs. Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 5 + README.md | 11 +- .../App/AppCoordinator.swift | 198 ++++++++++++++---- .../ThreeFingerSwitcher/App/AppDelegate.swift | 1 + .../App/InputActivityMonitor.swift | 7 + .../App/KeepAwakeController.swift | 11 +- .../Clipboard/ClipboardMonitor.swift | 36 +++- .../Clipboard/ClipboardStore.swift | 114 ++++++++-- .../Dock/AXDockReader.swift | 8 +- .../Dock/CursorMonitor.swift | 9 + .../Dock/DockPreviewController.swift | 32 ++- .../ThreeFingerSwitcher/Hub/BandsCanvas.swift | 6 +- .../ThreeFingerSwitcher/Hub/HubControls.swift | 22 +- .../Hub/HubFeaturePages.swift | 18 +- .../Hub/HubSwitcherDemo.swift | 6 +- .../Hub/HubWindowInspector.swift | 15 +- .../KeyboardLanguage/AXHostProvider.swift | 21 +- .../AppleEventsHostProvider.swift | 11 +- .../BrowserContextMonitor.swift | 1 + .../KeyboardLanguageService.swift | 4 + .../Launcher/FavoritesStore.swift | 41 +++- .../Launcher/IconCache.swift | 26 +++ .../Launcher/LaunchService.swift | 98 ++++++--- .../Onboarding/FirstRunState.swift | 16 +- .../Onboarding/FirstTouchWizardModel.swift | 3 +- .../Onboarding/FirstTouchWizardView.swift | 14 +- .../Overlay/LauncherView.swift | 36 +++- .../Permissions/PermissionsService.swift | 4 +- .../TouchInput/KeyboardSwitcherTap.swift | 3 + .../TouchInput/ScrollEventTap.swift | 3 + .../TouchInput/TouchEngine.swift | 14 +- .../Windows/AXPrivate.swift | 15 ++ .../Windows/HubSwitcherEntry.swift | 5 +- .../Windows/MRUTracker.swift | 4 + .../Windows/StageManager.swift | 19 +- .../Windows/ThumbnailService.swift | 11 +- .../Windows/WindowFocusTracker.swift | 15 +- .../Windows/WindowService.swift | 72 +++++-- .../Windows/WindowSnapMonitor.swift | 22 +- .../ResourceBoundsTests.swift | 89 ++++++++ .../proposal.md | 10 + .../specs/menubar-app-shell/spec.md | 12 ++ .../specs/touch-input/spec.md | 8 + .../fix-progressive-cpu-degradation/tasks.md | 53 ++++- 44 files changed, 922 insertions(+), 207 deletions(-) create mode 100644 Sources/ThreeFingerSwitcher/Launcher/IconCache.swift create mode 100644 Tests/ThreeFingerSwitcherTests/ResourceBoundsTests.swift diff --git a/CLAUDE.md b/CLAUDE.md index bb08489..571e861 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -39,6 +39,11 @@ The app once got slower the longer it ran (growing CPU, stale previews, a gestur - **Idempotent `start()`s + `.removeDuplicates()` on settings sinks.** `@Published` emits on every WRITE; `enable()` re-writes `settings.enabled` internally, and in the no-trackpad case that ping-ponged forever, stacking `NSWorkspace` activation observers (`MRUTracker`, `KeyboardLanguageService`). New observer-owning services must guard `start()` like `WindowFocusTracker` does. - **One sweep, one launcher graph, LRU + prune.** `ThumbnailService.prefetch` is single-slot (skip-if-busy, cancelled on overlay hide) — don't re-add fire-and-forget sweep Tasks; its cache is LRU and pruned to the snapshot's live ids on each open. `LauncherOverlayController` reuses ONE `NSHostingView` across its deliberately-disposable panels (the panel destruction is the ghost-on-Space-switch fix; the graph reuse avoids per-gesture SwiftUI construction) — don't move the hosting view back into `makePanel`'s per-call scope. - **The wizard is destroyed on close, not suspended** (`releaseWizardReferences` + `contentViewController = nil` in the willClose observer) — its `TimelineView` breathers are ungated, so a retained tree is the postmortem spin. If you make the wizard window reusable, gate the breathers like the Hub first. +- **Never read `is…Effective` / `isFree` / `isClaimed` on the gesture path.** The native-gesture configs (`VerticalGestureConfig`, `FourFingerGestureConfig`, …) answer those by **spawning `/usr/bin/defaults`** (fork + waitpid, 30–100 ms). `openSwitcher` reads `recognizer.rowSwitchingEnabled` (the gate `refreshRowSwitchingGate()` computed when the opt-in last changed) — keep it that way. Same rule for anything else that forks, hits disk, or walks a full payload: off the gesture path, cached for it (`IconCache`, `ClipboardStore.boundedCache`, `FirstRunStore`'s stage mirror, `StageManager`'s TTL). +- **`CGWindowID(NSWindow.windowNumber)` traps.** `windowNumber` is ≤ 0 for a window with no window device (the retained, closed Hub), and `CGWindowID(Int)` aborts on a negative. Use `AppCoordinator.hubWindowID` / guard `> 0` — this crashed every switcher commit after the Hub had been opened once. +- **No `as! AXUIElement` / `as! AXValue` on Accessibility reads.** `axCopy` is untyped and a misbehaving app's AX server can answer with any CF type; use `axElement(_:_:)` / `axValue(_:_:)` (type-checked) from `AXPrivate.swift`. +- **The scroll tap's finger count is time-bounded.** `currentFingerCount` is reset wherever the touch engine stops AND treated as 0 once `lastTouchFrameTime` is > 0.5 s old — a sleep with three fingers down used to leave the tap swallowing every scroll in every app until quit. `missionControlOpen` is likewise cleared on regular-app activation / Space change / sleep / `hideOverlay` (stale-true posted a stray Escape into the user's app on every commit). +- **Deferred actions carry a token.** Every `asyncAfter` / retry chain on the commit or Space-switch path checks `commitSeq` / a generation counter before acting (`afterSpaceSettles`, `raiseDeminimizing`, `recover`, the MC-dismiss deferral, the seed-retry sweeps). A new deferred step without one re-introduces "the previous target steals focus back". ## SwiftUI "liveness" animations — the idle-CPU-spin landmine diff --git a/README.md b/README.md index b409d58..780c42c 100644 --- a/README.md +++ b/README.md @@ -26,13 +26,14 @@ A lightweight **macOS menu-bar app** that recreates the **Windows Precision Touc - **Window switcher (core, always on).** Put **three fingers** on the trackpad and **slide left/right** → a live highlight scrubs across individual windows, one at a time. **Lift** to commit — the highlighted window is raised and focused. Works **across all Spaces**, including other desktops and full-screen apps. - **Space-row switching (optional).** With the switcher open, **slide up/down** → switch which **Space's** row of windows you're scrubbing (a 2D grid: horizontal = windows, vertical = Spaces). This is an **opt-in** (default off): turning it on moves Mission Control / App Exposé to four fingers, and the app then synthesizes them itself on idle three-finger up/down (see keystone). -- **Dock window previews (optional — the switcher by *mouse*).** An opt-in (default off): hover an app's **Dock icon** with the cursor and a row of that app's **windows on the current Space** (including **minimized**) fans out above the tile. Hovering a window tab brings the **real window to the front** so you see its live content (and a crisp static thumbnail lands in the tab); moving away **restores** the window you came from, and **clicking** a tab keeps it forward. It's the mouse-flow counterpart to the three-finger switcher — the app's one deliberately **cursor-driven** surface. Apps with no windows on the current Space show nothing. Like Clipboard/Files it **relocates no gesture and needs no re-login or new permission** (it reads the Dock's Accessibility tree and reuses the Screen Recording grant for thumbnails). Toggle it under **Hub → Switcher → Dock window previews**. +- **Dock window previews (optional — the switcher by *mouse*).** An opt-in (default off): hover an app's **Dock icon** with the cursor and a row of that app's **windows on the current Space** (including **minimized**) fans out above the tile. Hovering a window tab brings the **real window to the front** so you see its live content (and a crisp static thumbnail lands in the tab); moving away **restores** the window you came from, and **clicking** a tab keeps it forward. It's the mouse-flow counterpart to the three-finger switcher — the app's one deliberately **cursor-driven** surface. Apps with no windows on the current Space show nothing. Like Clipboard history, it **relocates no gesture and needs no re-login or new permission** (it reads the Dock's Accessibility tree and reuses the Screen Recording grant for thumbnails). Toggle it under **Hub → Switcher → Dock window previews**. - **Four-finger launcher (optional).** Slide **four fingers** horizontally → a launcher overlay of your favorites (apps, folders, URLs, Shortcuts, scripts, and "preset" workspaces) organized into **context bands**. The bands are a **vertical strip of band icons down the left** (each band has its own icon; only the active one is colored), with the active band's items in a grid on the right — **vertical** scrubs between bands, **horizontal** crosses into the grid. Scrub to an item, then **dwell** (≈500 ms, haptic tick + charge-ring) and **lift to fire**; a quick flick lifts off without firing. Once open you can **relax to two fingers** to navigate comfortably. Navigation is **step-based** (an odometer) — scrub to move one item per step, and **hold a contact at the trackpad edge** to auto-repeat, accelerating the longer you hold. Launching always **opens a window in the *current* Space** (or pulls a single-window app to you) instead of teleporting you away. Also an opt-in (default off). - **Clipboard history (optional, lives in the launcher).** A further opt-in (default off — it records copied content, so privacy-gated): when on, what you copy — text, images, files/folders, colors, links — is stored locally and shown as the **last band** in the launcher. Scrub the key list, see a live preview on the right (image / QuickLook file content / text / color), **lift to paste** it into the app you were in; **swipe right** on an entry to pin it, **left** to leave the band. Unlike the two gesture opt-ins it **relocates no gesture and needs no re-login or new permission**. - **Per-app & per-site keyboard language (optional).** An opt-in (default off) that gives macOS the Windows-style per-window keyboard language it lacks: it **remembers the input source you use in each app** — and, inside browsers, **per website (host)** — and **restores it automatically** on every app/page switch. It learns as you change the language by hand (no list to fill in). No gesture relocation and no re-login; per-site host reading uses Accessibility by default, with an optional Apple Events path for browsers that don't expose the URL. - **Mission Control / App Exposé always available** — natively on three-finger up/down when the opt-ins are off, or app-synthesized when they're on. The app never blocks the OS (see the keystone below). - **One configuration Hub.** Every setting, the favorites/bands editor, and the permissions/setup flow live in a **single Hub window** (menu → **Open Hub…**) with an Overview landing page and a grouped sidebar (Bands · Switcher · Launcher · Clipboard · Keyboard Language · Setup · General). The menu-bar menu is trimmed to quick actions. - No keypresses, no clicks — pure trackpad. (The lone exception is the optional **Dock window previews**, which are mouse-hover-driven by nature.) +- **Built to stay fast for days, not hours.** v2.0 closed a family of slow leaks that made the app degrade the longer it ran (the vendored multitouch framework orphaning a trackpad listener per sleep/wake, stacked preview sweeps, retained hidden animation clocks, unbounded caches), and changed its scheduling posture so a busy Mac can't starve it: it **opts out of App Nap** (system sleep unaffected), **bounds every Accessibility round-trip to 0.5 s** (the OS default is 6 s per call, answered by the *other* app's main thread), and re-arms its event taps out-of-band if macOS disables them under load. Idle CPU should read ~0% indefinitely; sustained idle CPU is a bug (see CLAUDE.md's guardrails). **Platform:** built and tested on **macOS 26 (Tahoe)**; deployment target **macOS 15.0+**. Apple Silicon + Intel (universal dep). **License: GPL-3.0.** @@ -112,6 +113,7 @@ This project was built spec-first with **OpenSpec**. The **canonical behavior** - **Optional features:** `optional-space-row-gesture` (runtime gesture ownership — the scroll tap + Mission Control synthesis substrate), `four-finger-launcher` (the launcher, favorites model, launch strategies, dwell-to-arm), `launcher-two-finger-nav` (drop-to-two-finger navigation), `launcher-vertical-band-list` (the band strip as a vertical icon list on the left; vertical switches bands, horizontal crosses into the grid), and `clipboard-history` (the launcher's Clipboard band: a polled-`changeCount` recorder + on-disk store, paste-into-front, pin/edge-accel — its `design.md` covers the band-not-keyboard-picker decision). - **Shell & later features:** `configuration-hub` (folds the Settings/Favorites/Setup windows into one **Hub**) and `hub-in-switcher` (a Hub item in the switcher so it's reachable in accessory mode); `per-app-keyboard-language` / `per-site-keyboard-language` (auto-remember the input source per app, and per website inside browsers). - **The great cleanup (`remove-local-ai`):** the on-device AI stack (Gemma/MLX runtime, the AI command band + canvas, background agents, the notch timeline, voice/computer-use) and the Files band were **removed** to refocus the app on the switcher, the launcher, and clipboard history. The `v1` branch / `v1.0.0` release preserve the full-featured app; the archived `ai-*` / `files-*` changes remain as design history. +- **Performance posture (v2.0):** `fix-progressive-cpu-degradation` — the progressive-slowdown investigation and its fixes (its `design.md` lists the rejected alternatives and the deferred architectural follow-ups: a warm window-list cache off the trigger path, `ThumbnailService` off the MainActor, a dedicated event-tap thread). CLAUDE.md's "Progressive-degradation guardrails" section names the pieces that look odd but are load-bearing. - **Dock window previews:** `dock-window-previews` (the mouse-hover Dock preview — read its `design.md` for the **reversed peek decision**: the peek *fronts the real window* via the SkyLight `setFront` handshake + `kAXMain`/`kAXFocusedWindow` and **restores** it on leave, because macOS won't render fresh pixels for an off-screen window; the tab itself is a static last-good capture, not a stream). ### B1. Repo map @@ -149,7 +151,7 @@ Sources/ThreeFingerSwitcher/ ── ThreeFingerSwitcherCore library (AL Sources/ThreeFingerSwitcherApp/main.swift thin executable: import Core, runThreeFingerSwitcher() Sources/TouchSpike/ throwaway harness to print raw touch frames (swift run TouchSpike) Sources/LauncherSpike/ throwaway harness for the launcher spikes (haptics, window move) — not bundled -Tests/ThreeFingerSwitcherTests/ 805 XCTest unit tests (pure-logic core; @testable import ThreeFingerSwitcherCore) +Tests/ThreeFingerSwitcherTests/ 783 XCTest unit tests (pure-logic core; @testable import ThreeFingerSwitcherCore) scripts/ build-app.sh, make-dev-cert.sh, allow-codesign-key.sh, install-launch-agent.sh openspec/ specs (canonical) + changes/archive (history) ``` @@ -160,7 +162,7 @@ The Core/App split exists so the test target can `@testable import ThreeFingerSw ```bash swift build # build the Core library + spikes (fast agent loop) -swift test # 805 unit tests (gesture machine + odometer navigation/edge auto-repeat, launcher latching, Dock-preview hover/anchor/layout, models, grouping, layout, settings, native-gesture config + relocation plan/markers, touch, first-run wizard machine) +swift test # 783 unit tests (gesture machine + odometer navigation/edge auto-repeat, launcher latching, Dock-preview hover/anchor/layout, models, grouping, layout, settings, native-gesture config + relocation plan/markers, touch, first-run wizard machine) swift run TouchSpike # print live multitouch frames (touch the trackpad) swift run LauncherSpike # throwaway launcher spike harness (haptics / window move) ./scripts/build-app.sh # assemble + sign ThreeFingerSwitcher.app (via xcodebuild) @@ -187,6 +189,7 @@ INSTALL=1 ./scripts/build-app.sh # also install in place to /Appli - **Off-Space Chromium windows (Chrome, Chrome Remote Desktop) have no remote-token AX element.** A fresh `_AXUIElementCreateWithRemoteToken` brute force returns nothing for them, so they used to vanish from the list *and* couldn't be raised. Two pieces fix this, both in `WindowService`: (1) **listing** falls back to a CGS-metadata heuristic (`alpha > 0 && min(width,height) ≥ 130`) when no element resolves — empirically separates real windows (incl. Stage-Manager strip thumbnails, min-dim ≥ 150) from sliver/toolbar/zero-alpha junk; (2) **raising** uses a persistent **`elementCache`** keyed by `CGWindowID`, seeded when an app activates (its windows are then on the current Space and resolvable via `kAXWindowsAttribute`) and during snapshots — a cached element stays valid across Spaces, so `kAXRaiseAction` on it *navigates* to the window. Limit: a Chromium window off-Space since before launch and never focused has no cached element and can't be navigated to (the AltTab/HyperSwitch limit). **Do NOT** try to switch Spaces with `CGSManagedDisplaySetCurrentSpace` — the WindowServer gates Space switching to Dock.app's privileged connection; the symbol resolves but no-ops for an unentitled, SIP-on process (it's why yabai needs SIP off). We tried it; it's removed. - **Off-Space focus is stolen by `WindowManager` ~300 ms after the Space switch** — a *different* mechanism from the current-Space singleton oscillation above. The +180 ms watchdog checks too early to see it, so `raise()` arms a bounded **polling hold-guard** (`offSpaceHoldTick`, off-Space + Stage-Manager only): poll every ~60 ms and re-front the target the instant the steal is detected (≈ one-frame flash), bounded to a few re-fronts so a daemon that fights back can't make it thrash. Don't turn it back into a fixed-delay re-assert (slower, visible flash) or drop the bound. - **The overlay panel is non-activating, `ignoresMouseEvents`, must never become key/main, and is always ordered out on gesture end.** On the common path it sits at `.popUpMenu` with **no `.stationary`** (a higher band / `.stationary` are Exposé-exempt and perturb focus/Space arbitration). The **one scoped exception:** while **Mission Control is open**, `OverlayController.show(aboveMissionControl:)` raises it to `.screenSaver` + `.stationary` so the switcher floats *above* MC instead of behind it — applied per-show only in that case, and a commit then dismisses MC (synthesized Escape, never a re-toggle) before re-raising the window from a clean state. Don't widen that elevated config to the normal path. +- **Several things exist only to keep the app fast over days — don't "clean them up".** `TouchEngine`'s ObjC-runtime `removeObserver` on the vendored `OpenMTManager` (its own wake handler double-starts trackpad devices — one orphan per sleep/wake, each re-processing every touch frame); the `AXUIElementSetMessagingTimeout(systemWide, 0.5)` + App Nap opt-out in `AppDelegate` (keep `.userInitiatedAllowingIdleSystemSleep` — plain `.userInitiated` would stop the Mac from sleeping); the event taps' 2 s `tapIsEnabled` watchdogs; `ThumbnailService`'s single-slot sweep + LRU + live-id prune; the launcher's one reused `NSHostingView` across disposable panels; and the wizard being *destroyed* on close rather than suspended. Every one of these undid a measured regression — CLAUDE.md's guardrails section has the why for each. - **The Dock window previews are mouse-driven and the peek *fronts the real window*.** This is the app's **only cursor-interactive surface** — `DockPreviewOverlay` sets `ignoresMouseEvents = false` (every other overlay is pass-through, non-key). Hovering a tab genuinely **activates** that window — SkyLight `setFront` handshake + `kAXMain`/the app's `kAXFocusedWindow` (the same reliable front the commit uses) — and **restores** the previously-front window on leave, because **macOS won't render fresh pixels for an off-screen window**, so there's no way to preview live content without bringing it forward (an `SCStream` version was tried and reverted). The tab thumbnail itself is a **single static capture taken ~0.5s after the window settles** (skipping the front-transition frame), not a continuous stream. Both the SkyLight handshake and the focus singletons are **skipped under Stage Manager** (they oscillate WindowManager — same landmine as the switcher raise). Full detail in CLAUDE.md's "Dock window previews" section. ### B4. How a gesture flows (mental model) @@ -212,5 +215,5 @@ All in `AppSettings` (persisted, live-applied, editable on the Hub's feature pag - **GPL-3.0** — see `LICENSE` and `NOTICE`. The window-raising/Space technique (the private `_AXUIElementGetWindow`, the remote-token brute force, the SkyLight front/key byte protocol, the CGS Space enumeration) is adapted from **[AltTab](https://github.com/lwouis/alt-tab-macos)** (GPL-3), which is why this project is GPL-3. - Raw multitouch via **[OpenMultitouchSupport](https://github.com/Kyome22/OpenMultitouchSupport)** (Kyome, MIT), wrapping the private `MultitouchSupport.framework`. -If you (Claude) end up extending this, keep the spec in `openspec/specs/` honest, keep the 805 tests green (`swift test`), and respect the landmines in **B3** — they each cost a real debugging session to learn. +If you (Claude) end up extending this, keep the spec in `openspec/specs/` honest, keep the 783 tests green (`swift test`), and respect the landmines in **B3** — they each cost a real debugging session to learn. diff --git a/Sources/ThreeFingerSwitcher/App/AppCoordinator.swift b/Sources/ThreeFingerSwitcher/App/AppCoordinator.swift index af814ea..470701e 100644 --- a/Sources/ThreeFingerSwitcher/App/AppCoordinator.swift +++ b/Sources/ThreeFingerSwitcher/App/AppCoordinator.swift @@ -135,11 +135,23 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate /// an app switch (which emits no `didActivateApplication`). Gated on BOTH the master keyboard-language /// toggle AND the per-site sub-toggle; fully inert otherwise. private lazy var keyboardLanguageBrowserMonitor = BrowserContextMonitor( - isSupportedBrowserFront: { - BrowserRegistry.isSupported(NSWorkspace.shared.frontmostApplication?.bundleIdentifier ?? "") + isSupportedBrowserFront: { [weak self] in + guard let front = NSWorkspace.shared.frontmostApplication else { return false } + // `NSRunningApplication.bundleIdentifier` is a LaunchServices round-trip, and this runs + // on every 0.5 s poll tick for as long as a browser is front — memoize it per pid. + let pid = front.processIdentifier + if let cached = self?.browserFrontByPid, cached.pid == pid { return cached.supported } + let supported = BrowserRegistry.isSupported(front.bundleIdentifier ?? "") + self?.browserFrontByPid = (pid, supported) + return supported }, onTick: { [weak self] in self?.keyboardLanguageService.reevaluate() } ) + private var browserFrontByPid: (pid: pid_t, supported: Bool)? + /// The pending delayed re-seed sweeps for the Hub's / wizard's switcher demos (cancelled and + /// replaced on each re-seed so repeated seeding can't stack a backlog of delayed sweeps). + private var hubSeedRetries: [DispatchWorkItem] = [] + private var wizardSeedRetries: [DispatchWorkItem] = [] private lazy var keyboardLanguageService = KeyboardLanguageService( store: keyboardLanguageStore, controller: CarbonInputSourceController(), @@ -167,6 +179,16 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate // The unified configuration Hub: one reusable window, its navigation state, and the wiring context. private var hubWindow: NSWindow? + /// The Hub window's visibility observers — held so they can be removed if the window is ever + /// released (discarded tokens would stack three permanent observers per re-creation). + private var hubWindowObservers: [NSObjectProtocol] = [] + /// The Hub's `CGWindowID`, or nil when it has no window device. `NSWindow.windowNumber` is ≤ 0 + /// for a closed (retained) window and `CGWindowID(Int)` traps on a negative — route every + /// conversion through here. + private var hubWindowID: CGWindowID? { + guard let number = hubWindow?.windowNumber, number > 0 else { return nil } + return CGWindowID(number) + } private let hubNav = HubNavigation() private lazy var hubContext: HubContext = makeHubContext() /// The Space the Hub was last presented on (captured in `showHub`). Used to place the synthetic @@ -190,8 +212,11 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate var onMenuBarPulse: (() -> Void)? /// Live finger count from the most recent touch frame; drives the scroll tap's consume rule - /// (the switcher owns all three-finger scroll so it never leaks to the background). + /// (the switcher owns all three-finger scroll so it never leaks to the background). Reset to 0 + /// wherever the touch engine stops, and treated as 0 by the tap once `lastTouchFrameTime` is + /// stale (see the consume predicate). private var currentFingerCount = 0 + private var lastTouchFrameTime: CFTimeInterval = 0 /// Pure decision for the scroll tap: consume (swallow) scroll while three or more fingers are /// down OR while the launcher overlay is open OR while the switcher overlay is open. The @@ -217,6 +242,7 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate touchEngine.onFrame = { [weak self] frame in guard let self else { return } self.currentFingerCount = frame.fingerCount + self.lastTouchFrameTime = frame.time // The wizard's live-hand act still mirrors every frame (read-only). The Hub gesture previews are // pure autoplay (they don't read the touch feed), so nothing else taps the stream here. self.onWizardTouchFrame?(frame) @@ -227,7 +253,12 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate } scrollTap.consumePredicate = { [weak self] in guard let self else { return false } - return Self.shouldConsumeScroll(fingerCount: self.currentFingerCount, + // Staleness guard: if the touch stream died mid-gesture (sleep with fingers down, the + // multitouch stream going silent), `currentFingerCount` would otherwise stay ≥ 3 and + // this tap would swallow EVERY scroll, in every app, until the app was quit. A finger + // count older than half a second is treated as no fingers. + let stale = CACurrentMediaTime() - self.lastTouchFrameTime > 0.5 + return Self.shouldConsumeScroll(fingerCount: stale ? 0 : self.currentFingerCount, launcherOpen: self.launcherOverlay.isVisible, switcherOpen: self.overlay.isVisible) } @@ -236,12 +267,16 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate // The wizard's demo strip listens along while it lives (the post-Screen-Recording reveal). self?.wizardModel?.demo.setThumbnail(image, for: id) } + // The menu-bar "Keep Awake — Active / Stop" line tracks the automation live (it otherwise + // only corrected itself on the next menu open). + keepAwakeController.onActiveChanged = { [weak self] in self?.onStateChange?() } launcherOverlay.onFire = { [weak self] item, band in self?.launchService.fire(item, inBand: band) } launcherOverlay.onTogglePin = { [weak self] item in guard case let .clipboardEntry(entry) = item.kind else { return } self?.clipboardStore.togglePin(id: entry.id) } observeSleepWake() + observeAccessibilityGrant() observeEnabledToggle() observeSpacesRearrangeToggle() observeVerticalGestureToggle() @@ -354,6 +389,7 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate guard isEnabled else { return } recognizer.reset() touchEngine.stop() + currentFingerCount = 0 scrollTap.stop() keyboardSwitcherTap.stop() keyboardSwitcher.forceCancel() // tear down any open ⌘-Tab session before hiding the overlay @@ -416,6 +452,23 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate .store(in: &cancellables) } + /// Re-arm the event taps when Accessibility is (re)granted mid-session. `CGEvent.tapCreate` fails + /// without the grant and its result was discarded — so a user who granted the permission outside + /// the wizard flow, or revoked and re-granted it, silently lost scroll consumption and ⌘-Tab until + /// a relaunch or a settings toggle happened to call the gate refresh. `removeDuplicates` keeps the + /// 1 Hz permission poll from re-running the refresh per tick. + private func observeAccessibilityGrant() { + permissions.$accessibility + .removeDuplicates() + .sink { [weak self] status in + MainActor.assumeIsolated { + guard status == .granted, let self, self.isEnabled else { return } + self.refreshRowSwitchingGate() + } + } + .store(in: &cancellables) + } + // MARK: - Sleep / wake recovery /// Observer tokens for the workspace sleep/wake notifications (removed in deinit). @@ -442,6 +495,23 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate MainActor.assumeIsolated { self?.handleWillSleep() } } wakeObservers.append(sleepToken) + + // `missionControlOpen` is our own latch — set when WE synthesize Mission Control — and nothing + // told us when the user closed it some other way (clicking a window, picking a Space, F3). + // Stale-true meant every later switcher open floated at screen-saver level and every commit + // posted a stray Escape into the user's app. The two observable ways out of MC — a regular + // app being activated, or the active Space changing — clear it. (The Dock is `.prohibited`, + // so MC opening itself never trips the activation clear.) + let activation = center.addObserver(forName: NSWorkspace.didActivateApplicationNotification, object: nil, queue: .main) { [weak self] note in + guard let app = note.userInfo?[NSWorkspace.applicationUserInfoKey] as? NSRunningApplication, + app.activationPolicy == .regular, app.processIdentifier != getpid() else { return } + MainActor.assumeIsolated { self?.missionControlOpen = false } + } + wakeObservers.append(activation) + let spaceChange = center.addObserver(forName: NSWorkspace.activeSpaceDidChangeNotification, object: nil, queue: .main) { [weak self] _ in + MainActor.assumeIsolated { self?.missionControlOpen = false } + } + wakeObservers.append(spaceChange) } private func handleWillSleep() { @@ -462,6 +532,10 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate // model downloads that span a sleep). Stopping pre-sleep makes the post-wake `stop()` a no-op, // and `restartTouchEngineAfterWake()` attaches a fresh listener on `didWake`. touchEngine.stop() + // The stream is gone, so the last frame's finger count must not linger (the scroll tap + // would otherwise keep consuming), and any Mission Control we had open is closed by sleep. + currentFingerCount = 0 + missionControlOpen = false // Tear down the focus tracker's AX observer (its main-run-loop source) pre-sleep, alongside // the multitouch listener; `restartTouchEngineAfterWake()` re-attaches a fresh one on wake. focus.stop() @@ -474,13 +548,17 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate recognizer.reset() touchEngine.stop() touchEngine.start() + currentFingerCount = 0 // Re-attach the focus tracker's AX observer torn down in `handleWillSleep`. Idempotent. focus.stop() focus.start() - // If the trackpad couldn't be re-acquired, reflect that in the menu state. + // If the trackpad couldn't be re-acquired, reflect that in the menu state — and stand the + // event taps down with it: flipping `isEnabled` alone left both taps armed against a dead + // engine, with `disable()` then a no-op (its guard already false), so nothing ever repaired it. let available = touchEngine.isAvailable if isEnabled != available { isEnabled = available + refreshRowSwitchingGate() onStateChange?() } } @@ -564,7 +642,11 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate aboveMissionControl: missionControlOpen, // Opted in but the relocation hasn't survived its re-login yet: the row dots // dim with a pending glyph so the gated vertical axis explains itself. - rowSwitchingPending: settings.manageVerticalGesture && !isSpaceRowSwitchingEffective, + // Read the gate the recognizer already holds — NOT `isSpaceRowSwitchingEffective`, + // which shells out to `/usr/bin/defaults` (fork + waitpid, 30–100 ms) on the + // main thread. That spawn sat inline in EVERY switcher open for anyone with + // Space-row switching on (the `&&` short-circuits it away on default installs). + rowSwitchingPending: settings.manageVerticalGesture && !recognizer.rowSwitchingEnabled, windowScale: CGFloat(settings.switcherWindowScale), groups: validatedWindowGroups(against: windows)) switcherOwner = owner @@ -684,7 +766,7 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate // Never attempt a ScreenCaptureKit capture of our OWN Hub window (it's the synthetic icon-only // card) — exclude its id from both the cache seed and the live prefetch so no self-capture is // tried; the switcher already renders the app icon for it. - let hubID = hubWindow.map { CGWindowID($0.windowNumber) } + let hubID = hubWindowID let windows = overlay.model.windows.filter { $0.id != hubID } thumbnails.seed(into: overlay.model, ids: windows.map(\.id)) // instant from cache (no icon-only flash) thumbnails.prefetch(windows) // refresh only cleanly-visible windows @@ -698,7 +780,7 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate /// Off-screen Spaces can't be freshly captured anyway, so cache is their only preview source; the /// seed is idempotent (identical frames don't republish), so re-seeding the current row is free. private func seedAllRows() { - let hubID = hubWindow.map { CGWindowID($0.windowNumber) } + let hubID = hubWindowID let allIDs = overlay.model.rows.flatMap { $0 }.map(\.id).filter { $0 != hubID } thumbnails.seed(into: overlay.model, ids: allIDs) } @@ -713,6 +795,7 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate previewRefreshTimer = Timer.scheduledTimer(withTimeInterval: Self.previewRefreshInterval, repeats: true) { [weak self] _ in MainActor.assumeIsolated { self?.prefetchCurrentRow() } } + previewRefreshTimer?.tolerance = 0.2 // drives a capture sweep, not an animation — let it coalesce } /// Stop the periodic preview refresh. Idempotent — safe when already stopped (the timer is nil) — so it @@ -725,8 +808,21 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate thumbnails.cancelSweeps() } + /// The one pending commit action deferred past Mission Control's close animation. Single-slot: + /// a second commit inside that 0.3 s (MC already marked closed, so it raises immediately) must + /// not be overridden when the FIRST commit's deferred raise fires a beat later. + private var missionControlDismissDeferral: DispatchWorkItem? + + private func deferPastMissionControlDismiss(_ body: @escaping @MainActor () -> Void) { + missionControlDismissDeferral?.cancel() + let work = DispatchWorkItem { MainActor.assumeIsolated { body() } } + missionControlDismissDeferral = work + DispatchQueue.main.asyncAfter(deadline: .now() + 0.3, execute: work) + } + func gestureDidCommit() { switcherOwner = .none // the session is ending regardless of which branch below runs + missionControlDismissDeferral?.cancel() // a newer commit supersedes a pending deferred raise guard overlay.isVisible, let window = overlay.model.selectedWindow else { overlay.hide() stopPreviewRefresh() @@ -744,9 +840,7 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate if missionControlOpen { missionControlOpen = false MissionControl.dismiss() - DispatchQueue.main.asyncAfter(deadline: .now() + 0.3) { [weak self] in - self?.present(self?.hubWindow) - } + deferPastMissionControlDismiss { [weak self] in self?.present(self?.hubWindow) } } else { present(hubWindow) } @@ -766,9 +860,7 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate if missionControlOpen { missionControlOpen = false MissionControl.dismiss() - DispatchQueue.main.asyncAfter(deadline: .now() + 0.3) { [weak self] in - self?.raiseCommitted(window) - } + deferPastMissionControlDismiss { [weak self] in self?.raiseCommitted(window) } } else { raiseCommitted(window) } @@ -1148,24 +1240,31 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate /// the window the user last had front there. private func focusFrontWindowAfterSpaceSwitch() { let before = SpaceService.currentModel()?.currentSpaceIDs ?? [] - afterSpaceSettles(before: before, attempt: 0) + // A newer Space-switch action supersedes any poll still running for the previous one: + // without the token, rapid consecutive switches stacked N concurrent 16 Hz CGS polls, and a + // stale chain could focus the wrong Space's window after the user had already moved on. + spaceSettleGeneration &+= 1 + afterSpaceSettles(before: before, attempt: 0, generation: spaceSettleGeneration) } + private var spaceSettleGeneration = 0 /// Poll until the active Space flips, then wait for the WindowServer transition (and the /// Stage-Manager front-steal ~300ms post-switch) to finish before focusing — acting on the flip - /// instant gets steamrolled by the rest of the transition. - private func afterSpaceSettles(before: Set, attempt: Int) { + /// instant gets steamrolled by the rest of the transition. `generation` retires the chain the + /// moment a newer switch starts one. + private func afterSpaceSettles(before: Set, attempt: Int, generation: Int) { DispatchQueue.main.asyncAfter(deadline: .now() + 0.06) { [weak self] in - guard let self else { return } + guard let self, self.spaceSettleGeneration == generation else { return } let now = SpaceService.currentModel()?.currentSpaceIDs ?? [] if !now.isEmpty, now != before { DispatchQueue.main.asyncAfter(deadline: .now() + 0.45) { [weak self] in - self?.focusFrontWindowOnCurrentSpace() + guard let self, self.spaceSettleGeneration == generation else { return } + self.focusFrontWindowOnCurrentSpace() } return } // A ⌃→ with no neighbour Space never flips — bail after ~1.9s rather than poll forever. - if attempt < 30 { afterSpaceSettles(before: before, attempt: attempt + 1) } + if attempt < 30 { afterSpaceSettles(before: before, attempt: attempt + 1, generation: generation) } } } @@ -1263,7 +1362,7 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate nohup /bin/sh -c ' exec >/tmp/tfs-relaunch.log 2>&1 echo "relaunch: waiting for pid \(pid)" - while /bin/kill -0 \(pid) 2>/dev/null; do /bin/sleep 0.1; done + for _ in $(/usr/bin/seq 300); do /bin/kill -0 \(pid) 2>/dev/null || break; /bin/sleep 0.1; done /bin/sleep 0.4 echo "relaunch: opening" /usr/bin/open "\(bundlePath)" || { echo "relaunch: retry"; /bin/sleep 1; /usr/bin/open "\(bundlePath)"; } @@ -1918,14 +2017,19 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate let windows = model.windows self.thumbnails.seed(into: model, ids: windows.map(\.id)) self.thumbnails.prefetch(windows) - for delay in [0.7, 2.0] { - DispatchQueue.main.asyncAfter(deadline: .now() + delay) { [weak self] in - guard let self, let demo = self.wizardModel?.demo else { return } - let missing = windows.filter { demo.thumbnails[$0.id] == nil } - guard !missing.isEmpty else { return } - self.thumbnails.seed(into: demo, ids: missing.map(\.id)) - self.thumbnails.prefetch(missing) + self.wizardSeedRetries.forEach { $0.cancel() } + self.wizardSeedRetries = [0.7, 2.0].map { delay in + let work = DispatchWorkItem { [weak self] in + MainActor.assumeIsolated { + guard let self, let demo = self.wizardModel?.demo else { return } + let missing = windows.filter { demo.thumbnails[$0.id] == nil } + guard !missing.isEmpty else { return } + self.thumbnails.seed(into: demo, ids: missing.map(\.id)) + self.thumbnails.prefetch(missing) + } } + DispatchQueue.main.asyncAfter(deadline: .now() + delay, execute: work) + return work } } ctx.trackpadClaimed = { [weak self] in self?.trackpadConfig.isClaimed ?? false } @@ -2036,13 +2140,13 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate // flag stops every clock. Registered once (the window is reused), scoped to THIS window. let nc = NotificationCenter.default for name in [NSWindow.willCloseNotification, NSWindow.didMiniaturizeNotification] { - nc.addObserver(forName: name, object: window, queue: .main) { [weak self] _ in + hubWindowObservers.append(nc.addObserver(forName: name, object: window, queue: .main) { [weak self] _ in MainActor.assumeIsolated { self?.pauseHubPreviews() } - } + }) } - nc.addObserver(forName: NSWindow.didDeminiaturizeNotification, object: window, queue: .main) { [weak self] _ in + hubWindowObservers.append(nc.addObserver(forName: NSWindow.didDeminiaturizeNotification, object: window, queue: .main) { [weak self] _ in MainActor.assumeIsolated { self?.resumeHubPreviews() } - } + }) hubWindow = window } present(hubWindow) @@ -2102,14 +2206,21 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate let windows = model.windows self.thumbnails.seed(into: model, ids: windows.map(\.id)) self.thumbnails.prefetch(windows) - for delay in [0.7, 2.0] { - DispatchQueue.main.asyncAfter(deadline: .now() + delay) { [weak self, weak model] in - guard let self, let model else { return } - let missing = windows.filter { model.thumbnails[$0.id] == nil } - guard !missing.isEmpty else { return } - self.thumbnails.seed(into: model, ids: missing.map(\.id)) - self.thumbnails.prefetch(missing) + // Every Hub page re-navigation re-seeds; cancel the previous retries so rapid page + // switching can't stack a growing backlog of delayed sweeps. + self.hubSeedRetries.forEach { $0.cancel() } + self.hubSeedRetries = [0.7, 2.0].map { delay in + let work = DispatchWorkItem { [weak self, weak model] in + MainActor.assumeIsolated { + guard let self, let model else { return } + let missing = windows.filter { model.thumbnails[$0.id] == nil } + guard !missing.isEmpty else { return } + self.thumbnails.seed(into: model, ids: missing.map(\.id)) + self.thumbnails.prefetch(missing) + } } + DispatchQueue.main.asyncAfter(deadline: .now() + delay, execute: work) + return work } } ctx.launcherBands = { [weak self] clipboardOn in @@ -2191,5 +2302,14 @@ final class AppCoordinator: GestureRecognizerDelegate, KeyboardSwitcherDelegate keyboardSwitcher.forceCancel() // never leave a ⌘-Tab session open behind the hidden overlay overlay.hide() stopPreviewRefresh() + switcherOwner = .none + missionControlOpen = false + } + + /// Land any coalesced persistence before the process exits (favorites edits are debounced; the + /// clipboard index is written on a serial queue). Bounded — quit is never held behind a slow write. + func flushStores() { + favoritesStore.flushPendingSave() + clipboardStore.flush(timeout: 1.0) } } diff --git a/Sources/ThreeFingerSwitcher/App/AppDelegate.swift b/Sources/ThreeFingerSwitcher/App/AppDelegate.swift index f67d742..3d1a167 100644 --- a/Sources/ThreeFingerSwitcher/App/AppDelegate.swift +++ b/Sources/ThreeFingerSwitcher/App/AppDelegate.swift @@ -103,6 +103,7 @@ final class AppDelegate: NSObject, NSApplicationDelegate { // Force-stop Keep Awake FIRST (synchronous) so a dimmed-to-black screen is restored and the // sleep/lock assertion released before the process exits. Idempotent — a no-op when inactive. coordinator?.stopKeepAwake() + coordinator?.flushStores() coordinator?.restoreSpacesRearrangeOnQuit() // NOTE: the vertical-gesture relocation is deliberately NOT restored on quit. It needs a // re-login to take effect, and logout quits the app — restoring here would undo the change diff --git a/Sources/ThreeFingerSwitcher/App/InputActivityMonitor.swift b/Sources/ThreeFingerSwitcher/App/InputActivityMonitor.swift index 1cf16ee..5cba3e8 100644 --- a/Sources/ThreeFingerSwitcher/App/InputActivityMonitor.swift +++ b/Sources/ThreeFingerSwitcher/App/InputActivityMonitor.swift @@ -49,4 +49,11 @@ final class InputActivityMonitor { globalMonitor = nil localMonitor = nil } + + deinit { + // Backstop (the GlobalCursorMonitor precedent): `removeMonitor` is safe off-main, and a + // skipped `stop()` must never leave two process-wide input monitors installed forever. + if let globalMonitor { NSEvent.removeMonitor(globalMonitor) } + if let localMonitor { NSEvent.removeMonitor(localMonitor) } + } } diff --git a/Sources/ThreeFingerSwitcher/App/KeepAwakeController.swift b/Sources/ThreeFingerSwitcher/App/KeepAwakeController.swift index 8ccb0eb..b27e487 100644 --- a/Sources/ThreeFingerSwitcher/App/KeepAwakeController.swift +++ b/Sources/ThreeFingerSwitcher/App/KeepAwakeController.swift @@ -103,13 +103,17 @@ final class KeepAwakeController { case armed // active + armed: the next contact stops us } + // `nonisolated`: immutable constants referenced from default-argument and default-initializer + // expressions, which Swift evaluates in the CALLER's isolation — hence the "main actor-isolated + // static property … from a nonisolated context" warnings (errors in Swift 6 mode). + /// The default dim level (minimum). `DisplayServices` clamps to the panel's real floor. - static let dimLevel: Float = 0 + nonisolated static let dimLevel: Float = 0 /// Heartbeat cadence: re-pin brightness + re-declare user activity. - static let heartbeatInterval: TimeInterval = 300 + nonisolated static let heartbeatInterval: TimeInterval = 300 /// Convert an item's optional 0…100 dim percent (nil = minimum) to a clamped 0…1 brightness fraction. - static func fraction(fromPercent percent: Double?) -> Float { + nonisolated static func fraction(fromPercent percent: Double?) -> Float { Float(max(0, min(100, percent ?? 0)) / 100) } @@ -173,6 +177,7 @@ final class KeepAwakeController { heartbeat = Timer.scheduledTimer(withTimeInterval: heartbeatInterval, repeats: true) { [weak self] _ in MainActor.assumeIsolated { self?.heartbeatTick() } } + heartbeat?.tolerance = 30 // a 5-minute heartbeat needn't force a precise wake on a dark screen onActiveChanged?() } diff --git a/Sources/ThreeFingerSwitcher/Clipboard/ClipboardMonitor.swift b/Sources/ThreeFingerSwitcher/Clipboard/ClipboardMonitor.swift index 1793e3e..850e0a7 100644 --- a/Sources/ThreeFingerSwitcher/Clipboard/ClipboardMonitor.swift +++ b/Sources/ThreeFingerSwitcher/Clipboard/ClipboardMonitor.swift @@ -59,6 +59,7 @@ final class ClipboardMonitor { let t = Timer.scheduledTimer(withTimeInterval: max(0.1, pollInterval), repeats: true) { [weak self] _ in MainActor.assumeIsolated { self?.poll() } } + t.tolerance = max(0.1, pollInterval) / 4 // a changeCount poll isn't deadline-critical timer = t } @@ -106,8 +107,8 @@ final class ClipboardMonitor { ?? item.data(forType: .init(ClipboardUTI.tiff)) else { return nil } let uti = item.data(forType: .init(ClipboardUTI.png)) != nil ? ClipboardUTI.png : ClipboardUTI.tiff reps[uti] = .inline(data) - let rep = NSBitmapImageRep(data: data) - key = ClipboardKey.fromImage(width: rep?.pixelsWide ?? 0, height: rep?.pixelsHigh ?? 0) + let dims = Self.imagePixelSize(data) + key = ClipboardKey.fromImage(width: dims.width, height: dims.height) fingerprint = "image:\(Self.hash(data))" case .color: @@ -151,10 +152,35 @@ final class ClipboardMonitor { sourceApp: sourceID, representations: reps, fingerprint: fingerprint) } - /// Cheap content hash (FNV-1a 64-bit) for image/color/rtf fingerprints. - private static func hash(_ data: Data) -> String { + /// Bounded content fingerprint (FNV-1a 64-bit over the byte count + a head and tail sample). + /// + /// This runs on the MAIN thread from the 0.5 s pasteboard poll, once per copy. The previous + /// whole-payload `for byte in data` walk cost 2–10 ns/byte through `Data`'s generic iterator — + /// a 10 MB screenshot froze the app (gestures included) for up to ~100 ms on every ⌘⇧4. Sampling + /// makes it O(1) in payload size; the length + both ends distinguish real-world clipboard payloads + /// (a collision merely de-dups two copies into one entry — a fingerprint, not a checksum). + static func hash(_ data: Data) -> String { var h: UInt64 = 0xcbf29ce484222325 - for byte in data { h ^= UInt64(byte); h = h &* 0x100000001b3 } + @inline(__always) func mix(_ byte: UInt8) { h ^= UInt64(byte); h = h &* 0x100000001b3 } + withUnsafeBytes(of: UInt64(data.count).littleEndian) { for b in $0 { mix(b) } } + let sample = min(data.count, hashSampleBytes) + data.withUnsafeBytes { (buf: UnsafeRawBufferPointer) in + for b in buf.prefix(sample) { mix(b) } + if data.count > sample { for b in buf.suffix(sample) { mix(b) } } + } return String(h, radix: 16) } + private static let hashSampleBytes = 64 * 1024 + + /// Pixel dimensions from the image HEADER only (ImageIO properties, no decode). The previous + /// `NSBitmapImageRep(data:)` fully decompressed the bitmap just to label the entry "Image W×H". + private static func imagePixelSize(_ data: Data) -> (width: Int, height: Int) { + let options = [kCGImageSourceShouldCache: false] as CFDictionary + guard let source = CGImageSourceCreateWithData(data as CFData, options), + let props = CGImageSourceCopyPropertiesAtIndex(source, 0, options) as? [CFString: Any] else { + return (0, 0) + } + return ((props[kCGImagePropertyPixelWidth] as? Int) ?? 0, + (props[kCGImagePropertyPixelHeight] as? Int) ?? 0) + } } diff --git a/Sources/ThreeFingerSwitcher/Clipboard/ClipboardStore.swift b/Sources/ThreeFingerSwitcher/Clipboard/ClipboardStore.swift index 7ffd319..d64141b 100644 --- a/Sources/ThreeFingerSwitcher/Clipboard/ClipboardStore.swift +++ b/Sources/ThreeFingerSwitcher/Clipboard/ClipboardStore.swift @@ -19,13 +19,13 @@ final class ClipboardStore { static let `default` = Retention(maxCount: 200, maxBytes: 256 * 1024 * 1024, maxAge: 0) } - static let currentSchemaVersion = 1 + nonisolated static let currentSchemaVersion = 1 /// Inline payloads larger than this are externalized to a blob file on save. - private static let blobThreshold = 16 * 1024 + nonisolated private static let blobThreshold = 16 * 1024 /// Upper bound on the bytes a **band** item carries for its value preview. Textual payloads are /// truncated to this at band-build time so the band never holds (or renders) a large payload; the /// full content is materialized on demand for paste. Small payloads (≤ this) pass through whole. - static let previewByteCap = 16 * 1024 + nonisolated static let previewByteCap = 16 * 1024 private let directory: URL private var blobsDir: URL { directory.appendingPathComponent("blobs", isDirectory: true) } @@ -75,9 +75,21 @@ final class ClipboardStore { /// selected preview / paste (`materializedEntry(id:)`). Use this for the launcher band; use /// `recentWindow` where full bytes are needed. func bandWindow(limit: Int) -> [ClipboardEntry] { - Self.recentWindow(entries, limit: limit).map(boundedForBand) + Self.recentWindow(entries, limit: limit).map { entry in + if let memo = boundedCache[entry.id] { return memo } + let bounded = boundedForBand(entry) + boundedCache[entry.id] = bounded + return bounded + } } + /// `boundedForBand` results memoized per entry. Entries loaded from disk hold `.blob` references, + /// so building the band after a relaunch did one `open`+`read` per large entry — synchronously, on + /// the main thread, at launcher-gesture activation. An entry's payload never changes after insert, + /// so the memo is invalidated only where its OTHER fields can change (recency refresh on a + /// duplicate copy, pin toggle) and pruned to live ids on insert. + private var boundedCache: [UUID: ClipboardEntry] = [:] + /// The single, fully-materialized entry for `id` (all blob payloads resolved to inline), or nil if the /// id is unknown (e.g. evicted/cleared). On-demand full fetch for the paste path and the image preview, /// which the light `bandWindow` entries deliberately don't carry. @@ -101,6 +113,12 @@ final class ClipboardStore { func insert(_ entry: ClipboardEntry) { entries = Self.dedup(inserting: entry, into: entries) entries = Self.evict(entries, retention: retention, now: entry.capturedAt) + let live = Set(entries.map(\.id)) + boundedCache = boundedCache.filter { live.contains($0.key) } + // A duplicate copy refreshed an existing entry's recency under its old id — drop that memo. + if let refreshed = entries.first(where: { $0.fingerprint == entry.fingerprint }) { + boundedCache[refreshed.id] = nil + } save() } @@ -111,6 +129,7 @@ final class ClipboardStore { func togglePin(id: UUID) -> Bool? { guard let i = entries.firstIndex(where: { $0.id == id }) else { return nil } entries[i].pinned.toggle() + boundedCache[id] = nil save() return entries[i].pinned } @@ -118,6 +137,7 @@ final class ClipboardStore { /// Clear history. By default keeps pinned entries; pass `includingPinned: true` to wipe everything. func clear(includingPinned: Bool = false) { entries = includingPinned ? [] : entries.filter(\.pinned) + boundedCache.removeAll() save() } @@ -207,18 +227,53 @@ final class ClipboardStore { /// termination). The in-memory `entries` are always inline, so a concurrent bounded band read never /// races a blob write (only new, content-hashed blobs are written, and only once). private func save() { - let snapshot = entries + // Coalesce: while a drain is queued but hasn't run, a newer save only replaces the snapshot + // it will write. A burst of copies behind one slow multi-MB blob write previously parked N + // full copies of the store on the queue — transient memory that scaled with copy rate × store + // size. `flush()`'s `ioQueue.sync {}` still waits for the queued drain, so durability holds. + guard pendingSave.put(entries) else { return } let blobs = blobsDir let index = indexURL + let box = pendingSave ioQueue.async { + guard let snapshot = box.take() else { return } Self.persist(snapshot, schemaVersion: Self.currentSchemaVersion, blobsDir: blobs, indexURL: index) } } + /// Latest-wins handoff between the actor (`put`, on every save) and `ioQueue` (`take`, once per + /// queued drain). Lock-protected: the two sides run on different threads by design. + private final class PendingSave: @unchecked Sendable { + private let lock = NSLock() + private var latest: [ClipboardEntry]? + /// Stores the snapshot; returns true when no drain is queued yet (the caller must enqueue one). + func put(_ snapshot: [ClipboardEntry]) -> Bool { + lock.lock(); defer { lock.unlock() } + let needsDrain = latest == nil + latest = snapshot + return needsDrain + } + func take() -> [ClipboardEntry]? { + lock.lock(); defer { lock.unlock() } + let snapshot = latest + latest = nil + return snapshot + } + } + private let pendingSave = PendingSave() + /// Block until all queued persistence has been written. For deterministic durability where async would /// race a subsequent read (a test that reloads the store, or an app-termination flush). func flush() { ioQueue.sync {} } + /// Bounded flush for app termination: wait for the queued drain, but never block quit longer than + /// `timeout` behind a multi-megabyte blob write (`flush()`'s unbounded `sync` is for tests). + func flush(timeout: TimeInterval) { + let done = DispatchSemaphore(value: 0) + ioQueue.async { done.signal() } + _ = done.wait(timeout: .now() + timeout) + } + /// The disk half of `save`, run on `ioQueue`. `nonisolated static` so it touches no actor state — every /// input (the entries snapshot, the directory URLs) is passed in, keeping it safe to run off-main. nonisolated private static func persist(_ entries: [ClipboardEntry], schemaVersion: Int, @@ -231,7 +286,25 @@ final class ClipboardStore { try? data.write(to: indexURL, options: .atomic) // Prune against the names the just-written index references. pruneOrphanBlobs(keeping: referencedBlobs(externalized), blobsDir: blobsDir) + // Keep the name memo bounded to entries that still exist. + let live = Set(entries.map(\.id.uuidString)) + blobNames.names = blobNames.names.filter { live.contains($0.key.entryID) } + } + + /// Blob names memoized per (entry id, UTI). The content hash behind a name walks the FULL + /// payload byte by byte, and `persist` re-externalizes EVERY entry on EVERY copy (the in-memory + /// entries stay inline), so without the memo a history holding a few large images re-hashed tens + /// of megabytes per copy — CPU that grew with the store for the whole session. An entry's payload + /// never changes after insert (dedup re-inserts under a NEW id), so the memo can't go stale. + /// Touched only on the serial `ioQueue`, hence the unchecked conformance. + private final class BlobNameCache: @unchecked Sendable { + var names: [BlobKey: String] = [:] + } + private struct BlobKey: Hashable { + let entryID: String + let uti: String } + nonisolated private static let blobNames = BlobNameCache() nonisolated private static func referencedBlobs(_ entries: [ClipboardEntry]) -> Set { Set(entries.flatMap { entry in @@ -245,20 +318,29 @@ final class ClipboardStore { /// Small payloads stay inline. Deterministic blob names (by content) make re-saves idempotent. nonisolated private static func externalizedForStorage(_ entry: ClipboardEntry, blobsDir: URL) -> ClipboardEntry { var e = entry - e.representations = entry.representations.mapValues { payload in + var reps: [String: ClipboardPayload] = [:] + for (uti, payload) in entry.representations { switch payload { case .blob: - return payload // already external + reps[uti] = payload // already external case .inline(let data): - guard data.count > blobThreshold else { return payload } - let name = "\(entry.id.uuidString)-\(stableName(for: data)).bin" + guard data.count > blobThreshold else { reps[uti] = payload; continue } + let key = BlobKey(entryID: entry.id.uuidString, uti: uti) + let name: String + if let memo = blobNames.names[key] { + name = memo + } else { + name = "\(entry.id.uuidString)-\(stableName(for: data)).bin" + blobNames.names[key] = name + } let url = blobsDir.appendingPathComponent(name) if !FileManager.default.fileExists(atPath: url.path) { try? data.write(to: url, options: .atomic) } - return .blob(name) + reps[uti] = .blob(name) } } + e.representations = reps return e } @@ -368,11 +450,15 @@ final class ClipboardStore { } nonisolated private static func stableName(for data: Data) -> String { - // Cheap, dependency-free content hash (FNV-1a 64-bit) for a deterministic blob filename. + // Cheap, dependency-free content hash (FNV-1a 64-bit) for a deterministic blob filename. Over + // the raw buffer, not `Data`'s generic iterator (which costs several ns per byte) — this is + // the full payload, since a blob NAME must be unique per content. var hash: UInt64 = 0xcbf29ce484222325 - for byte in data { - hash ^= UInt64(byte) - hash = hash &* 0x100000001b3 + data.withUnsafeBytes { (buf: UnsafeRawBufferPointer) in + for byte in buf { + hash ^= UInt64(byte) + hash = hash &* 0x100000001b3 + } } return String(hash, radix: 16) } diff --git a/Sources/ThreeFingerSwitcher/Dock/AXDockReader.swift b/Sources/ThreeFingerSwitcher/Dock/AXDockReader.swift index 7188dd7..b48e726 100644 --- a/Sources/ThreeFingerSwitcher/Dock/AXDockReader.swift +++ b/Sources/ThreeFingerSwitcher/Dock/AXDockReader.swift @@ -78,10 +78,10 @@ final class AXDockReader: DockReader { private func tileFrame(_ item: AXUIElement) -> CGRect? { var origin = CGPoint.zero var size = CGSize.zero - guard let posValue = axCopy(item, kAXPositionAttribute as String), - let sizeValue = axCopy(item, kAXSizeAttribute as String) else { return nil } - AXValueGetValue(posValue as! AXValue, .cgPoint, &origin) - AXValueGetValue(sizeValue as! AXValue, .cgSize, &size) + guard let posValue = axValue(item, kAXPositionAttribute as String), + let sizeValue = axValue(item, kAXSizeAttribute as String) else { return nil } + AXValueGetValue(posValue, .cgPoint, &origin) + AXValueGetValue(sizeValue, .cgSize, &size) guard size.width > 0, size.height > 0 else { return nil } return Self.cocoaRect(fromAXTopLeft: CGRect(origin: origin, size: size)) } diff --git a/Sources/ThreeFingerSwitcher/Dock/CursorMonitor.swift b/Sources/ThreeFingerSwitcher/Dock/CursorMonitor.swift index 226330c..5e458e7 100644 --- a/Sources/ThreeFingerSwitcher/Dock/CursorMonitor.swift +++ b/Sources/ThreeFingerSwitcher/Dock/CursorMonitor.swift @@ -8,6 +8,11 @@ import CoreGraphics /// `onMove` is called with the cursor in **Cocoa global coordinates** (bottom-left origin) — the same /// space as `DockTile.frame`. `start()` installs the monitor; `stop()` removes it. Installed ONLY while /// the feature is enabled (the controller calls `start`/`stop` off the opt-in toggle). +/// +/// `@MainActor`: every conformer is main-actor-bound (the callbacks are written and read on main). A +/// nonisolated protocol let a nonisolated holder of `any CursorMonitor` write the callbacks from any +/// thread while the monitor read them on main — the Swift 6 conformance warning was pointing at that. +@MainActor protocol CursorMonitor: AnyObject { var onMove: ((CGPoint) -> Void)? { get set } /// A right-click (anywhere) reported in Cocoa global coordinates. Observed passively — used to yield @@ -17,6 +22,9 @@ protocol CursorMonitor: AnyObject { /// consumed, so the native Dock click is unaffected) — used to commit the highlighted preview when the /// click lands on the shown app's Dock tile. var onLeftDown: ((CGPoint) -> Void)? { get set } + /// A left-mouse-up (anywhere), Cocoa global coordinates; passive. Window-groups snap uses the + /// down/up pair for drag-end detection. + var onLeftUp: ((CGPoint) -> Void)? { get set } func start() func stop() } @@ -26,6 +34,7 @@ final class ManualCursorMonitor: CursorMonitor { var onMove: ((CGPoint) -> Void)? var onRightClick: ((CGPoint) -> Void)? var onLeftDown: ((CGPoint) -> Void)? + var onLeftUp: ((CGPoint) -> Void)? private(set) var running = false func start() { running = true } func stop() { running = false } diff --git a/Sources/ThreeFingerSwitcher/Dock/DockPreviewController.swift b/Sources/ThreeFingerSwitcher/Dock/DockPreviewController.swift index 5136b7e..041a2d0 100644 --- a/Sources/ThreeFingerSwitcher/Dock/DockPreviewController.swift +++ b/Sources/ThreeFingerSwitcher/Dock/DockPreviewController.swift @@ -37,6 +37,16 @@ final class DockPreviewController { /// auto-hidden Dock reads empty, and re-walking it per mouse move is the same waste. private var cachedDockRead: (at: TimeInterval, snap: DockSnapshot?)? private static let dockReadTTL: TimeInterval = 0.08 + + /// The Dock snapshot, re-walked at most once per `dockReadTTL` (every cursor/click path goes + /// through here so none bypasses the throttle). + private func currentDockRead() -> DockSnapshot? { + let tick = now() + if let cached = cachedDockRead, tick - cached.at < Self.dockReadTTL { return cached.snap } + let snap = reader.read() + cachedDockRead = (at: tick, snap: snap) + return snap + } private var shownPID: pid_t? private var emptyPID: pid_t? /// The tile whose native action menu we just opened with a right-click. While the cursor lingers on @@ -111,14 +121,7 @@ final class DockPreviewController { // (60–125 Hz near an edge) — a large steady main-thread tax competing with the gesture // path. Tile frames only matter at ~UI rates, so a snapshot ≤80 ms old is reused; the // hover model still gets every cursor sample at full rate for grace/anchor timing. - let tick = now() - let snap: DockSnapshot? - if let cached = cachedDockRead, tick - cached.at < Self.dockReadTTL { - snap = cached.snap - } else { - snap = reader.read() - cachedDockRead = (at: tick, snap: snap) - } + let snap = currentDockRead() snapshot = snap let tiles = snap?.tiles ?? [] @@ -135,6 +138,11 @@ final class DockPreviewController { switch decision { case .idle: + // Reset the per-app suppressions here too: an app with no current-Space windows set + // `emptyPID` with the popup never shown, so `dismiss()` (the only other clearer) never + // ran, and that app's preview stayed dead until some OTHER preview opened and closed. + emptyPID = nil + menuSuppressedPID = nil if overlay.isVisible { dismiss() } case .dismiss: dismiss() @@ -157,7 +165,7 @@ final class DockPreviewController { /// opens unmodified. private func handleRightClick(_ point: CGPoint) { guard enabled else { return } - let tiles = reader.read()?.tiles ?? snapshot?.tiles ?? [] + let tiles = currentDockRead()?.tiles ?? snapshot?.tiles ?? [] guard hover.rightClick(at: point, tiles: tiles) == .dismiss, let pid = DockHoverModel.tile(at: point, in: tiles)?.pid else { return } if overlay.isVisible { dismiss(restore: true) } @@ -175,7 +183,7 @@ final class DockPreviewController { /// so the native Dock still receives it unmodified (and also acts — see design D3). private func handleLeftClick(_ point: CGPoint) { guard enabled, overlay.isVisible else { return } - let tiles = reader.read()?.tiles ?? snapshot?.tiles ?? [] + let tiles = currentDockRead()?.tiles ?? snapshot?.tiles ?? [] guard hover.leftClick(at: point, tiles: tiles) != nil, let id = overlay.model.highlightedID else { return } commit(id) @@ -252,6 +260,10 @@ final class DockPreviewController { // so the tab only refreshed on enter/leave. Swaps happen via the next card's `inside == true`; // the highlight is cleared in `dismiss`. guard inside else { return } + // The same flicker re-enters the SAME card too: without this guard each false/true cycle + // republished an unchanged highlight (full row rebuild), re-issued the AX peek raise, and + // restarted the 0.5 s capture timer — so the tab's thumbnail could never actually land. + guard overlay.model.highlightedID != id else { return } overlay.model.highlightedID = id peek(id) } diff --git a/Sources/ThreeFingerSwitcher/Hub/BandsCanvas.swift b/Sources/ThreeFingerSwitcher/Hub/BandsCanvas.swift index 7961103..4a8b88a 100644 --- a/Sources/ThreeFingerSwitcher/Hub/BandsCanvas.swift +++ b/Sources/ThreeFingerSwitcher/Hub/BandsCanvas.swift @@ -288,7 +288,7 @@ private struct AppBrowser: View { kind: .app(bundleURL: app.url, strategy: nil))) } label: { GridTile(title: app.name, subtitle: "App") { - Image(nsImage: NSWorkspace.shared.icon(forFile: app.url.path)).resizable() + Image(nsImage: IconCache.icon(forFile: app.url.path)).resizable() .frame(width: 44, height: 44) } } @@ -1611,11 +1611,11 @@ struct LaunchItemIconView: View { switch item.icon { case .appDefault: if case let .app(url, _) = item.kind { - Image(nsImage: NSWorkspace.shared.icon(forFile: url.path)).resizable() + Image(nsImage: IconCache.icon(forFile: url.path)).resizable() } else { symbol("app.dashed") } case .fileIcon: if case let .path(url) = item.kind { - Image(nsImage: NSWorkspace.shared.icon(forFile: url.path)).resizable() + Image(nsImage: IconCache.icon(forFile: url.path)).resizable() } else { symbol("doc") } case .sfSymbol(let n): Image(systemName: n).resizable().scaledToFit() diff --git a/Sources/ThreeFingerSwitcher/Hub/HubControls.swift b/Sources/ThreeFingerSwitcher/Hub/HubControls.swift index f8576bf..bf4e218 100644 --- a/Sources/ThreeFingerSwitcher/Hub/HubControls.swift +++ b/Sources/ThreeFingerSwitcher/Hub/HubControls.swift @@ -165,26 +165,36 @@ struct HubExcludedAppsEditor: View { } } Menu("Add app…") { - ForEach(runningApps(), id: \.bundleID) { app in + ForEach(apps, id: \.bundleID) { app in Button(app.name) { if !excluded.contains(app.bundleID) { excluded.append(app.bundleID) } } } } } + // Enumerate once per appearance, not per render: this editor lives on a page with four + // sliders bound to AppSettings, so `body` re-ran (and re-walked + ICU-sorted every running + // app, plus a LaunchServices name lookup per excluded row) on every slider tick. + .onAppear { refreshApps() } } - private func displayName(_ bundleID: String) -> String { - NSRunningApplication.runningApplications(withBundleIdentifier: bundleID).first?.localizedName ?? bundleID - } + @State private var apps: [(bundleID: String, name: String)] = [] + @State private var names: [String: String] = [:] - private func runningApps() -> [(bundleID: String, name: String)] { - NSWorkspace.shared.runningApplications + private func refreshApps() { + apps = NSWorkspace.shared.runningApplications .filter { $0.activationPolicy == .regular } .compactMap { app in guard let id = app.bundleIdentifier else { return nil } return (id, app.localizedName ?? id) } .sorted { $0.name.localizedCaseInsensitiveCompare($1.name) == .orderedAscending } + names = Dictionary(apps.map { ($0.bundleID, $0.name) }, uniquingKeysWith: { a, _ in a }) + } + + private func displayName(_ bundleID: String) -> String { + names[bundleID] + ?? NSRunningApplication.runningApplications(withBundleIdentifier: bundleID).first?.localizedName + ?? bundleID } } diff --git a/Sources/ThreeFingerSwitcher/Hub/HubFeaturePages.swift b/Sources/ThreeFingerSwitcher/Hub/HubFeaturePages.swift index 11a835e..61b0118 100644 --- a/Sources/ThreeFingerSwitcher/Hub/HubFeaturePages.swift +++ b/Sources/ThreeFingerSwitcher/Hub/HubFeaturePages.swift @@ -18,6 +18,8 @@ struct SwitcherPage: View { /// from `docs/postmortem-idle-cpu-spin.md`; the old free-running driver stays deleted). @StateObject private var demo = HubSwitcherDemo() @State private var seeded = false + /// Coalesces the "Window size" slider's live re-solve (see the `onChange` below). + @State private var scaleDebounce: DispatchWorkItem? /// The base autoplay gesture — the teaching story, its open swipe scaled to the activation threshold. @State private var gesture: GesturePose.DemoGesture /// The hover-demo override pushed into the preview by the direction pickers: hovering the windows-axis @@ -85,8 +87,16 @@ struct SwitcherPage: View { ) .onAppear { seedIfNeeded() } // Live window-size: dragging "Window size" grows/shrinks the preview cards in real time. + // Coalesced to ~16 updates/s: each update re-solves every Space's grid from scratch + // (binary searches over the packer) inside a fresh 0.25 s animation, and a 60 Hz drag + // otherwise stacked ~15 overlapping re-solves + retargeting animations per frame. .onChange(of: settings.switcherWindowScale) { _, scale in - demo.setMaxScale(SwitcherLayout.kMax * CGFloat(scale)) + scaleDebounce?.cancel() + let work = DispatchWorkItem { + MainActor.assumeIsolated { demo.setMaxScale(SwitcherLayout.kMax * CGFloat(scale)) } + } + scaleDebounce = work + DispatchQueue.main.asyncAfter(deadline: .now() + 0.06, execute: work) } // The demo's opening swipe tracks the real activation distance as you tune it. .onChange(of: settings.activationThreshold) { _, threshold in @@ -846,7 +856,9 @@ struct LauncherActionMap: View { let detail: String } - private let steps: [Step] = [ + // `static`: see SwitcherActionMap — stable identities, so re-renders update rows instead of + // rebuilding them. + private static let steps: [Step] = [ Step(symbol: "hand.raised.fill", title: "Slide four fingers", detail: "Swipe sideways to open your launcher"), Step(symbol: "hand.point.up.left.fill", title: "Lift two fingers", detail: "Keep two fingers resting to navigate"), Step(symbol: "arrow.up.arrow.down", title: "Up / down", detail: "Move between bands"), @@ -856,7 +868,7 @@ struct LauncherActionMap: View { var body: some View { VStack(alignment: .leading, spacing: 10) { - ForEach(Array(steps.enumerated()), id: \.element.id) { index, step in + ForEach(Array(Self.steps.enumerated()), id: \.element.id) { index, step in HStack(alignment: .center, spacing: 12) { ZStack { Circle().fill(Color.accentColor.opacity(0.16)).frame(width: 26, height: 26) diff --git a/Sources/ThreeFingerSwitcher/Hub/HubSwitcherDemo.swift b/Sources/ThreeFingerSwitcher/Hub/HubSwitcherDemo.swift index 2cc5483..1f07796 100644 --- a/Sources/ThreeFingerSwitcher/Hub/HubSwitcherDemo.swift +++ b/Sources/ThreeFingerSwitcher/Hub/HubSwitcherDemo.swift @@ -220,7 +220,9 @@ struct SwitcherActionMap: View { let detail: String } - private let steps: [Step] = [ + // `static`: a stored instance property was rebuilt with the view struct on every parent + // re-render, minting five fresh UUIDs each time — ForEach then tore down and rebuilt all rows. + private static let steps: [Step] = [ Step(symbol: "hand.raised.fill", title: "Slide three fingers", detail: "Swipe sideways to open the switcher"), Step(symbol: "hand.point.up.left.fill", title: "Lift one finger", detail: "Keep two fingers resting to navigate"), Step(symbol: "arrow.up.arrow.down", title: "Up / down", detail: "Move between Spaces (rows)"), @@ -230,7 +232,7 @@ struct SwitcherActionMap: View { var body: some View { VStack(alignment: .leading, spacing: 10) { - ForEach(Array(steps.enumerated()), id: \.element.id) { index, step in + ForEach(Array(Self.steps.enumerated()), id: \.element.id) { index, step in HStack(alignment: .center, spacing: 12) { ZStack { Circle().fill(Color.accentColor.opacity(0.16)).frame(width: 26, height: 26) diff --git a/Sources/ThreeFingerSwitcher/Hub/HubWindowInspector.swift b/Sources/ThreeFingerSwitcher/Hub/HubWindowInspector.swift index c45dc27..43d1373 100644 --- a/Sources/ThreeFingerSwitcher/Hub/HubWindowInspector.swift +++ b/Sources/ThreeFingerSwitcher/Hub/HubWindowInspector.swift @@ -12,9 +12,14 @@ struct HubWindowInspector: View { @State private var entries: [WindowInspectorEntry] = [] @State private var loaded = false + /// Grouped + sorted ONCE per snapshot. This view observes `AppSettings` and shares a page with + /// five sliders, so it re-renders on every slider tick — re-grouping and ICU-sorting every + /// window (twice: body + summary) per tick was pure waste. + @State private var groups: [(key: String, name: String, icon: NSImage?, rows: [WindowInspectorEntry])] = [] - private var groups: [(key: String, name: String, icon: NSImage?, rows: [WindowInspectorEntry])] { - Dictionary(grouping: entries, by: \.appKey) + private func setEntries(_ new: [WindowInspectorEntry]) { + entries = new + groups = Dictionary(grouping: new, by: \.appKey) .map { (key: $0.key, name: $0.value.first?.appName ?? $0.key, icon: $0.value.first?.appIcon, rows: $0.value) } .sorted { $0.name.localizedCaseInsensitiveCompare($1.name) == .orderedAscending } @@ -26,7 +31,7 @@ struct HubWindowInspector: View { Text(summary) .font(.caption).foregroundStyle(.secondary) Spacer() - Button { entries = inspect() } label: { + Button { setEntries(inspect()) } label: { Label("Refresh", systemImage: "arrow.clockwise") } } @@ -59,7 +64,7 @@ struct HubWindowInspector: View { .onAppear { guard !loaded else { return } loaded = true - entries = inspect() + setEntries(inspect()) } } @@ -138,7 +143,7 @@ struct HubWindowInspector: View { } else { settings.windowAppRules.removeValue(forKey: key) } - entries = inspect() + setEntries(inspect()) })) { Text("Follow global setting").tag("global") Text("Include all windows").tag(WindowAppRule.include.rawValue) diff --git a/Sources/ThreeFingerSwitcher/KeyboardLanguage/AXHostProvider.swift b/Sources/ThreeFingerSwitcher/KeyboardLanguage/AXHostProvider.swift index ccad447..1df4352 100644 --- a/Sources/ThreeFingerSwitcher/KeyboardLanguage/AXHostProvider.swift +++ b/Sources/ThreeFingerSwitcher/KeyboardLanguage/AXHostProvider.swift @@ -43,7 +43,9 @@ final class AXHostProvider: HostProvider { // Guard: never record a private/incognito window's host (best-effort; "if unsure, skip"). guard !looksPrivate(window) else { return nil } - guard let field = addressField(in: window) else { return nil } + guard let field = cachedAddressField(pid: app.processIdentifier, window: window) + ?? addressField(in: window) else { return nil } + lastField = (app.processIdentifier, window, field) // Guard: if the address field is the focused element, its value is the user's typed text mid-edit, // not a committed host — treat as no-host so we never learn/apply from a half-typed URL. @@ -75,6 +77,23 @@ final class AXHostProvider: HostProvider { // MARK: - Address-bar search + /// The last resolved address field, keyed by the window it was found in. This provider is driven + /// by a 0.5 s poll for as long as a browser is frontmost, and each `addressField` search is a + /// breadth-first AX walk of up to 400 elements — each a synchronous round-trip serviced by the + /// BROWSER's main thread. Re-walking per tick was a standing 2 Hz main-thread tax; re-validating + /// the remembered field costs one round-trip instead. + private var lastField: (pid: pid_t, window: AXUIElement, field: AXUIElement)? + + /// The remembered field when it belongs to this exact window and still reads as a text field + /// (the one-call liveness probe — a closed/rebuilt toolbar fails it and triggers a fresh walk). + private func cachedAddressField(pid: pid_t, window: AXUIElement) -> AXUIElement? { + guard let last = lastField, last.pid == pid, CFEqual(last.window, window), + axString(last.field, kAXRoleAttribute as String) == (kAXTextFieldRole as String) else { + return nil + } + return last.field + } + /// Find the window's address-bar text field. Browser AX trees vary, so this is heuristic and /// defensive: we breadth-first walk the window subtree (depth-bounded so a deep toolbar can't stall /// us) and pick the first `AXTextField` that carries a URL-ish identifier/description. Returns nil diff --git a/Sources/ThreeFingerSwitcher/KeyboardLanguage/AppleEventsHostProvider.swift b/Sources/ThreeFingerSwitcher/KeyboardLanguage/AppleEventsHostProvider.swift index edd5de9..73b5532 100644 --- a/Sources/ThreeFingerSwitcher/KeyboardLanguage/AppleEventsHostProvider.swift +++ b/Sources/ThreeFingerSwitcher/KeyboardLanguage/AppleEventsHostProvider.swift @@ -66,7 +66,16 @@ final class AppleEventsHostProvider: HostProvider { /// tab vocabulary differs by family: Chromium says "active tab", Safari says "current tab". private static func source(bundleID: String, family: BrowserFamily) -> String { let tab = (family == .safari) ? "current tab" : "active tab" - return "tell application id \"\(bundleID)\" to return URL of \(tab) of front window" + // `with timeout`: AppleScript's DEFAULT reply timeout is two minutes, and this executes + // synchronously on the main thread from a 0.5 s poll — a browser whose main thread is busy + // (heavy page, modal, beachball) would otherwise freeze the whole app, gestures included, for + // up to that long. One second is far beyond a responsive browser's reply and still bounded; + // a timed-out read returns nil → the caller falls back to the Accessibility reader. + return """ + with timeout of 1 second + tell application id "\(bundleID)" to return URL of \(tab) of front window + end timeout + """ } // MARK: - Parsing diff --git a/Sources/ThreeFingerSwitcher/KeyboardLanguage/BrowserContextMonitor.swift b/Sources/ThreeFingerSwitcher/KeyboardLanguage/BrowserContextMonitor.swift index 72a6070..be95e05 100644 --- a/Sources/ThreeFingerSwitcher/KeyboardLanguage/BrowserContextMonitor.swift +++ b/Sources/ThreeFingerSwitcher/KeyboardLanguage/BrowserContextMonitor.swift @@ -68,6 +68,7 @@ final class BrowserContextMonitor { let t = Timer.scheduledTimer(withTimeInterval: max(0.1, pollInterval), repeats: true) { [weak self] _ in MainActor.assumeIsolated { self?.poll() } } + t.tolerance = max(0.1, pollInterval) / 2 // a host re-check isn't deadline-critical timer = t } diff --git a/Sources/ThreeFingerSwitcher/KeyboardLanguage/KeyboardLanguageService.swift b/Sources/ThreeFingerSwitcher/KeyboardLanguage/KeyboardLanguageService.swift index e326dcf..878c755 100644 --- a/Sources/ThreeFingerSwitcher/KeyboardLanguage/KeyboardLanguageService.swift +++ b/Sources/ThreeFingerSwitcher/KeyboardLanguage/KeyboardLanguageService.swift @@ -107,6 +107,10 @@ final class KeyboardLanguageService { settledSourceForActiveContext = nil } + deinit { + if let activationObserver { NSWorkspace.shared.notificationCenter.removeObserver(activationObserver) } + } + // MARK: - Picker source list /// The user's selectable keyboard / input-method sources (id + localized name), forwarded from the diff --git a/Sources/ThreeFingerSwitcher/Launcher/FavoritesStore.swift b/Sources/ThreeFingerSwitcher/Launcher/FavoritesStore.swift index 531b9c9..4dba697 100644 --- a/Sources/ThreeFingerSwitcher/Launcher/FavoritesStore.swift +++ b/Sources/ThreeFingerSwitcher/Launcher/FavoritesStore.swift @@ -16,12 +16,15 @@ final class FavoritesStore: ObservableObject { @Published private(set) var favorites: Favorites - private convenience init() { self.init(defaults: .standard) } + // The app's store coalesces saves; `AppCoordinator.flushStores()` lands a pending edit at quit. + private convenience init() { self.init(defaults: .standard, saveDelay: 0.3) } /// Test/seam initializer: inject an isolated `UserDefaults`. Loads the stored record (migrating - /// older schema versions forward) or seeds the starter bands on first run. - init(defaults: UserDefaults) { + /// older schema versions forward) or seeds the starter bands on first run. `saveDelay` defaults + /// to 0 (synchronous saves) so tests can reload immediately after a mutation. + init(defaults: UserDefaults, saveDelay: TimeInterval = 0) { self.defaults = defaults + self.saveDelay = saveDelay if let data = defaults.data(forKey: key), let decoded = try? JSONDecoder().decode(Favorites.self, from: data) { let storedVersion = decoded.schemaVersion @@ -38,11 +41,41 @@ final class FavoritesStore: ObservableObject { // MARK: - Mutation - /// Apply an edit and persist it. All editor/quick-add paths funnel through here. + /// Apply an edit and persist it. All editor/quick-add paths funnel through here. The in-memory + /// model updates synchronously (so the editor stays live); persistence is coalesced behind + /// `saveDelay` — eight Bands-editor fields call this per KEYSTROKE, and each save re-encodes the + /// whole favorites tree (every band, item, and script body) to JSON on the main thread. func mutate(_ block: (inout Favorites) -> Void) { var copy = favorites block(©) favorites = copy + scheduleSave() + } + + /// Seconds a save is deferred to coalesce bursts. `0` saves synchronously (the test seam — the + /// default for `init(defaults:)`; the app's shared store passes a real delay). + private let saveDelay: TimeInterval + private var pendingSave: DispatchWorkItem? + + private func scheduleSave() { + guard saveDelay > 0 else { save(); return } + pendingSave?.cancel() + let work = DispatchWorkItem { [weak self] in + MainActor.assumeIsolated { + guard let self else { return } + self.pendingSave = nil + self.save() + } + } + pendingSave = work + DispatchQueue.main.asyncAfter(deadline: .now() + saveDelay, execute: work) + } + + /// Write any coalesced edit now (app termination, window close). + func flushPendingSave() { + guard let work = pendingSave else { return } + work.cancel() + pendingSave = nil save() } diff --git a/Sources/ThreeFingerSwitcher/Launcher/IconCache.swift b/Sources/ThreeFingerSwitcher/Launcher/IconCache.swift new file mode 100644 index 0000000..badd83f --- /dev/null +++ b/Sources/ThreeFingerSwitcher/Launcher/IconCache.swift @@ -0,0 +1,26 @@ +import AppKit + +/// Process-wide memo for `NSWorkspace.shared.icon(forFile:)`. +/// +/// The launcher grid and the Hub's band editor render an app/file icon per cell — and SwiftUI +/// re-evaluates those bodies on every model change (every gesture step, every slider tick). Each +/// `icon(forFile:)` call is an IconServices round-trip that returns a FRESH `NSImage` instance, so +/// besides the IPC, SwiftUI could never identity-match the image against the previous frame and +/// re-rasterized every cell per step. One stable instance per path fixes both. +/// +/// Bounded: the memo is cleared wholesale past `capacity` (favorites are a few dozen items; a full +/// clear is simpler and just as fast as LRU bookkeeping here). Stale-icon risk is accepted — an app +/// updating its icon mid-session shows the old one until relaunch, which is what the Dock does too. +@MainActor +enum IconCache { + private static var icons: [String: NSImage] = [:] + private static let capacity = 256 + + static func icon(forFile path: String) -> NSImage { + if let cached = icons[path] { return cached } + if icons.count >= capacity { icons.removeAll(keepingCapacity: true) } + let image = NSWorkspace.shared.icon(forFile: path) + icons[path] = image + return image + } +} diff --git a/Sources/ThreeFingerSwitcher/Launcher/LaunchService.swift b/Sources/ThreeFingerSwitcher/Launcher/LaunchService.swift index 45cddb9..4ebe6a2 100644 --- a/Sources/ThreeFingerSwitcher/Launcher/LaunchService.swift +++ b/Sources/ThreeFingerSwitcher/Launcher/LaunchService.swift @@ -228,8 +228,10 @@ final class LaunchService { var writes = pasteboardWrites(for: entry) writes = expandImageFormats(writes) // offer both PNG + TIFF for broad image paste appendColorTextFallback(&writes, entry: entry) // hex text for a copied color - pasteboard.clearContents() + // Nothing to write (e.g. the entry's blob was pruned between band build and fire) must NOT + // wipe the user's current clipboard — clear only once there is content to replace it with. guard !writes.isEmpty else { return false } + pasteboard.clearContents() let pbItem = NSPasteboardItem() for write in writes { pbItem.setData(write.data, forType: NSPasteboard.PasteboardType(write.uti)) @@ -420,10 +422,14 @@ final class LaunchService { } private func emptyTrash() { - let fm = FileManager.default - guard let trash = try? fm.url(for: .trashDirectory, in: .userDomainMask, appropriateFor: nil, create: false), - let items = try? fm.contentsOfDirectory(at: trash, includingPropertiesForKeys: nil) else { return } - for item in items { try? fm.removeItem(at: item) } + // Off the main thread: a large Trash (thousands of items) took seconds to unlink, freezing + // the app — gestures included — right after the launcher dismissed. Nothing is reported back. + DispatchQueue.global(qos: .utility).async { + let fm = FileManager.default + guard let trash = try? fm.url(for: .trashDirectory, in: .userDomainMask, appropriateFor: nil, create: false), + let items = try? fm.contentsOfDirectory(at: trash, includingPropertiesForKeys: nil) else { return } + for item in items { try? fm.removeItem(at: item) } + } } // MARK: - URL / link opening @@ -436,16 +442,22 @@ final class LaunchService { private func openURL(_ url: URL, handler: URL?, newWindow: Bool) { if newWindow, let appName = Self.handlerAppName(handler: handler, for: url) { let script = Self.newWindowScript(url: url, appName: appName) - DispatchQueue.global(qos: .userInitiated).async { [weak self] in - let p = Process() - p.executableURL = URL(fileURLWithPath: "/usr/bin/osascript") - p.arguments = ["-e", script] - let started = (try? p.run()) != nil - if started { p.waitUntilExit() } - if !started || p.terminationStatus != 0 { - DispatchQueue.main.async { self?.plainOpen(url, handler: handler) } + // Completion via `terminationHandler` (see `run`): an `osascript` blocked on a pending + // Automation prompt otherwise held a global-queue thread for as long as the prompt sat. + let p = Process() + p.executableURL = URL(fileURLWithPath: "/usr/bin/osascript") + p.arguments = ["-e", script] + p.terminationHandler = { [weak self] finished in + Task { @MainActor in + self?.runningProcesses.remove(finished) + if finished.terminationStatus != 0 { self?.plainOpen(url, handler: handler) } } } + if (try? p.run()) != nil { + runningProcesses.insert(p) + } else { + plainOpen(url, handler: handler) + } return } plainOpen(url, handler: handler) @@ -709,6 +721,10 @@ final class LaunchService { /// lands on the current Space), falling back to a synthesized ⌘N if the item can't be pressed. private func makeNewWindow(for app: NSRunningApplication) { let pid = app.processIdentifier + // One in-flight new-window request per app: a double-lift on the dwell-arm (two fires + // within the deferral) otherwise pressed New Window twice and opened two windows. + guard !pendingNewWindowPids.contains(pid) else { return } + pendingNewWindowPids.insert(pid) // Do NOT activate first. Activating fronts the app's existing window and, if it lives on // another Space, teleports the user there before the new window even exists. Triggering the // new window while we stay put makes it appear on the CURRENT Space (AX menu-press and ⌘N @@ -719,10 +735,13 @@ final class LaunchService { // The new window is being created on the current Space; bring the app forward to it once it // exists. Deferred so we never activate while the only window is still off-Space. DispatchQueue.main.asyncAfter(deadline: .now() + 0.12) { [weak self] in + guard let self else { return } + self.pendingNewWindowPids.remove(pid) app.activate(options: []) - self?.raiseFrontWindow(pid: pid) + self.raiseFrontWindow(pid: pid) } } + private var pendingNewWindowPids: Set = [] /// Focus a single-window app's existing window. If a window is already on the current Space we /// focus it locally (no teleport). If it lives only off-Space, macOS won't let an unprivileged app @@ -759,13 +778,20 @@ final class LaunchService { /// become the active one after the reopen activates the app. An app that responds to neither needs /// the explicit `.quitAndReopenHere` strategy. private func reopenWindowlessApp(_ app: NSRunningApplication, bundleURL: URL) { - launch(bundleURL: bundleURL, newInstance: false) let pid = app.processIdentifier + // Same single-flight rule as `makeNewWindow`: two rapid fires on a windowless app must not + // both see "still no window" at +0.5 s and escalate to two New Window presses. + guard !pendingReopenPids.contains(pid) else { return } + pendingReopenPids.insert(pid) + launch(bundleURL: bundleURL, newInstance: false) DispatchQueue.main.asyncAfter(deadline: .now() + 0.5) { [weak self] in - guard let self, self.windowCount(pid: pid) == 0 else { return } + guard let self else { return } + self.pendingReopenPids.remove(pid) + guard self.windowCount(pid: pid) == 0 else { return } self.makeNewWindow(for: app) } } + private var pendingReopenPids: Set = [] /// Number of the app's AX windows (across all Spaces) — used to tell whether a reopen actually /// produced a window before escalating to a new-window command. @@ -839,25 +865,31 @@ final class LaunchService { } } - /// Run a process off the main thread and report success/failure when it exits. + /// Run a process and report success/failure when it exits. Completion arrives via + /// `terminationHandler` — NOT `waitUntilExit()` on a global queue, which parked one GCD worker + /// thread per running script for its whole lifetime (a hung script held it forever, and enough + /// of them starved every other global-queue user in the process). private func run(executable: String, args: [String], title: String) { - DispatchQueue.global(qos: .userInitiated).async { [weak self] in - let process = Process() - process.executableURL = URL(fileURLWithPath: executable) - process.arguments = args - var ok = false - var message = "" - do { - try process.run() - process.waitUntilExit() - ok = process.terminationStatus == 0 - if !ok { message = "Exited with status \(process.terminationStatus)." } - } catch { - message = error.localizedDescription + let process = Process() + process.executableURL = URL(fileURLWithPath: executable) + process.arguments = args + process.terminationHandler = { [weak self] finished in + let ok = finished.terminationStatus == 0 + let message = ok ? "Done." : "Exited with status \(finished.terminationStatus)." + Task { @MainActor in + self?.runningProcesses.remove(finished) + self?.notify(title: title, body: message, success: ok) } - Task { @MainActor in self?.notify(title: title, body: ok ? "Done." : message, success: ok) } + } + do { + try process.run() // spawn only; returns immediately + runningProcesses.insert(process) + } catch { + notify(title: title, body: error.localizedDescription, success: false) } } + /// Keeps spawned processes alive until their termination handler fires. + private var runningProcesses: Set = [] // MARK: - Accessibility: menu-bar new-window @@ -884,8 +916,8 @@ final class LaunchService { /// Walk the app's menu bar for a File-menu item whose title matches a new-window candidate. private func findNewWindowItem(pid: pid_t) -> AXUIElement? { let appEl = AXUIElementCreateApplication(pid) - guard let menuBar = axCopy(appEl, kAXMenuBarAttribute as String), - let topItems = axChildren(menuBar as! AXUIElement) else { return nil } + guard let menuBar = axElement(appEl, kAXMenuBarAttribute as String), + let topItems = axChildren(menuBar) else { return nil } for top in topItems { // The File menu's single child is the AXMenu holding the items. guard let submenus = axChildren(top), let menu = submenus.first, diff --git a/Sources/ThreeFingerSwitcher/Onboarding/FirstRunState.swift b/Sources/ThreeFingerSwitcher/Onboarding/FirstRunState.swift index 52a9063..e2cc71f 100644 --- a/Sources/ThreeFingerSwitcher/Onboarding/FirstRunState.swift +++ b/Sources/ThreeFingerSwitcher/Onboarding/FirstRunState.swift @@ -111,13 +111,23 @@ final class FirstRunStore { self.defaults = defaults } + /// In-memory mirror of the persisted stage. `isCompleted` sits on the recognizer's per-frame + /// path (`wizardOwnsGestures` → `launcherFocusIsOnBandList`, 60–120×/s mid-gesture), and the + /// defaults read + String bridge + raw-value parse per frame was free to eliminate. The store is + /// the only writer of the key, so write-through keeps the mirror exact. + private var cachedStage: FirstRunStage? + var stage: FirstRunStage { get { - guard let raw = defaults.string(forKey: Self.stageKey), - let stage = FirstRunStage(rawValue: raw) else { return .fresh } + if let cachedStage { return cachedStage } + let stage = defaults.string(forKey: Self.stageKey).flatMap(FirstRunStage.init(rawValue:)) ?? .fresh + cachedStage = stage return stage } - set { defaults.set(newValue.rawValue, forKey: Self.stageKey) } + set { + cachedStage = newValue + defaults.set(newValue.rawValue, forKey: Self.stageKey) + } } var isCompleted: Bool { stage == .completed } diff --git a/Sources/ThreeFingerSwitcher/Onboarding/FirstTouchWizardModel.swift b/Sources/ThreeFingerSwitcher/Onboarding/FirstTouchWizardModel.swift index 9533676..8eedab0 100644 --- a/Sources/ThreeFingerSwitcher/Onboarding/FirstTouchWizardModel.swift +++ b/Sources/ThreeFingerSwitcher/Onboarding/FirstTouchWizardModel.swift @@ -360,7 +360,8 @@ final class FirstTouchWizardModel: ObservableObject { // [weak self]: a strong capture would make the timer retain the model (and its demo // SwitcherModel + thumbnails) past teardown if the timer is ever left running. attractTimer = Timer.scheduledTimer(withTimeInterval: 1.0 / 30.0, repeats: true) { [weak self] _ in - Task { @MainActor in self?.attractTick() } + // The timer fires on the main run loop; assumeIsolated avoids allocating a Task 30×/s. + MainActor.assumeIsolated { self?.attractTick() } } } diff --git a/Sources/ThreeFingerSwitcher/Onboarding/FirstTouchWizardView.swift b/Sources/ThreeFingerSwitcher/Onboarding/FirstTouchWizardView.swift index 69c80e5..1508483 100644 --- a/Sources/ThreeFingerSwitcher/Onboarding/FirstTouchWizardView.swift +++ b/Sources/ThreeFingerSwitcher/Onboarding/FirstTouchWizardView.swift @@ -67,11 +67,15 @@ private struct WizardDemoStrip: View { .scaleEffect(model.liveTouchActive ? 0.845 : 0.82) .frame(height: SwitcherLayout.panelHeight * 0.85) .allowsHitTesting(false) - .background( - BreathingGlowBackdrop() - .opacity(model.liveTouchActive ? 1 : 0) - .animation(.easeInOut(duration: 0.6), value: model.liveTouchActive) - ) + .background { + // Mounted only while active: at `.opacity(0)` the backdrop's 15 Hz TimelineView + // kept ticking and re-drawing a blurred gradient nobody could see. + if model.liveTouchActive { + BreathingGlowBackdrop() + .transition(.opacity) + } + } + .animation(.easeInOut(duration: 0.6), value: model.liveTouchActive) .animation(WizardMotion.arrival, value: model.liveTouchActive) if model.stage == .hand { FingerDotsPad(dots: model.liveTouchActive ? model.fingerDots : model.ghostDots, diff --git a/Sources/ThreeFingerSwitcher/Overlay/LauncherView.swift b/Sources/ThreeFingerSwitcher/Overlay/LauncherView.swift index 7af600f..eb73d09 100644 --- a/Sources/ThreeFingerSwitcher/Overlay/LauncherView.swift +++ b/Sources/ThreeFingerSwitcher/Overlay/LauncherView.swift @@ -17,10 +17,10 @@ extension Color { struct LauncherView: View { @ObservedObject var model: LauncherModel - private var columns: [GridItem] { + private static let columns: [GridItem] = Array(repeating: GridItem(.fixed(LauncherGridLayout.cellWidth), spacing: LauncherGridLayout.spacing), count: LauncherGridLayout.columns) - } + private var columns: [GridItem] { Self.columns } var body: some View { HStack(spacing: 0) { @@ -119,8 +119,14 @@ struct LauncherView: View { } .padding(.top, LauncherGridLayout.gridTopInset) } - .onChange(of: model.selectedIndex) { scroll(proxy) } - .onChange(of: model.focus) { scroll(proxy) } + .onChange(of: model.selectedIndex) { old, new in + // Animate only when the step crosses a ROW: a within-row step every 30–60 ms + // otherwise starts a new 160 ms scroll animation before the previous one ends, + // stacking 3–5 in-flight animations (each re-measuring the grid) for the whole scrub. + let cols = LauncherGridLayout.columns + scroll(proxy, animated: old / cols != new / cols) + } + .onChange(of: model.focus) { scroll(proxy, animated: true) } .onChange(of: model.currentBand) { proxy.scrollTo(scrollTarget, anchor: .top) } } } @@ -130,9 +136,13 @@ struct LauncherView: View { return model.items[model.selectedIndex].id } - private func scroll(_ proxy: ScrollViewProxy) { + private func scroll(_ proxy: ScrollViewProxy, animated: Bool) { guard let target = scrollTarget else { return } - withAnimation(.easeInOut(duration: 0.16)) { proxy.scrollTo(target, anchor: .center) } + if animated { + withAnimation(.easeInOut(duration: 0.16)) { proxy.scrollTo(target, anchor: .center) } + } else { + proxy.scrollTo(target, anchor: .center) + } } @ViewBuilder @@ -202,19 +212,25 @@ struct LauncherView: View { } } + // Memoized (IconCache): this runs per cell per render, at gesture rate — see IconCache's doc. private func appIcon(for item: LaunchItem) -> NSImage { if case let .app(bundleURL, _) = item.kind { - return NSWorkspace.shared.icon(forFile: bundleURL.path) + return IconCache.icon(forFile: bundleURL.path) } - return NSImage(systemSymbolName: "app.dashed", accessibilityDescription: nil) ?? NSImage() + return Self.placeholderAppIcon } private func fileIcon(for item: LaunchItem) -> NSImage { if case let .path(url) = item.kind { - return NSWorkspace.shared.icon(forFile: url.path) + return IconCache.icon(forFile: url.path) } - return NSImage(systemSymbolName: "folder", accessibilityDescription: nil) ?? NSImage() + return Self.placeholderFolderIcon } + + private static let placeholderAppIcon = + NSImage(systemSymbolName: "app.dashed", accessibilityDescription: nil) ?? NSImage() + private static let placeholderFolderIcon = + NSImage(systemSymbolName: "folder", accessibilityDescription: nil) ?? NSImage() } /// The single selection highlight: a Liquid Glass rounded square that starts nearly transparent and diff --git a/Sources/ThreeFingerSwitcher/Permissions/PermissionsService.swift b/Sources/ThreeFingerSwitcher/Permissions/PermissionsService.swift index d0d4187..edebb7f 100644 --- a/Sources/ThreeFingerSwitcher/Permissions/PermissionsService.swift +++ b/Sources/ThreeFingerSwitcher/Permissions/PermissionsService.swift @@ -54,8 +54,10 @@ final class PermissionsService: ObservableObject { /// Injectable for tests: builds the (already-scheduled) repeating poll timer. var pollTimerFactory: (TimeInterval, @escaping @MainActor () -> Void) -> Timer = { interval, tick in + // Fires on the main run loop: assumeIsolated runs the tick synchronously (no per-tick Task, + // and no late tick landing after `stopPolling()` has invalidated the timer). Timer.scheduledTimer(withTimeInterval: interval, repeats: true) { _ in - Task { @MainActor in tick() } + MainActor.assumeIsolated { tick() } } } diff --git a/Sources/ThreeFingerSwitcher/TouchInput/KeyboardSwitcherTap.swift b/Sources/ThreeFingerSwitcher/TouchInput/KeyboardSwitcherTap.swift index 1c8e160..8289102 100644 --- a/Sources/ThreeFingerSwitcher/TouchInput/KeyboardSwitcherTap.swift +++ b/Sources/ThreeFingerSwitcher/TouchInput/KeyboardSwitcherTap.swift @@ -79,6 +79,9 @@ final class KeyboardSwitcherTap { MainActor.assumeIsolated { self?.reviveIfDisabled() } } watchdog?.tolerance = Self.watchdogInterval / 2 // cheap check; let the OS coalesce it + // `.common` so a revive isn't deferred for as long as a menu or drag keeps the run loop in a + // tracking mode (the tap source itself is already in `.commonModes`). + if let watchdog { RunLoop.main.add(watchdog, forMode: .common) } isRunning = true return true } diff --git a/Sources/ThreeFingerSwitcher/TouchInput/ScrollEventTap.swift b/Sources/ThreeFingerSwitcher/TouchInput/ScrollEventTap.swift index 0652dae..5ff665b 100644 --- a/Sources/ThreeFingerSwitcher/TouchInput/ScrollEventTap.swift +++ b/Sources/ThreeFingerSwitcher/TouchInput/ScrollEventTap.swift @@ -60,6 +60,9 @@ final class ScrollEventTap { MainActor.assumeIsolated { self?.reviveIfDisabled() } } watchdog?.tolerance = Self.watchdogInterval / 2 // cheap check; let the OS coalesce it + // `.common` so a revive isn't deferred for as long as a menu or drag keeps the run loop in a + // tracking mode (the tap source itself is already in `.commonModes`). + if let watchdog { RunLoop.main.add(watchdog, forMode: .common) } isRunning = true return true } diff --git a/Sources/ThreeFingerSwitcher/TouchInput/TouchEngine.swift b/Sources/ThreeFingerSwitcher/TouchInput/TouchEngine.swift index d0379bc..82acf13 100644 --- a/Sources/ThreeFingerSwitcher/TouchInput/TouchEngine.swift +++ b/Sources/ThreeFingerSwitcher/TouchInput/TouchEngine.swift @@ -103,17 +103,25 @@ final class TouchEngine { isListening = true resetMotion() + // Generation-tagged: `stop()` cancels the task, but a task parked in `next()` with no + // fingers down only observes cancellation once a frame arrives. If the shared stream's side + // ever outlived that cancellation, a stale consumer would feed the engine alongside the new + // one — the app-layer twin of the framework's orphaned-device bug. The tag makes a stale + // side inert the moment it wakes, whatever the stream does. + consumerGeneration &+= 1 + let generation = consumerGeneration consumer = Task { @MainActor [weak self] in - guard let self else { return } - for await frame in manager.touchDataStream { - if Task.isCancelled { break } + for await frame in OMSManager.shared.touchDataStream { + guard let self, !Task.isCancelled, self.consumerGeneration == generation else { break } self.process(frame) } } } + private var consumerGeneration = 0 func stop() { guard isListening else { return } + consumerGeneration &+= 1 // retire the current consumer even before its cancellation lands consumer?.cancel() consumer = nil _ = manager.stopListening() diff --git a/Sources/ThreeFingerSwitcher/Windows/AXPrivate.swift b/Sources/ThreeFingerSwitcher/Windows/AXPrivate.swift index 95469f4..e5bc2a5 100644 --- a/Sources/ThreeFingerSwitcher/Windows/AXPrivate.swift +++ b/Sources/ThreeFingerSwitcher/Windows/AXPrivate.swift @@ -25,6 +25,21 @@ func axString(_ element: AXUIElement, _ attribute: String) -> String? { axCopy(element, attribute) as? String } +/// An `AXUIElement`-valued attribute, type-checked before the cast. `axCopy` returns an untyped +/// `CFTypeRef`, and `as!` on a CF value TRAPS on a type mismatch — nothing stops a misbehaving app's +/// AX server (Electron, Qt, Java, Wine…) from answering a window attribute with a string or number. +/// Every element-valued read must go through here, never `as! AXUIElement`. +func axElement(_ element: AXUIElement, _ attribute: String) -> AXUIElement? { + guard let value = axCopy(element, attribute), CFGetTypeID(value) == AXUIElementGetTypeID() else { return nil } + return (value as! AXUIElement) +} + +/// An `AXValue`-valued attribute (position / size), type-checked — same rationale as `axElement`. +func axValue(_ element: AXUIElement, _ attribute: String) -> AXValue? { + guard let value = axCopy(element, attribute), CFGetTypeID(value) == AXValueGetTypeID() else { return nil } + return (value as! AXValue) +} + /// Build the 20-byte remote token AltTab uses with `_AXUIElementCreateWithRemoteToken`: /// pid (4) + 0 (4) + magic 0x636f636f "coco" (4) + axUiElementId (8). private func remoteToken(pid: pid_t, id: UInt) -> Data { diff --git a/Sources/ThreeFingerSwitcher/Windows/HubSwitcherEntry.swift b/Sources/ThreeFingerSwitcher/Windows/HubSwitcherEntry.swift index 3078cc6..d1394e8 100644 --- a/Sources/ThreeFingerSwitcher/Windows/HubSwitcherEntry.swift +++ b/Sources/ThreeFingerSwitcher/Windows/HubSwitcherEntry.swift @@ -88,7 +88,10 @@ enum HubSwitcherEntry { /// window via `present` instead of the cross-Space SkyLight raise). A `nil` `windowNumber` (the Hub /// was never created) never matches. static func isHub(selectedID: CGWindowID, hubWindowNumber: Int?) -> Bool { - guard let hubWindowNumber else { return false } + // `NSWindow.windowNumber` is ≤ 0 for a window with no window device (e.g. the retained Hub + // after it was closed), and `CGWindowID(Int)` TRAPS on a negative value — this ran on every + // switcher commit once the Hub had been opened and closed. + guard let hubWindowNumber, hubWindowNumber > 0 else { return false } return selectedID == CGWindowID(hubWindowNumber) } } diff --git a/Sources/ThreeFingerSwitcher/Windows/MRUTracker.swift b/Sources/ThreeFingerSwitcher/Windows/MRUTracker.swift index 3b52c41..a7b3bd9 100644 --- a/Sources/ThreeFingerSwitcher/Windows/MRUTracker.swift +++ b/Sources/ThreeFingerSwitcher/Windows/MRUTracker.swift @@ -34,6 +34,10 @@ final class MRUTracker { observer = nil } + deinit { + if let observer { NSWorkspace.shared.notificationCenter.removeObserver(observer) } + } + func promote(_ pid: pid_t) { order.removeAll { $0 == pid } order.insert(pid, at: 0) diff --git a/Sources/ThreeFingerSwitcher/Windows/StageManager.swift b/Sources/ThreeFingerSwitcher/Windows/StageManager.swift index b89e72f..e8c2357 100644 --- a/Sources/ThreeFingerSwitcher/Windows/StageManager.swift +++ b/Sources/ThreeFingerSwitcher/Windows/StageManager.swift @@ -13,16 +13,25 @@ import Foundation /// /// There is no public API for this; reading the `com.apple.WindowManager` preference domain is the /// community-standard method (used by Keyboard Maestro, Alfred, MDM scripts). `cfprefsd` caches the -/// value, so we synchronize before each read so a mid-session Stage Manager toggle is reflected. +/// value, so we synchronize before each read so a mid-session Stage Manager toggle is reflected — +/// but at most once per `ttl`: the synchronize is a round-trip to `cfprefsd` that deliberately drops +/// the cached domain (so the read is a second trip), and it was being paid twice per commit, once per +/// Dock-preview hover, and up to six more times per off-Space hold-guard tick. +@MainActor enum StageManager { private static let appID = "com.apple.WindowManager" as CFString + private static var cached: (at: TimeInterval, value: Bool)? + /// Long enough to collapse a commit's burst of reads into one; short enough that a mid-session + /// toggle is still picked up within a couple of seconds. + private static let ttl: TimeInterval = 2.0 /// True when Stage Manager is currently enabled (`GloballyEnabled == 1`). static var isEnabled: Bool { + let now = ProcessInfo.processInfo.systemUptime + if let cached, now - cached.at < ttl { return cached.value } CFPreferencesAppSynchronize(appID) - guard let value = CFPreferencesCopyAppValue("GloballyEnabled" as CFString, appID) else { - return false - } - return (value as? NSNumber)?.boolValue ?? false + let value = (CFPreferencesCopyAppValue("GloballyEnabled" as CFString, appID) as? NSNumber)?.boolValue ?? false + cached = (now, value) + return value } } diff --git a/Sources/ThreeFingerSwitcher/Windows/ThumbnailService.swift b/Sources/ThreeFingerSwitcher/Windows/ThumbnailService.swift index 9755aff..11ffa89 100644 --- a/Sources/ThreeFingerSwitcher/Windows/ThumbnailService.swift +++ b/Sources/ThreeFingerSwitcher/Windows/ThumbnailService.swift @@ -95,9 +95,10 @@ final class ThumbnailService { guard !targets.isEmpty else { return } sweepGeneration += 1 let generation = sweepGeneration - sweepTask = Task { - await self.refreshBatch(targets) - if self.sweepGeneration == generation { self.sweepTask = nil } + sweepTask = Task { [weak self] in + await self?.refreshBatch(targets) + guard let self, self.sweepGeneration == generation else { return } + self.sweepTask = nil } } @@ -356,7 +357,11 @@ final class ThumbnailService { static func captureDimensions(windowSize: CGSize, backingScale: CGFloat, cap: CGSize) -> (width: Int, height: Int) { let nativeW = max(windowSize.width * backingScale, 1) let nativeH = max(windowSize.height * backingScale, 1) + // `Swift.max(NaN, 1)` is NaN, and `Int(NaN)` / `Int(inf)` TRAP — a degenerate SCWindow + // frame (zero-size, or a compositor glitch) must yield a 1×1 capture, not a crash. + guard nativeW.isFinite, nativeH.isFinite, nativeW > 0, nativeH > 0 else { return (1, 1) } let fit = min(cap.width / nativeW, cap.height / nativeH, 1) + guard fit.isFinite, fit > 0 else { return (1, 1) } return (max(Int(nativeW * fit), 1), max(Int(nativeH * fit), 1)) } diff --git a/Sources/ThreeFingerSwitcher/Windows/WindowFocusTracker.swift b/Sources/ThreeFingerSwitcher/Windows/WindowFocusTracker.swift index 0aa943c..c4d39e7 100644 --- a/Sources/ThreeFingerSwitcher/Windows/WindowFocusTracker.swift +++ b/Sources/ThreeFingerSwitcher/Windows/WindowFocusTracker.swift @@ -68,8 +68,11 @@ final class WindowFocusTracker { // installed with an unretained refcon (`passUnretained`) — drop it before the tracker dies so // a late focus-change callback can never dereference freed memory. if let axObserver { - CFRunLoopRemoveSource(CFRunLoopGetMain(), AXObserverGetRunLoopSource(axObserver), .defaultMode) + // Must match the mode the source was ADDED in (`.commonModes`) — removing from a + // different mode leaves it attached, and the callback would then reach freed memory. + CFRunLoopRemoveSource(CFRunLoopGetMain(), AXObserverGetRunLoopSource(axObserver), .commonModes) } + if let activationObserver { NSWorkspace.shared.notificationCenter.removeObserver(activationObserver) } } /// Move `wid` to the front (most recent). Idempotent — a re-promote of the already-front window @@ -96,8 +99,7 @@ final class WindowFocusTracker { func focusedWindowID(pid: pid_t) -> CGWindowID? { guard pid != getpid(), AXIsProcessTrusted() else { return nil } let appEl = AXUIElementCreateApplication(pid) - guard let raw = axCopy(appEl, kAXFocusedWindowAttribute as String) else { return nil } - let element = raw as! AXUIElement + guard let element = axElement(appEl, kAXFocusedWindowAttribute as String) else { return nil } return axWindowID(element) } @@ -120,7 +122,10 @@ final class WindowFocusTracker { // the app. Promote is idempotent, so a double-fire is harmless. AXObserverAddNotification(observer, appEl, kAXFocusedWindowChangedNotification as CFString, context) AXObserverAddNotification(observer, appEl, kAXMainWindowChangedNotification as CFString, context) - CFRunLoopAddSource(CFRunLoopGetMain(), AXObserverGetRunLoopSource(observer), .defaultMode) + // `.commonModes`, not `.defaultMode`: the main run loop sits in a tracking mode during menu + // tracking, window drags, and scroll momentum — a focus change landing then was silently + // missed (the source isn't serviced in `.defaultMode` only), leaving the MRU order stale. + CFRunLoopAddSource(CFRunLoopGetMain(), AXObserverGetRunLoopSource(observer), .commonModes) axObserver = observer axObservedPID = pid } @@ -128,7 +133,7 @@ final class WindowFocusTracker { /// Remove the current observer's run-loop source and drop it. private func teardownObserver() { if let axObserver { - CFRunLoopRemoveSource(CFRunLoopGetMain(), AXObserverGetRunLoopSource(axObserver), .defaultMode) + CFRunLoopRemoveSource(CFRunLoopGetMain(), AXObserverGetRunLoopSource(axObserver), .commonModes) } axObserver = nil axObservedPID = nil diff --git a/Sources/ThreeFingerSwitcher/Windows/WindowService.swift b/Sources/ThreeFingerSwitcher/Windows/WindowService.swift index a91c75f..2554cc5 100644 --- a/Sources/ThreeFingerSwitcher/Windows/WindowService.swift +++ b/Sources/ThreeFingerSwitcher/Windows/WindowService.swift @@ -69,7 +69,15 @@ final class WindowService { private func seedElementCache(pid: pid_t) { guard pid != getpid(), AXIsProcessTrusted() else { return } for (wid, el) in currentSpaceElements(pid: pid) { elementCache[wid] = el } + // The cache is pruned to live windows only inside `snapshot()`; a long session that never + // opens the switcher (Dock previews / ⌘-Tab off) would otherwise accumulate an element for + // every window of every app ever activated. Past a generous ceiling, drop elements whose + // owning process is gone — `AXUIElementGetPid` is a local token read, no IPC. + guard elementCache.count > Self.elementCacheSoftCap else { return } + let running = Set(NSWorkspace.shared.runningApplications.map(\.processIdentifier)) + elementCache = elementCache.filter { axPid($0.value).map(running.contains) ?? false } } + private static let elementCacheSoftCap = 512 private struct CGMeta { let pid: pid_t @@ -216,6 +224,7 @@ final class WindowService { // (ids are unique per window lifetime, so a stale id can never mis-rank a new window). focus.evict(keepingLive: Set(spaceForWindow.keys)) mru.evict(keepingLive: Set(appsByPid.keys)) + bundleKeyByPid = bundleKeyByPid.filter { appsByPid[$0.key] != nil } let meta = metadata(for: Array(spaceForWindow.keys)) @@ -273,9 +282,9 @@ final class WindowService { // apart from a real window. Genuine off-Space windows — including Chromium reachable only // via a cached element (Bug A) — keep listing via `elementCache`. `isSwitchable` still // excludes minimized windows and non-standard subroles (dialogs/sheets). - guard let element, isSwitchable(element) else { continue } + guard let element, let facts = switchableFacts(element) else { continue } - let title = axString(element, kAXTitleAttribute as String) + let title = facts.candidate.title ?? m.name ?? (app.localizedName ?? "") @@ -286,14 +295,14 @@ final class WindowService { title: title, appIcon: app.icon, frame: m.bounds, - realFrame: axFrame(element), + realFrame: facts.frame, axElement: element, isOnCurrentSpace: onCurrent, spaceID: placement.space, spaceIndex: model.indexBySpace[placement.space] ?? Int.max, // Only minimized windows that passed the relaxed gate reach here, so the flag is false unless - // the opt-in is on; gating the AX read keeps the default (setting-off) hot path unchanged. - isMinimized: settings.includeMinimizedWindows ? axBool(element, kAXMinimizedAttribute as String) : false + // the opt-in is on (the candidate read it for the verdict either way). + isMinimized: settings.includeMinimizedWindows ? facts.candidate.isMinimized : false ) rows.append((info, focus.rank(wid), mru.rank(m.pid), onCurrent, model.indexBySpace[placement.space] ?? Int.max, placement.z)) } @@ -672,9 +681,8 @@ final class WindowService { let app = NSWorkspace.shared.frontmostApplication, app.processIdentifier != getpid() else { return nil } let appEl = AXUIElementCreateApplication(app.processIdentifier) - guard let focusedRef = axCopy(appEl, kAXFocusedWindowAttribute as String) else { return nil } - let el = focusedRef as! AXUIElement - guard let wid = axWindowID(el) else { return nil } + guard let el = axElement(appEl, kAXFocusedWindowAttribute as String), + let wid = axWindowID(el) else { return nil } return WindowInfo( id: wid, pid: app.processIdentifier, appName: app.localizedName ?? "", title: axString(el, kAXTitleAttribute as String) ?? "", appIcon: app.icon, @@ -825,7 +833,7 @@ final class WindowService { return } - recover(window, attempt: attempt) + recover(window, attempt: attempt, token: token) scheduleWatchdog(window, token: token, attempt: attempt + 1) } @@ -833,7 +841,7 @@ final class WindowService { /// Attempt 2: the "benign nudge" — bounce activation through our own agent for one runloop /// tick, then re-activate the target, re-seating the WindowServer's key arbitration without /// the user touching Mission Control. - private func recover(_ window: WindowInfo, attempt: Int) { + private func recover(_ window: WindowInfo, attempt: Int, token: UInt64) { if attempt == 0 { // Re-validate the wid against the re-resolved element so we don't poke a dead id. if let el = resolveElement(window), let liveWid = axWindowID(el), liveWid == window.id { @@ -843,10 +851,13 @@ final class WindowService { NSRunningApplication(processIdentifier: window.pid)?.activate() } } else { - // Benign nudge: activate ourselves for one tick, then re-activate the target. + // Benign nudge: activate ourselves for one tick, then re-activate the target. The hop is + // token-guarded like every other deferred step on this path: a NEWER commit landing in + // that one run-loop turn must not be stomped by the previous target's focus sequence. NSApp.activate(ignoringOtherApps: true) DispatchQueue.main.async { [weak self] in - self?.focusSequence(window, offSpaceHandshake: !window.isOnCurrentSpace) + guard let self, self.commitSeq == token else { return } + self.focusSequence(window, offSpaceHandshake: !window.isOnCurrentSpace) } } } @@ -958,17 +969,37 @@ final class WindowService { /// `WindowFilter` (design D1). Size is the AX (real) size, so a Stage-Manager strip proxy — /// small in CGWindowList bounds but really large — is measured by its true size and not /// mistaken for a helper window. - private func candidate(for axWin: AXUIElement) -> WindowCandidate { + private func candidate(for axWin: AXUIElement, frame: CGRect? = nil) -> WindowCandidate { WindowCandidate( role: axString(axWin, kAXRoleAttribute as String), subrole: axString(axWin, kAXSubroleAttribute as String), title: axString(axWin, kAXTitleAttribute as String), - size: axFrame(axWin).size, + size: (frame ?? axFrame(axWin)).size, hasCloseButton: axCopy(axWin, kAXCloseButtonAttribute as String) != nil, isMinimized: axBool(axWin, kAXMinimizedAttribute as String) ) } + /// The snapshot's fused per-window read: ONE set of attribute round-trips yields both the + /// switchability verdict and the facts the listing needs (title, real frame, minimized). The + /// main loop used to re-read those three after `isSwitchable` had already fetched them — a + /// third of the blocking AX calls per window, inline in the gesture. Returns nil when the + /// window is not listed. + private func switchableFacts(_ axWin: AXUIElement) -> (candidate: WindowCandidate, frame: CGRect)? { + let frame = axFrame(axWin) + let cand = candidate(for: axWin, frame: frame) + guard WindowFilter.verdict(cand, policy: policy(for: axWin)) == .listed else { return nil } + return (cand, frame) + } + + /// The filter policy for a window element: its owning app's rule, or the global policy when + /// the pid can't be read. + private func policy(for axWin: AXUIElement) -> WindowFilterPolicy { + axPid(axWin).map(policy(forPid:)) + ?? WindowFilterPolicy(relaxed: settings.includeNonStandardWindows, + includeMinimized: settings.includeMinimizedWindows) + } + /// The per-app rule key: bundle ID, falling back to executable name (Qt/CLI-hosted apps like the /// Android emulator can lack a bundle ID). Cached per pid — bounded by the process count over the /// app's lifetime, and pid reuse across launches is rare enough that a stale hit is harmless (the @@ -1012,10 +1043,7 @@ final class WindowService { /// `minimizeAllWindows()` skips already-minimized windows BEFORE this gate, so admitting /// minimized windows here never re-minimizes one. private func isSwitchable(_ axWin: AXUIElement) -> Bool { - let pol = axPid(axWin).map(policy(forPid:)) - ?? WindowFilterPolicy(relaxed: settings.includeNonStandardWindows, - includeMinimized: settings.includeMinimizedWindows) - return WindowFilter.verdict(candidate(for: axWin), policy: pol) == .listed + WindowFilter.verdict(candidate(for: axWin), policy: policy(for: axWin)) == .listed } // MARK: - Window Inspector @@ -1063,11 +1091,11 @@ final class WindowService { private func axFrame(_ axWin: AXUIElement) -> CGRect { var origin = CGPoint.zero var size = CGSize.zero - if let posValue = axCopy(axWin, kAXPositionAttribute as String) { - AXValueGetValue(posValue as! AXValue, .cgPoint, &origin) + if let posValue = axValue(axWin, kAXPositionAttribute as String) { + AXValueGetValue(posValue, .cgPoint, &origin) } - if let sizeValue = axCopy(axWin, kAXSizeAttribute as String) { - AXValueGetValue(sizeValue as! AXValue, .cgSize, &size) + if let sizeValue = axValue(axWin, kAXSizeAttribute as String) { + AXValueGetValue(sizeValue, .cgSize, &size) } return CGRect(origin: origin, size: size) } diff --git a/Sources/ThreeFingerSwitcher/Windows/WindowSnapMonitor.swift b/Sources/ThreeFingerSwitcher/Windows/WindowSnapMonitor.swift index 3b2afc6..d5271e3 100644 --- a/Sources/ThreeFingerSwitcher/Windows/WindowSnapMonitor.swift +++ b/Sources/ThreeFingerSwitcher/Windows/WindowSnapMonitor.swift @@ -102,6 +102,7 @@ final class WindowSnapMonitor { } private func schedule(after delay: TimeInterval, _ body: @escaping @MainActor () -> Void) { + settleWork?.cancel() // one pending settle read at a time — a superseded one must never fire let work = DispatchWorkItem { MainActor.assumeIsolated { body() } } settleWork = work DispatchQueue.main.asyncAfter(deadline: .now() + delay, execute: work) @@ -183,14 +184,21 @@ final class WindowSnapMonitor { return (hit.id, hit.frame) } - /// The window's current frame, re-read through the SAME on-screen enumeration the hit-test uses - /// (one option set, one coordinate space). Deliberately NOT `CGWindowListCreateDescriptionFromArray`: - /// that legacy API expects each array element to be the raw window ID cast directly into the - /// pointer slot — `[id] as CFArray` bridges to CFNumbers, which it silently fails to unbox, - /// returning nil for EVERY window (the documented landmine `WindowService.windowIDArray` exists - /// to avoid; here the enumeration is simpler than the pointer-cast dance). + /// The window's current frame via a single-id `CGWindowList` query (same `kCGWindowBounds` + /// coordinate space as the hit-test enumeration). This runs on EVERY global mouse-up while window + /// groups are on; re-running the full on-screen enumeration here doubled the per-click cost. + /// Deliberately NOT `CGWindowListCreateDescriptionFromArray`: that legacy API expects each array + /// element to be the raw window ID cast directly into the pointer slot — `[id] as CFArray` bridges + /// to CFNumbers, which it silently fails to unbox (the documented `WindowService.windowIDArray` + /// landmine). `.optionIncludingWindow` takes the id directly. private static func frame(of id: CGWindowID) -> CGRect? { - onScreenWindows().first { $0.id == id }?.frame + guard let list = CGWindowListCopyWindowInfo([.optionIncludingWindow], id) as? [[String: Any]], + let info = list.first, + let boundsDict = info[kCGWindowBounds as String] as? [String: Any], + let bounds = CGRect(dictionaryRepresentation: boundsDict as CFDictionary) else { + return nil + } + return bounds } private static func changed(_ a: CGRect, _ b: CGRect) -> Bool { diff --git a/Tests/ThreeFingerSwitcherTests/ResourceBoundsTests.swift b/Tests/ThreeFingerSwitcherTests/ResourceBoundsTests.swift new file mode 100644 index 0000000..1ad481e --- /dev/null +++ b/Tests/ThreeFingerSwitcherTests/ResourceBoundsTests.swift @@ -0,0 +1,89 @@ +import XCTest +import AppKit +@testable import ThreeFingerSwitcherCore + +/// Guards the bounded-resource behavior added by `fix-progressive-cpu-degradation`: the +/// thumbnail cache is a true LRU pruned to live windows, and the MRU tracker forgets dead pids. +/// These are the accumulation bugs that made the app slower the longer it ran — keep them pinned. +@MainActor +final class ResourceBoundsTests: XCTestCase { + + private func image() -> NSImage { NSImage(size: NSSize(width: 2, height: 2)) } + + // MARK: - ThumbnailService cache + + func testRetainOnlyDropsFramesOfClosedWindows() { + let service = ThumbnailService() + service.inject(image(), for: 1) + service.inject(image(), for: 2) + service.inject(image(), for: 3) + + service.retain(only: [1, 3]) + + XCTAssertNotNil(service.cached(1)) + XCTAssertNil(service.cached(2), "a window no longer enumerated must not keep pinning its frame") + XCTAssertNotNil(service.cached(3)) + } + + func testRetainOnlyIgnoresAnEmptyLiveSet() { + let service = ThumbnailService() + service.inject(image(), for: 1) + + service.retain(only: []) // an enumeration hiccup must never wipe the last-good-frame store + + XCTAssertNotNil(service.cached(1)) + } + + func testReStoreRefreshesEvictionOrder() { + // The cache holds 64 frames. Fill it, touch the OLDEST again, then overflow by one: a true + // LRU evicts the second-oldest (id 2); the previous FIFO evicted id 1 even though it was + // just refreshed — which let long-dead windows outlive actively-refreshed ones. + let service = ThumbnailService() + for id in 1...64 { service.inject(image(), for: CGWindowID(id)) } + service.inject(image(), for: 1) + service.inject(image(), for: 65) + + XCTAssertNotNil(service.cached(1), "a re-stored frame must be treated as most recently used") + XCTAssertNil(service.cached(2), "the least recently stored frame is the one evicted") + XCTAssertNotNil(service.cached(65)) + } + + func testInjectNotifiesObserver() { + let service = ThumbnailService() + var delivered: [CGWindowID] = [] + service.onThumbnail = { id, _ in delivered.append(id) } + + service.inject(image(), for: 7) + + XCTAssertEqual(delivered, [7]) + XCTAssertNotNil(service.cached(7)) + } + + // MARK: - MRUTracker + + func testEvictForgetsDeadPidsAndKeepsRelativeOrder() { + let mru = MRUTracker() + mru.promote(10) + mru.promote(20) + mru.promote(30) // order: 30, 20, 10 + + mru.evict(keepingLive: [10, 30]) + + XCTAssertEqual(mru.rank(30), 0) + XCTAssertEqual(mru.rank(10), 1) + XCTAssertEqual(mru.rank(20), Int.max, "an exited app's pid must not linger in the MRU order") + XCTAssertEqual(mru.order, [30, 10]) + } + + func testStartIsIdempotent() { + // A second start() without stop() must not stack a second activation observer; stop() + // must then fully tear down (a third start() re-arms from clean). + let mru = MRUTracker() + mru.start() + mru.start() + mru.stop() + mru.start() + mru.stop() + // No observable crash/leak surface beyond "doesn't throw"; the guard is the contract. + } +} diff --git a/openspec/changes/fix-progressive-cpu-degradation/proposal.md b/openspec/changes/fix-progressive-cpu-degradation/proposal.md index 862399e..be1f523 100644 --- a/openspec/changes/fix-progressive-cpu-degradation/proposal.md +++ b/openspec/changes/fix-progressive-cpu-degradation/proposal.md @@ -25,6 +25,16 @@ The app degrades over hours of use: CPU creeps up, switcher previews refresh slo - **Dock reads throttled.** The per-mouse-move full AX walk of Dock.app (60–125 Hz near any screen edge) is cached for 80 ms; the hover model still sees every cursor sample. The `com.apple.dock` defaults handle is reused. The window-snap monitor no longer installs the per-move monitor pair it never consumed. - **Launcher graph reused.** The launcher rebuilt its entire SwiftUI hosting view on EVERY open (main-thread graph construction at trigger time, plus a dead-graph subscription leak risk). The panel stays disposable (the ghost-on-Space-switch fix is untouched); the hosting view is now built once and re-parented. +**Second sweep (five new lenses) — what the first pass missed:** + +- **A subprocess on the gesture path.** With Space-row switching on, every switcher open shelled out to `/usr/bin/defaults` on the main thread (30–100 ms, at the moment the overlay should appear). The open now reads the recognizer's cached gate. +- **Per-copy main-thread cost.** The clipboard poll hashed the whole payload byte-by-byte and fully decoded every image just to label it; a 10 MB screenshot froze gestures ~100 ms per ⌘⇧4. Bounded sample hash + header-only dimensions. +- **Standing 2 Hz taxes** (browser AX tree-walk, `cfprefsd` round-trips per raise/hover, LaunchServices lookups per cell per render in the launcher grid) are memoized or throttled. +- **Two latent crashes** (`CGWindowID(windowNumber)` on the closed Hub — every commit after the Hub had been opened once; unguarded `as!` on cross-process Accessibility data) and **one system-wide stuck state** (a sleep with three fingers down left the scroll tap swallowing all scrolling in every app) are fixed. +- **Deferred actions are tokened** (Space-settle poll, de-minimize raise, focus recovery, MC-dismiss commit, new-window single-flight), **ownership edges closed** (generation-tagged touch consumer, held observer tokens, `deinit` removals), and the **Swift 6 isolation warnings** in `KeepAwakeController` / `CursorMonitor` are resolved at the root. + +Full itemized list in `tasks.md` §3. + ## Capabilities ### Modified Capabilities diff --git a/openspec/changes/fix-progressive-cpu-degradation/specs/menubar-app-shell/spec.md b/openspec/changes/fix-progressive-cpu-degradation/specs/menubar-app-shell/spec.md index b8a1ce3..5fd766c 100644 --- a/openspec/changes/fix-progressive-cpu-degradation/specs/menubar-app-shell/spec.md +++ b/openspec/changes/fix-progressive-cpu-degradation/specs/menubar-app-shell/spec.md @@ -15,3 +15,15 @@ The process SHALL opt out of App Nap for its whole lifetime (a never-frontmost ` #### Scenario: The Mac still sleeps - **WHEN** the machine reaches its idle-sleep timeout with the app running - **THEN** system sleep proceeds normally (the opt-out covers App Nap only) + +### Requirement: No subprocess or disk I/O on the gesture path + +Opening the switcher or the launcher from a gesture SHALL NOT spawn a subprocess, block on disk I/O, or perform a full-payload computation on the main thread. State that is expensive to read (native-gesture relocation state read via `defaults`, clipboard previews backed by blob files, app icons) SHALL be read off the gesture path and cached for it. + +#### Scenario: Switcher open with Space-row switching on +- **WHEN** the user opens the switcher with the Space-row opt-in active +- **THEN** the overlay appears without a `defaults` subprocess being spawned (the relocation state comes from the gate computed when the opt-in last changed) + +#### Scenario: Launcher open after a relaunch with large clipboard entries +- **WHEN** the launcher opens with clipboard history containing entries whose payloads live in blob files +- **THEN** the band builds from cached bounded previews after the first build, without re-reading the blobs on the gesture path diff --git a/openspec/changes/fix-progressive-cpu-degradation/specs/touch-input/spec.md b/openspec/changes/fix-progressive-cpu-degradation/specs/touch-input/spec.md index 4e4b842..ea00f6f 100644 --- a/openspec/changes/fix-progressive-cpu-degradation/specs/touch-input/spec.md +++ b/openspec/changes/fix-progressive-cpu-degradation/specs/touch-input/spec.md @@ -12,6 +12,14 @@ The app SHALL own the multitouch listener's sleep/wake policy exclusively: at mo - **WHEN** the system posts its wake notifications - **THEN** only the app's own restart path re-attaches the listener; the package's internal wake handler does not start an additional device +### Requirement: Scroll consumption never outlives the touch stream + +The scroll-consuming tap's finger-count input SHALL be treated as zero whenever the touch stream has stopped or gone silent — explicitly on disable / pre-sleep / wake-restart, and implicitly once the last frame is older than a short staleness bound. A gesture interrupted by sleep (or by the multitouch stream dying) SHALL NOT leave the tap consuming scroll events. + +#### Scenario: Sleep with fingers down does not disable scrolling system-wide +- **WHEN** the machine sleeps while three fingers are on the trackpad and later wakes +- **THEN** two-finger scrolling works in every app without relaunching ThreeFingerSwitcher + ### Requirement: Consuming event taps self-heal Each consuming `CGEventTap` the app installs (scroll consume, ⌘-Tab) SHALL be re-enabled within a bounded interval after the system disables it (`tapDisabledByTimeout` / `tapDisabledByUserInput`), independently of event delivery — the in-band re-enable alone drops the first post-stall event, which presents as a gesture that "needs to wake up". Tap teardown SHALL destroy the underlying mach port deterministically. diff --git a/openspec/changes/fix-progressive-cpu-degradation/tasks.md b/openspec/changes/fix-progressive-cpu-degradation/tasks.md index 6770764..468820d 100644 --- a/openspec/changes/fix-progressive-cpu-degradation/tasks.md +++ b/openspec/changes/fix-progressive-cpu-degradation/tasks.md @@ -21,11 +21,56 @@ - [x] 2.6 `GlobalCursorMonitor.start()`: install the per-move monitor pair only when `onMove` is wired (the window-snap consumer uses only down/up). - [x] 2.7 `LauncherOverlayController`: one lazily-built `NSHostingView` re-parented across disposable panels; `hide()` detaches it before `close()`. -## 3. Verification +## 3. Second sweep — five new lenses (retain cycles, main-thread I/O, SwiftUI storms, retry/poll correctness, crash/safety) -- [x] 3.1 `swift build` clean (no new warnings in touched files). -- [x] 3.2 `swift test` — full suite green (777 tests, 0 failures; the poll-tick NSApp guard keeps `PermissionsPollingTests` deterministic). -- [ ] 3.3 On the user's stable-signed build: after a sleep/wake cycle, confirm gestures stay single-processed (idle CPU ~0%, no growth across wakes); confirm previews stay fresh during a long switcher dwell; confirm the trigger stays instant with a CPU-loaded background app. +### 3a. Main-thread cost on hot paths +- [x] `openSwitcher` read `isSpaceRowSwitchingEffective`, which shells out to `/usr/bin/defaults` (fork + waitpid, 30–100 ms) — inline in EVERY switcher open for anyone with Space-row switching on. Now reads the recognizer's cached gate. +- [x] `ClipboardMonitor`: per-copy whole-payload FNV walk through `Data`'s generic iterator + a full `NSBitmapImageRep` decode (for "Image W×H") on main → bounded head/tail/length sample hash + ImageIO header-only dimensions. `ClipboardStore.stableName` walks the raw buffer. +- [x] `AXHostProvider`: the 2 Hz breadth-first AX walk of the browser (≤400 round-trips serviced by the browser's main thread) now re-validates the remembered address field (one call) and re-walks only when it fails. +- [x] `StageManager.isEnabled`: `CFPreferencesAppSynchronize` round-trip per raise / Dock hover / hold-guard tick → 2 s TTL cache. +- [x] `WindowSnapMonitor.frame(of:)`: single-id `CGWindowList` query instead of a second full on-screen enumeration per global mouse-up. +- [x] `LauncherView` / Bands editor: `NSWorkspace.icon(forFile:)` per cell per render at gesture rate → process-wide `IconCache` (stable `NSImage` identity also lets SwiftUI skip re-rasterizing). `columns` hoisted to a static. +- [x] `FavoritesStore`: full-tree JSON re-encode per keystroke → 0.3 s coalesced save (`flushStores()` at quit; tests keep synchronous saves via `saveDelay: 0`). +- [x] `ClipboardStore.bandWindow`: `.blob` entries loaded from disk did an `open`+`read` per large entry at launcher activation → per-id memo of the bounded preview. +- [x] `FirstRunStore.stage`: defaults read + String bridge per touch frame (via `wizardOwnsGestures`) → in-memory mirror with write-through. +- [x] `LaunchService.emptyTrash` off main; `run()` / `osascript` new-window use `terminationHandler` instead of parking a GCD thread in `waitUntilExit` per script. +- [x] Dock right/left-click reads route through the same 80 ms TTL as cursor moves; `AXDockReader` reuses one `UserDefaults(suiteName:)`. +- [x] SwiftUI: Hub "Window size" slider re-solve debounced (60 Hz → ~16/s); `HubWindowInspector` groups computed once per snapshot (was twice per render, per slider tick); `HubExcludedAppsEditor` enumerates running apps on appear, not per render; `SwitcherActionMap`/`LauncherActionMap` steps are `static` (stable `ForEach` identities); launcher scroll animates only on row changes; wizard `BreathingGlowBackdrop` mounted only while active. +- [x] Timer tolerances on every non-critical poll (preview refresh, clipboard, browser host, keep-awake heartbeat); tap watchdogs added to `.common` run-loop mode; per-tick `Task` allocations in the wizard attract loop and permissions poll replaced with `assumeIsolated`. + +### 3b. Retry / poll / deferred-action correctness +- [x] `afterSpaceSettles`: generation token (rapid Space switches stacked N concurrent 16 Hz CGS polls, and a stale chain could focus the WRONG Space's window). +- [x] `raiseDeminimizing`'s deferred raise and `recover(attempt: 1)`'s async hop are `commitSeq`-guarded (a newer commit was being stomped / its watchdog tokens invalidated). +- [x] Mission-Control-dismiss commit deferral is a single cancellable slot. +- [x] `makeNewWindow` / `reopenWindowlessApp`: single-flight per pid (a double-lift opened two windows). +- [x] Hub / wizard `seedThumbnails` retry sweeps are cancellable and superseded per re-seed. +- [x] `WindowSnapMonitor.schedule` cancels the previous settle item. +- [x] `WindowFocusTracker` AX source in `.commonModes` (focus changes during menu/drag tracking were missed) — add, teardown AND `deinit` all agree on the mode (a mismatch there would leave a source pointing at freed memory). +- [x] `elementCache` soft-capped between snapshots; `bundleKeyByPid` pruned per snapshot; `BrowserContextMonitor`'s per-tick `bundleIdentifier` IPC memoized per pid. + +### 3c. Ownership / lifecycle +- [x] `TouchEngine` consumer is generation-tagged (a stale stream side can never feed the engine alongside the new one — the app-layer twin of the framework's orphaned-device bug). +- [x] `ThumbnailService` sweep task captures `self` weakly; Hub window observers are held (not discarded); `MRUTracker` / `KeyboardLanguageService` / `WindowFocusTracker` / `InputActivityMonitor` remove their observers in `deinit`. +- [x] `KeepAwakeController.onActiveChanged` is finally wired (the menu's Active/Stop line only corrected itself on the next menu open). +- [x] `ClipboardStore.save` coalesces snapshots (latest-wins) instead of queuing a full store copy per copy. + +### 3d. Crashes and stuck states +- [x] `CGWindowID(NSWindow.windowNumber)` trapped on the ≤ 0 number of the retained-but-closed Hub — on EVERY switcher commit / preview tick after the Hub had been opened once. One `hubWindowID` chokepoint + a guard in `HubSwitcherEntry.isHub`. +- [x] Unguarded `as! AXUIElement` / `as! AXValue` on cross-process AX data (traps when a misbehaving app's AX server returns another CF type) → typed `axElement` / `axValue` helpers. +- [x] `captureDimensions` traps on NaN/inf (`Swift.max(NaN, 1)` is NaN) → finite guard. +- [x] `currentFingerCount` was never reset when the touch stream stopped — sleep with three fingers down left the scroll tap swallowing EVERY scroll in every app until quit. Reset on disable / will-sleep / wake-restart, plus a 0.5 s staleness guard in the consume predicate. +- [x] `missionControlOpen` latched true when MC was closed any way but ours (stray Escape into the user's app + screen-saver-level panel on every later open) → cleared on regular-app activation, active-Space change, will-sleep, and `hideOverlay`. +- [x] `restartTouchEngineAfterWake` left both taps armed against a dead engine → `refreshRowSwitchingGate()` when availability flips. +- [x] `DockPreviewController.emptyPID` stuck (an app with no current-Space windows never got a preview again until another preview cycled) → cleared on `.idle`. +- [x] Taps re-arm on an Accessibility grant mid-session (`tapCreate` failure was discarded with nothing observing the permission). +- [x] `writeToPasteboard` no longer wipes the clipboard when it has nothing to write; `flushStores()` at quit lands coalesced favorites + clipboard writes (bounded 1 s). +- [x] `CursorMonitor` protocol is `@MainActor` (+ `onLeftUp`); `KeepAwakeController` constants are `nonisolated` — the two Swift-6 isolation warnings are gone. + +## 4. Verification + +- [x] 4.1 `swift build` clean (no new warnings in touched files). +- [x] 4.2 `swift test` — full suite green (783 tests, 0 failures; `ResourceBoundsTests` pins the cache LRU/prune and MRU eviction; the poll-tick NSApp guard keeps `PermissionsPollingTests` deterministic). +- [ ] 4.3 On the user's stable-signed build: after a sleep/wake cycle, confirm gestures stay single-processed (idle CPU ~0%, no growth across wakes); confirm previews stay fresh during a long switcher dwell; confirm the trigger stays instant with a CPU-loaded background app; confirm two-finger scroll still works after sleeping mid-gesture. ## Deferred (follow-ups, deliberately out of scope — see design.md Rejected) From 4ba631f45f35e63983fafaf4cf28a0f9531edfaf Mon Sep 17 00:00:00 2001 From: Amitay Keisar Date: Thu, 20 Aug 2026 21:50:48 +0300 Subject: [PATCH 5/5] docs(releasing): verify the app with spctl -t exec, not -t open MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit -t open against the .app (or the DMG) reports 'rejected · Insufficient Context' on current macOS even for a notarized artifact — the wrong assessment type, not a broken release. Verified against the published v2.0.0 DMG: -t exec → accepted, source=Notarized Developer ID. Story-origin: routine — a verify recipe that cried wolf on a good release. Co-Authored-By: Claude Fable 5 --- docs/RELEASING.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/docs/RELEASING.md b/docs/RELEASING.md index ec67cef..c8a1ae0 100644 --- a/docs/RELEASING.md +++ b/docs/RELEASING.md @@ -72,13 +72,17 @@ The workflow derives `CFBundleShortVersionString` from the tag (`v0.1.0` → `0. Download the DMG from the Release, then: ```bash hdiutil attach ThreeFingerSwitcher-0.1.0.dmg -spctl -a -t open --context context:primary-signed -v "/Volumes/ThreeFingerSwitcher 0.1.0/ThreeFingerSwitcher.app" +spctl -a -t exec -vv "/Volumes/ThreeFingerSwitcher 0.1.0/ThreeFingerSwitcher.app" codesign --verify --deep --strict --verbose=2 "/Volumes/ThreeFingerSwitcher 0.1.0/ThreeFingerSwitcher.app" xcrun stapler validate ThreeFingerSwitcher-0.1.0.dmg hdiutil detach "/Volumes/ThreeFingerSwitcher 0.1.0" ``` -All four should pass: Gatekeeper accepts it, the signature is valid + sealed, and the -notarization ticket is stapled (works offline). +All three should pass: Gatekeeper accepts the app (`accepted · source=Notarized Developer ID`), +the signature is valid + sealed, and the notarization ticket is stapled (works offline). + +> Use `-t exec` for the **app**. `spctl -a -t open …` against the .app (or the DMG) reports +> `rejected · source=Insufficient Context` on current macOS even for a perfectly notarized +> artifact — it's the wrong assessment type, not a broken release (verified on v2.0.0). ## Troubleshooting