feat(ai): answer inline suggestions and query actions with Apple Intelligence by datlechin · Pull Request #3749 · TableProApp/TablePro

datlechin · 2026-10-21T08:02:38Z

Summary

Settings > AI gets an Apple Intelligence section with Use for inline suggestions and query actions, off by default.
On, inline suggestions and Review, Explain, Optimize and Fix run on the macOS on-device model (FoundationModels, SystemLanguageModel.default): no key, no server, no network request. The chat and Agent mode keep the active provider (AI Chat: Apple Intelligence transport via Foundation Models #1048).
The switch is dimmed with the reason below macOS 26, on an ineligible Mac, with Apple Intelligence off, or while the model downloads. A choice already on can still be switched off. macOS 13 stays the minimum; FoundationModels loads weak.
Every request is measured before generation: tokenCount(for:) on macOS 26.4+, a high-side estimate below it. An action tries the full table structure, then only the columns the statement names, matched in the connection's quoting. Neither fits: nothing is generated, the error says so and Send to answers the same turn through the active provider. The reply is capped at what is left of the context.
Inline suggestions try with the schema, then without, and are dropped when neither fits.
On-device replies are plain messages labelled Apple Intelligence, without the walkthrough. Regenerate and Retry of one stay on the device, also after the conversation is restored, and a Copilot conversation is rebuilt around the local turn.
Query actions need no provider with Apple Intelligence on, and are dimmed with the reason while it cannot run. The chat's error banner now shows without an active provider too.
Never and Ask Each Time gate on-device requests as they gate provider requests, and the policy is read again after measuring. New for a user: the four actions and inline suggestions work with no provider configured. Nothing new reaches the network.
apple/scripts/probes/check-on-device-model.sh runs the shipped on-device code against the Mac's model and prints the token ratios the estimate rests on.

Test plan

AIChatViewModelOnDeviceTests: an action answered on-device with no provider; too large refused before generation with the fallback offered; the fallback and a follow-up take the provider path; Regenerate, also of a restored answer, stays on-device; unavailable explains itself; Ask Each Time asks first; the setting off takes the provider path.
AppleIntelligenceInlineSourceTests (incl. a policy turned to Never while measuring), OnDeviceFitTests, OnDeviceSnapshotTests (grapheme-extending snapshots), QueryContextUsedColumnsTests, AppleIntelligenceAvailabilityTests, new cases in AIQueryActionAvailabilityTests, InlineSuggestionSourceKindTests and AISettingsTests. All fail on main: the types do not exist.
check-on-device-model.sh on macOS 27.0.1: available, context 8192, SQL 2.74 / table 2.87 / CJK 2.17 characters a token, oversized request refused, reply streamed.
otool -l on the Debug app and on a macOS 13 target build: FoundationModels is LC_LOAD_WEAK_DYLIB.

Docs: TableProApp/docs#100

Fixes #3611

…lligence

…ext high

…story whole around on-device turns

…t a provider

J2TeamNNL

Code review: 8 findings, inline.

J2TeamNNL · 2026-10-21T09:26:22Z

+                    for try await snapshot in session.streamResponse(to: request.prompt, options: options) {
+                        let content = snapshot.content
+                        guard let delta = OnDeviceSnapshot.delta(from: emitted, to: content) else {
+                            Self.logger.fault("On-device snapshot did not extend the text already shown")


When a streamed snapshot is not an extension of emitted, this logs a fault and continues without updating emitted. Every later snapshot is compared against the stale prefix, so the reply freezes at the pre-divergence text and the stream finishes normally. If FoundationModels revises earlier text mid-stream (retracted partial token, whitespace normalisation), the user sees a silently truncated answer with no error.

J2TeamNNL · 2026-10-21T09:26:22Z

+        self.omittedColumnCount = omittedColumnCount
+    }
+
+    func keepingColumns(named used: Set<String>) -> QueryContextTableStructure {


keepingColumns(named:) drops every column the statement does not name, so SELECT * keeps zero columns. For Explain/Review/Optimize on SELECT * FROM big_table with an oversized structure, the trimmed snapshot lists no columns and says none are named in the statement, which is misleading since * covers all. Keep all columns or refuse for wildcards. The match is also exact on lowercased tokens, so a quoted/backticked identifier the scanner does not unquote loses its column too.

J2TeamNNL · 2026-10-21T09:26:22Z

+        let wasLast = lastAssistantIndex == messages.count - 1
        messages.remove(at: lastAssistantIndex)
+        if wasLast, answeredOnDevice, let asked = messages.last, asked.role == .user {
+            onDeviceTurnID = asked.id


Regenerating the last on-device answer re-arms onDeviceTurnID from the old turn's providerId, ignoring the current useAppleIntelligence setting and availability. If the user turned Apple Intelligence off and configured a provider, Regenerate still routes to the on-device model, and fails if it is now unavailable. Same for a reloaded persisted conversation.

J2TeamNNL · 2026-10-21T09:26:22Z

+                format: String(localized: "Apple Intelligence could not answer: %@"),
+                error.localizedDescription
+            )
+            streamingState = .failed(nil)


On a generation error the partial assistant turn is finalized with .failed(nil) but the conversation is not persisted. The transcript keeps a truncated answer beside the error banner, the saved conversation differs from the visible one, and Regenerate treats the truncated reply as a valid last assistant turn (empty-turn cleanup only runs when there are no blocks).

J2TeamNNL · 2026-10-21T09:26:22Z

-        guard settings.enabled, settings.inlineSuggestionsEnabled, let active = settings.activeProvider else {
-            return .off
+        guard settings.enabled, settings.inlineSuggestionsEnabled else { return .off }
+        if settings.useAppleIntelligence {


With useAppleIntelligence on, resolve returns .appleIntelligence unconditionally and never falls back to the active provider. If the on-device model is unavailable (still downloading, or Apple Intelligence off in System Settings), requestSuggestion returns nil on every keystroke: inline suggestions go silently off even though a working provider exists.

J2TeamNNL · 2026-10-21T09:26:22Z

+        if settings.includeSchema, let provider = schemaProvider,
+           let schema = await provider.buildSchemaContextForAI(settings: settings), !schema.isEmpty {
+            candidates.append(OnDeviceRequest(
+                instructions: AIPromptTemplates.inlineSuggestSystemPrompt(schemaContext: schema),


The on-device inline prompt omits the connection's AI rules and database dialect that the chat-completion inline source supplies; only the generic inlineSuggestSystemPrompt is used. MySQL/Postgres connections with per-connection aiRules, or non-SQL engines, get generic SQL suggestions, unlike the provider path.

J2TeamNNL · 2026-10-21T09:26:22Z

+        return messages.last?.id == onDeviceTurnID
+    }
+
+    var fallbackProviderName: String? {


fallbackProviderName re-implements provider resolution (selected provider id, else active) instead of reusing AIProviderFactory.resolve. They can disagree (missing API key, copilot provider, override model), so "Send to X" can name a provider that startStreaming then rejects with "No AI provider configured".

J2TeamNNL · 2026-10-21T09:26:22Z

+
+        let fit = await model.fit(candidates, reservingForReply: Self.onDeviceReplyReserve)
+        guard !Task.isCancelled else { return }
+        if connection != nil, resolveConnectionPolicy(settings: services.appSettings.ai) == .never {


The connection-policy .never check runs only after the DB metadata fetch and token measurement; buildQueryContext already queried the connection's schema by then. Check the policy before the context build so a Never connection is not queried.

J2TeamNNL · 2026-10-21T09:26:22Z

+    // MARK: - Apple Intelligence
+
+    private var appleIntelligenceSection: some View {
+        let availability = SystemOnDeviceLanguageModel().availability


appleIntelligenceSection builds a new SystemOnDeviceLanguageModel() and reads availability on every body evaluation (the same pattern recurs in MainContentCoordinator+AIQueryActions on each toolbar refresh). That is repeated system calls on hot UI paths, and availability is neither cached nor observed, so the UI can show stale state after the user enables Apple Intelligence.

J2TeamNNL · 2026-10-21T09:29:50Z

Follow-up to the review above. Fixes are in J2TeamNNL@d46aa69

Fixed:

Dropped columns on SELECT *: a wildcard select item now keeps every column (count(*) still trims). Test added.
Stream freeze on a diverging snapshot: emitted now follows the new content, so later deltas still flow.
Regenerate ignoring the setting: the on-device turn is re-armed only while Apple Intelligence is on.
Partial turn not persisted after an error: the conversation is saved in the failure path.
Inline suggestions off when the on-device model is unavailable: resolve falls back to the active provider. Test added.
Connection policy checked late: the .never check also runs before the context is built.

Skipped:

Inline prompt missing AI rules and dialect: the existing chat-completion inline source also uses only inlineSuggestSystemPrompt, so the paths match.
fallbackProviderName vs AIProviderFactory.resolve: both pick the selected provider, else the active one. resolve also loads the keychain key, which a button label should not do.
Availability read in AISettingsView: a cheap in-process property read, and no measured cost.

Not built or linted locally (no Xcode build; swiftlint cannot load sourcekit here). CI will check.

Cherry-pick: git fetch https://github.com/J2TeamNNL/TablePro review-fix/3749 && git cherry-pick d46aa693723dd046de861be571cde61ec9f90f31

datlechin added 4 commits October 21, 2026 14:26

feat(ai): answer inline suggestions and query actions with Apple Inte…

7491f28

…lligence

fix(ai): split on-device snapshots by scalar and estimate non-ASCII t…

aa776f8

…ext high

fix(ai): trim columns in the connection's quoting and keep Copilot hi…

8d3f3d5

…story whole around on-device turns

docs(changelog): add the Apple Intelligence fragment

41166b6

datlechin mentioned this pull request Oct 21, 2026

docs(features): document Apple Intelligence for inline suggestions and query actions TableProApp/docs#100

Merged

fix(ai): name the engine to the on-device model and offer Stop withou…

30f9ca6

…t a provider

datlechin enabled auto-merge (squash) October 21, 2026 08:49

J2TeamNNL reviewed Oct 21, 2026

View reviewed changes

datlechin merged commit 1dd5863 into main Oct 21, 2026
30 checks passed

datlechin deleted the feat/3611-apple-on-device-model branch October 21, 2026 11:13

Navigation Menu

Repository navigation

feat(ai): answer inline suggestions and query actions with Apple Intelligence - #3749

feat(ai): answer inline suggestions and query actions with Apple Intelligence#3749
datlechin merged 5 commits into
mainfrom
feat/3611-apple-on-device-model

datlechin commented Oct 21, 2026 •

edited

Loading

Uh oh!

J2TeamNNL left a comment

Uh oh!

J2TeamNNL Oct 21, 2026

Uh oh!

J2TeamNNL Oct 21, 2026

Uh oh!

J2TeamNNL Oct 21, 2026

Uh oh!

J2TeamNNL Oct 21, 2026

Uh oh!

J2TeamNNL Oct 21, 2026

Uh oh!

J2TeamNNL Oct 21, 2026

Uh oh!

J2TeamNNL Oct 21, 2026

Uh oh!

J2TeamNNL Oct 21, 2026

Uh oh!

J2TeamNNL Oct 21, 2026

Uh oh!

J2TeamNNL commented Oct 21, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

datlechin commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

J2TeamNNL left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

J2TeamNNL commented Oct 21, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

datlechin commented Oct 21, 2026 •

edited

Loading