feat!: add query_logs tool for custom log queries

[jordienr]

What

Adds a new query_logs tool to the debugging feature group. It runs a custom ClickHouse SQL query against a project's unified logs stream, for cases where the get_logs service presets are too coarse (filtering, aggregating, or joining across log fields).

Tracks AI-701. Builds on the ClickHouse logs endpoint work in O11Y-1813 and the get_logs ClickHouse migration (#326).

Breaking change

This ships as feat!: per team alignment (see PR discussion): on platforms that implement ClickHouse-backed querying (hosted/production), query_logs is the tool discoverable via tools/list, and get_logs is hidden from discovery — though it remains callable via tools/call for any client still holding an older tool list. On platforms without ClickHouse support (CLI/self-hosted), the reverse holds: get_logs stays listed and query_logs is not registered at all.

DebuggingOperations.queryLogs is optional on the SupabasePlatform type (published via the ./platform subpath export). Implementers outside this repo (CLI, studio, the hosted controller) are unaffected if they don't implement it — they simply keep get_logs as their only logs tool.

How it addresses the observability team's concerns

1. Prompt injection / security — log content is user-controllable (same class of risk as execute_sql), so the result is wrapped in wrapWithUntrustedDataBoundary, the same best-effort guardrail execute_sql uses. Not foolproof, but consistent with the existing arbitrary-query tool.
2. Cost / scalability — starts conservative: defaults to a 24h window, and both the client (resolveLogWindow) and the analytics endpoint cap the requested range at 24h. No unbounded queries.
3. ClickHouse, not BigQuery — hits GET /v1/projects/{ref}/analytics/endpoints/logs (ClickHouse / logs.all.otel) and takes raw ClickHouse-dialect SQL. Does not touch the deprecated BigQuery-backed logs.all.
4. No POST on v1 — uses GET with sql as a query param. No new POST handler.
5. CLI / self-hosted — query_logs is only registered when the platform implements queryLogs; CLI/self-hosted platforms don't, so they keep get_logs as their only logs tool and never see query_logs at all (see Breaking change above).

Details

• query_logs params: project_id, sql, optional iso_timestamp_start/iso_timestamp_end.
• Timestamps are validated as ISO 8601 with an explicit UTC Z suffix or offset (z.iso.datetime({ offset: true })), so the constraint is visible in the tool's JSON schema and offset-less (ambiguous local-time) timestamps are rejected up front.
• resolveLogWindow (shared by get_logs and query_logs) anchors the default start to 24h before the resolved end, normalizes accepted timestamps to canonical UTC ISO strings, and rejects an inverted/equal window or one exceeding 24h — unit-tested directly in debugging-tools.test.ts.
• Read-only enforcement of the SQL itself is left to the backend; no client-side SQL parsing.
• Tool descriptions carry no environment-specific wording (hosted/self-hosted/CLI) — routing is handled structurally via registration/hiding, not prose the model has to reason about.

Verification

• tsc --noEmit clean, biome ci clean.
• All unit + integration suites pass (incl. logs.test.ts, server.test.ts, debugging-tools.test.ts). The 5 failing |e2e| checks need live SUPABASE_ACCESS_TOKEN / ANTHROPIC_API_KEY, which this fork PR doesn't receive — confirmed passing on an identical-commit mirror pushed directly to the repo (feat: add query_logs tool for custom log queries (CI mirror of #333) #341).
• Manually verified query_logs end-to-end against a real staging project (ClickHouse query executed, results returned, untrusted-data wrapping applied).
• Eval coverage: feat(eval): query_logs tool selection on hosted-shaped platform evals#100 (draft, blocked on this shipping in a released @supabase/mcp-server-supabase version) verifies the agent successfully uses query_logs on a hosted-shaped fixture where it's the only logs tool listed.

Post-merge follow-ups

Per Rodriguespn — this adds a tool and changes get_logs's discoverability, so per CONTRIBUTING's breaking-change guidance:

• Release a new @supabase/mcp-server-supabase version (automatic via release-please once merged as feat!:)
• Update the tool list at supabase.com/mcp
• Bump MCP_SERVER_VERSION in supabase/evals and un-draft evals#100
• Open a PR on platform to add query_logs to the remote MCP server
• Update the @supabase/mcp-server-supabase version pin inside studio
• Resubmit the ChatGPT app to pick up the new tool and reworded get_logs description (frozen fields)

Note

Per CONTRIBUTING, feature PRs should track an accepted issue — this tracks AI-701.

[barryroodt]

Really clean addition, Jordi. It mirrors the merged get_logs sibling almost exactly (same endpoint, same untrusted-data wrapping, same 24h defaulting), and I reproduced the checks against the PR head: tsc --noEmit, biome ci ., and the server.test.ts + logs.test.ts suites (111 tests) all pass. Nothing here blocks merge. A couple of small things worth a look:

The sql description is missing function_edge_logs. The source list guides the model on what to filter by, and right now it lists function_logs (edge-function runtime console output) but not function_edge_logs (the invocation/request logs the get_logs edge-function preset queries, see logs.ts:37). Since that description is effectively the contract the model writes SQL against, a model following it can't reach invocation logs and will filter on the wrong source. Everything else in the list looks right.

query_logs has no execution test yet. The test change adds it to the tool-listing assertion, which is exactly right for registration, but the new timestamp-defaulting / passthrough / wrapping logic in execute is the bug-prone part and it's currently untested (whereas get_logs has three). server.test.ts:1567 is a ready-made template and the existing /endpoints/logs mock already covers it, so no new mock needed. Happy to leave this as a fast-follow if you'd rather not expand scope here.

Tiny nit: sql: z.string() could take .min(1) to match executeSqlOptionsSchema.query, so an empty query gives a clear validation message instead of an opaque backend error. Very much a nice-to-have.

None of this is load-bearing for shipping. The function_edge_logs line is the one I'd genuinely want fixed before merge; the rest are optional.

[jordienr]

Thanks for the thorough review! Addressed all three in f37cfc5:

• function_edge_logs missing from the sql description — added it to the source-hint list, so a model can now reach edge function invocation logs (not just function_logs runtime output).
• No execution test for query_logs — added three, mirroring the get_logs template at server.test.ts: sql passthrough + timestamp defaulting, custom-window forwarding, and empty-query rejection.
• sql nit — added .min(1) on both the tool schema and queryLogsOptionsSchema, matching executeSqlOptionsSchema.query.

tsc --noEmit, biome ci ., and the server.test.ts + logs.test.ts suites (114 tests) all pass.

🤖 Addressed by Claude Code

[barryroodt]

Thanks for the quick turnaround @jordienr

• function_edge_logs added to the source list
• .min(1) on the schema
• behavioral tests for query_logs

LGTM

@mattrossman, @Rodriguespn - since I'm still the new guy, perhaps a quick scan and thumbs-up from either of you would be advisable 😁

[pkg-pr-new]

Open in StackBlitz

pnpm add https://pkg.pr.new/@supabase/mcp-server-postgrest@caadd9d

pnpm add https://pkg.pr.new/@supabase/mcp-server-supabase@caadd9d

pnpm add https://pkg.pr.new/@supabase/mcp-utils@caadd9d

commit: caadd9d

[Ziinc]

on further thought, let's leave both tools and potentially deprecate and remove getLogs in favour of queryLogs.
Note that Clickhouse backed querying is only available for production, cli/self-hosted will not have it yet and we need to document that discrepancy as this will break if people try querying against cli. they should use the getLogs tool until full support is provided.

[jordienr]

Addressed the deprecation + platform-availability guidance in a4d7228:

• get_logs is now marked deprecated on hosted (production) projects in favour of query_logs, while remaining the documented path for local (CLI) and self-hosted projects.
• query_logs description now states it's hosted-only — ClickHouse-backed querying isn't available on CLI/self-hosted yet, and to use get_logs there — so a model won't try it against an unsupported project and break.

🤖 Addressed by Claude Code

[barryroodt]

Following up with fresh numbers against the current head (ecc456d)

Rerun of our eval A/B against the new head: passed all 3 eval checks on the first attempt, and the a4d7228 deprecation note fully flipped tool selection, 8 query_logs calls and zero get_logs (previously the model mixed both). The first genuine ClickHouse aggregation (countIf(toInt32OrZero(...))) succeeded as-is. For contrast, the get_logs-only baseline on main had one run die on context length counting raw preset rows, so the deprecation direction looks well supported.

One finding: the 24 hour default still silently widens narrow questions. The eval asks about the last 15 minutes; all 8 query_logs calls in the fresh run passed no timestamps, inheriting the 24 hour window, even though the description now mentions iso_timestamp_start/iso_timestamp_end. Mentioning the window permissively isn't landing; in production that means wrong counts whenever older logs exist.

Suggestion: same mechanism as a4d7228 - make the line directive, something like "when the user asks about a specific time range, always pass iso_timestamp_start/iso_timestamp_end to match it".

Unrelated side effect: this exercise surfaced drift in our own evals fixture, fixed in supabase/evals#99.

[jordienr]

Thanks for the fresh eval run — great to see the deprecation note flip selection cleanly, and good catch on the time-range default.

Fixed in ea6d163: the timestamp guidance is now a directive rather than a permissive mention, using the same mechanism as the deprecation line. Both query_logs and get_logs now say:

When the user asks about a specific time range, always pass iso_timestamp_start and iso_timestamp_end to match it; otherwise the query defaults to the last 24 hours and will return results from a wider window than intended.

Applied to get_logs too since it shares the identical 24h defaulting. Whenever you next rerun the A/B I'd be curious whether the 15-minute question now passes an explicit window.

🤖 Addressed by Claude Code

[barryroodt]

Nice! Reran the A/B against ea6d163 and the directive line does exactly what we hoped: the model now passes an explicit 15 minute window matching the question on every analytical query_logs call (4 of 4), and the only windowless calls are orientation probes (a now() check and a quick sample of the data), which is the behaviour you'd want. Still 3/3 checks on the first attempt, still 100% query_logs over get_logs on hosted.

[Rodriguespn]

Since this adds a tool and rewords the get_logs description, two post-merge follow-ups from CONTRIBUTING apply and are worth tracking in the PR description:

1. Updating the tool list at Supabase MCP docs
2. A ChatGPT app resubmission (tool list and descriptions are frozen fields).

Also linking the eval evidence here: the pkg.pr.new preview build of this branch passed the full supabase/evals regression suite (8/8, identical to the published-version baseline - CI run) and locally against the three debugging evals that exercise the analytics logs endpoint (evals#79, with transcripts showing agents successfully using query_logs with source = 'function_edge_logs' ClickHouse SQL.

Conditions of the run: model claude-sonnet-5 (reasoning effort high)

Thanks for taking care of this @jordienr!

[Rodriguespn]

LGTM, thanks for taking another look @jordienr. Could you please address the nit comments I left, whether implementing them or closing them. Up to you 🙁

Important

One last thing before merging, this is also a breaking change as we changed get_logs description and inputSchema.
Can you change the title of this PR to include feat!: and also make sure that the merged commit that lands on main starts with feat!: please?

Check CONTRIBUTING's expand/contract rule for more details.

[Rodriguespn]

Please check this slack message and the previous ones in that thread for more context on the step back

[Rodriguespn]

Ran the CI locally and it's green. In general LGTM, thanks for the patience @jordienr. I'm approving this PR to unblock you, but wanted to ask if we can consider implementing the logsDialect + dynamic description approach that Matt suggested.

Seems like a low-hanging fruit to roll out query_logs across all envs without having to juggle multiple conditions depending on the environment. We're not suggesting implementing ClickHouse for logs on self-hosted/CLI for this — self-hosted/CLI would keep using its existing BigQuery-dialect logs endpoint (Logflare logs.all). The idea is just to hide get_logs on all envs and change query_logs's description/behavior depending on the env's dialect, using the strategy Matt described.

Essentially, what I'm suggesting is the following:

1. Add a logsDialect to DebuggingOperations so each platform can declare which SQL its logs endpoint speaks — platform/types.ts#L220-L223:

export type DebuggingOperations = {
  logsDialect?: 'clickhouse' | 'bigquery'; // <- new
  getLogs(projectId: string, options: GetLogsOptions): Promise<unknown>;
  queryLogs?(projectId: string, options: QueryLogsOptions): Promise<unknown>;
  ...
};

2. Each platform hardcodes its dialect and implements queryLogs. Hosted/platform-API already implements queryLogs (ClickHouse) — just add the dialect at platform/api-platform.ts#L255:

const debugging: DebuggingOperations = {
  logsDialect: 'clickhouse', // <- new
  ...
};

Self-hosted/CLI sets logsDialect: 'bigquery' and implements queryLogs by passing the model's SQL straight through to the same BigQuery-backed analytics endpoint getLogs already uses (the endpoint takes an arbitrary sql param), in supabase/apps/studio/lib/api/self-hosted/mcp.ts getDebuggingOperations:

return {
  logsDialect: 'bigquery', // <- new
  async getLogs(projectRef, options) { /* unchanged: getLogQuery(service) */ },
  async queryLogs(projectRef, options) { // <- new, mirrors getLogs
    const { data, error } = await retrieveAnalyticsData({
      name: 'logs.all',
      projectRef,
      params: {
        sql: options.sql, // pass the model's SQL instead of a preset
        iso_timestamp_start: options.iso_timestamp_start,
        iso_timestamp_end: options.iso_timestamp_end,
      },
    })
    if (error) throw error
    return data
  },
  ...
}

3. query_logs's description switches on the dialect — no string juggling, just pick the whole description. When the dialect is bigquery, reuse the existing get_logs description; when it's clickhouse, use the one this PR already writes for query_logs. In getDebuggingTools, at the query_logs registration:

query_logs: injectableTool({
  ...debuggingToolDefs.query_logs,
  description:
    debugging.logsDialect === 'bigquery'
      ? debuggingToolDefs.get_logs.description
      : debuggingToolDefs.query_logs.description,
  inject: { project_id },
  execute: async (...) => { ... },
}),

The same whole-string switch applies to any other dialect-gated copy (e.g. the sql param hint, which today hardcodes the ClickHouse logs table / log_attributes[...] schema).

4. The gating you already wrote does the rest. hidden: Boolean(queryLogs) stays exactly as-is — once every platform implements queryLogs, get_logs is hidden everywhere and query_logs is the single listed logs tool on all envs, each with a dialect-appropriate description. No new environment conditionals beyond the one logsDialect field.

Not blocking on this, happy to ship #333 as-is to keep the ball rolling. Just flagging it now since it looks like a small, self-contained change that removes the hosted-vs-self-hosted split entirely. Wdyt?