Skip to content

docs: add connection parameter reference (Thrift vs Kernel) - #913

Merged
eric-wang-1990 merged 7 commits into
mainfrom
docs/connection-parameters-reference
Aug 18, 2026
Merged

docs: add connection parameter reference (Thrift vs Kernel)#913
eric-wang-1990 merged 7 commits into
mainfrom
docs/connection-parameters-reference

Conversation

@eric-wang-1990

@eric-wang-1990 eric-wang-1990 commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

What

Adds docs/connection-parameters.md — a single reference listing every public connection / session parameter databricks.sql.connect(...) accepts, and whether each is honored on the Thrift backend (default) or the Kernel backend (use_kernel=True). Linked from the README.

Format

Modeled on the databricks-sql-nodejs connection-parameter reference (#457): a per-section table with columns Option · Type · Thrift · Kernel · Default Value · Note, a ✅/❌/⚠️/— legend, a "Sources of truth" list, and a "Summary of gaps." Grouped by concern: connection identity · authentication · HTTP/proxy/retries · TLS/SSL · results & type rendering · session defaults & transactions · telemetry.

SEA column dropped

Earlier revision had a third SEA column. Removed — the pure-Python SEA backend (use_sea=True) is being deprecated, so it's intentionally omitted; the reference now compares Thrift vs Kernel only.

How support was determined

Every ✅/❌/⚠️ verified against source, not guessed:

  • Connection.__init__ and Session._create_backend — the kernel path receives only a curated, named subset of kwargs (no **kwargs splat), so unsupported params are silently ignored there; called out explicitly.
  • ThriftDatabricksClient, KernelDatabricksClient + auth_bridge.py, and the shared UnifiedHttpClient / build_client_context / auth layer.

Notable gaps documented:

  • Kernel-ignored: oauth_client_secret (M2M) & custom oauth_scopes, experimental_oauth_persistence, Azure SP OAuth, _use_cert_as_auth, cloud-fetch / LZ4 / _disable_pandas, arrow-native decimals/timestamps, staging_allowed_local_path, most retry fine-tuning, _socket_timeout / _port / _connection_uri.
  • identity_federation_client_id marked Kernel-supported (matches feat(kernel): forward identity federation client ID #910 on main).
  • oauth_client_secret / custom oauth_scopes are effectively Kernel-only — the Thrift auth path never reads them.
  • Retry defaults reflect the real _retry_policy table (count 30).

Generated against connector version 4.4.0. Docs-only — no code or behavior changed.

This pull request and its description were written by Isaac.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new documentation reference that enumerates databricks.sql.connect(...) / Connection.__init__ keyword arguments, including types, defaults, and which of the three backends (Thrift / SEA / Kernel) actually consume each parameter.

Changes:

  • Link the new connection-parameter reference from the README’s Documentation section.
  • Add docs/connection-parameters.md with backend-selection explanation and a consolidated parameter table.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
README.md Adds a direct link to the new connection-parameters reference under Documentation.
docs/connection-parameters.md New reference doc enumerating connect kwargs with types/defaults and per-backend consumption.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread docs/connection-parameters.md Outdated

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No issues identified by the review bot.

@eric-wang-1990 eric-wang-1990 changed the title docs: add connection parameters reference (Thrift/SEA/Kernel) docs: add connection parameter reference (Thrift vs Kernel) Aug 17, 2026
Add docs/connection-parameters.md documenting every sql.connect() keyword
argument: type, default, per-backend support (Thrift / SEA / Kernel), and
meaning. Verified against Connection.__init__, the per-backend clients,
and the shared HTTP/auth layer at connector 4.4.0. Linked from the README
Documentation section.

Co-authored-by: Isaac
Signed-off-by: eric-wang-1990 <e.wang@databricks.com>
… format

Drop the SEA column (backend is being deprecated) and restructure the doc to
match the databricks-sql-nodejs connection-parameter reference (#457): a
two-backend Thrift/Kernel table per section with a ✅/❌/⚠️/— legend, a
'Sources of truth' list, and a 'Summary of gaps'.

Co-authored-by: Isaac
Signed-off-by: eric-wang-1990 <e.wang@databricks.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Solid, accurate docs-only PR. I spot-checked the surprising claims against source — Thrift ignoring oauth_client_secret and custom oauth_scopes, the curated kernel-forwarded subset in Session._create_backend, and the Thrift retry/socket defaults — and they all hold. One low-severity nuance: the single "Default Value" column reflects Thrift defaults, but for kernel-✅ retry rows unset values are forwarded as None, so the effective kernel default may differ.

Comment thread CONNECTION_PARAMETERS.md Outdated

@vuanhphung vuanhphung left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there instructions to keep this in sync as we develop new features? (or we rely on agents to do this automatically)

- credentials_provider is rejected on the kernel path (NotSupportedError),
  not supported: mark Kernel ❌.
- Per-statement query_tags IS supported on the kernel via set_query_tags:
  drop the stale 'not supported' caveat.
- azure_* fields are the Entra M2M gap; note azure-oauth U2M browser flow
  does work on the kernel.
- Fix a backwards Summary-of-gaps entry (oauth_client_secret/oauth_scopes are
  kernel-supported; the real Thrift-only auth gap is credentials_provider).

Co-authored-by: Isaac
Signed-off-by: eric-wang-1990 <e.wang@databricks.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Docs-only PR; the per-backend ✅/❌/⚠️ claims I spot-checked (Thrift ignoring oauth_client_secret/custom oauth_scopes, kernel rejecting credentials_provider with NotSupportedError, the kernel-forwarded retry/auth subset) all match source. One minor wording inaccuracy on oauth_redirect_port being "required" — otherwise accurate and well-sourced.

Comment thread CONNECTION_PARAMETERS.md
@eric-wang-1990 eric-wang-1990 added the engineer-bot Maintainer-applied gate: triggers engineer-bot (bug-fix on issue / take-over on PR). label Aug 18, 2026
Addresses:
  - #3799244504 at docs/connection-parameters.md:6
  - #3799383910 at docs/connection-parameters.md:90
  - #3799696042 at docs/connection-parameters.md:72

Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No issues identified by the review bot.

The oauth_redirect_port row implied the kernel path treats the port
differently from Thrift. It doesn't: auth.py:127-130 (Thrift) and the
kernel auth_bridge use the identical gate — an explicit port is only
honored when a custom oauth_client_id is also supplied; otherwise the
full 8020-8024 range is used with first-free-port binding. Reword the
note to describe this shared behavior (both backends), keeping the
existing checkmark/checkmark support columns.

Co-authored-by: Isaac
Signed-off-by: eric-wang-1990 <e.wang@databricks.com>
Match the databricks-sql-go layout (databricks/databricks-sql-go#445):
put the reference at the repo root as CONNECTION_PARAMETERS.md and link
it from README's docs section, instead of docs/connection-parameters.md.
Retitle to the plural 'Connection parameters reference' and fix the
relative proxy link (proxy.md -> docs/proxy.md) now that the file sits
one level up.

Co-authored-by: Isaac
Signed-off-by: eric-wang-1990 <e.wang@databricks.com>
Plural title (matches databricks-sql-go) and repoint the in-table proxy
link proxy.md to docs/proxy.md now that the file lives at the repo root.
Also update the README reference path.

Co-authored-by: Isaac
Signed-off-by: eric-wang-1990 <e.wang@databricks.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No issues identified by the review bot.

@eric-wang-1990
eric-wang-1990 added this pull request to the merge queue Aug 18, 2026
Merged via the queue into main with commit 0a8f1d2 Aug 18, 2026
49 of 50 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-assisted engineer-bot Maintainer-applied gate: triggers engineer-bot (bug-fix on issue / take-over on PR).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants