|
| 1 | +# Connection parameters |
| 2 | + |
| 3 | +This is a reference for every keyword argument accepted by |
| 4 | +`databricks.sql.connect(...)` (which forwards straight into |
| 5 | +`Connection.__init__`). For each parameter it lists the type, default, |
| 6 | +which backend(s) actually consume it, and what it does. |
| 7 | + |
| 8 | +```python |
| 9 | +from databricks import sql |
| 10 | + |
| 11 | +connection = sql.connect( |
| 12 | + server_hostname="********.databricks.com", |
| 13 | + http_path="/sql/1.0/warehouses/abc123", |
| 14 | + access_token="dapi...", |
| 15 | + # ...any of the parameters below... |
| 16 | +) |
| 17 | +``` |
| 18 | + |
| 19 | +## The three backends |
| 20 | + |
| 21 | +The connector can talk to Databricks through one of three backend |
| 22 | +implementations. You pick one at connect time: |
| 23 | + |
| 24 | +| Backend | Selected by | Status | Notes | |
| 25 | +| --- | --- | --- | --- | |
| 26 | +| **Thrift** | *(default)* | GA | Thrift-over-HTTP client. Works against SQL warehouses **and** all-purpose (interactive) clusters. | |
| 27 | +| **SEA** | `use_sea=True` | Public preview | Pure-Python client for the [Statement Execution API](https://docs.databricks.com/api/workspace/statementexecution). SQL warehouses only. | |
| 28 | +| **Kernel** | `use_kernel=True` | Early access | Routes through the native Rust core (`databricks-sql-kernel`) via PyO3. Requires Python ≥ 3.10 and the `[kernel]` extra. See the "Rust kernel backend" section of the [README](../README.md). | |
| 29 | + |
| 30 | +`use_sea` and `use_kernel` are **mutually exclusive** — passing both raises |
| 31 | +`ValueError`. |
| 32 | + |
| 33 | +## How to read the backend columns |
| 34 | + |
| 35 | +The **Thrift**, **SEA**, and **Kernel** columns say whether that backend |
| 36 | +actually reads the parameter: |
| 37 | + |
| 38 | +- **✓** — consumed by that backend. |
| 39 | +- **✗** — accepted (it's just `**kwargs`) but ignored by that backend, so it |
| 40 | + has no effect. |
| 41 | +- **—** — not applicable (e.g. a backend-selection flag). |
| 42 | + |
| 43 | +> **Kernel is early access.** The session layer forwards only a *curated, |
| 44 | +> named subset* of parameters to the kernel (it does **not** splat |
| 45 | +> `**kwargs`). Any parameter marked **✗** for Kernel is silently dropped on |
| 46 | +> that path rather than raising — so it will not take effect. Result-format |
| 47 | +> and transport tuning (cloud fetch, LZ4, download threads, arrow-native |
| 48 | +> decimals/timestamps) is managed inside the kernel and is not user-tunable |
| 49 | +> from here yet. Kernel capabilities are still landing; treat its column as a |
| 50 | +> snapshot of connector `4.4.0`. |
| 51 | +
|
| 52 | +Parameters whose name begins with an underscore (e.g. `_socket_timeout`) are |
| 53 | +**internal / advanced** knobs. They are not part of the stable public API and |
| 54 | +may change without notice, but they are documented here because they are |
| 55 | +commonly used in the field. |
| 56 | + |
| 57 | +## Parameters |
| 58 | + |
| 59 | +| Parameter | Type | Default | Thrift | SEA | Kernel | Meaning | |
| 60 | +| --- | --- | --- | :---: | :---: | :---: | --- | |
| 61 | +| **Connection target** | | | | | | | |
| 62 | +| `server_hostname` | `str` | *required* | ✓ | ✓ | ✓ | Databricks workspace hostname, e.g. `dbc-12345.cloud.databricks.com`. | |
| 63 | +| `http_path` | `str` | *required* | ✓ | ✓ | ✓ | HTTP path to a SQL warehouse (`/sql/1.0/warehouses/...`) or, for Thrift only, an all-purpose cluster (`/sql/protocolv1/o/.../...`). SEA and Kernel require a warehouse/endpoint path. | |
| 64 | +| `_port` | `int` | `443` | ✓ | ✓ | ✗ | TCP port. Advanced/testing only. | |
| 65 | +| `_connection_uri` | `str` | `None` | ✓ | ✗ | ✗ | Overrides `server_hostname`/`http_path` with a full URI. Internal. | |
| 66 | +| **Authentication** | | | | | | | |
| 67 | +| `access_token` | `str` | `None` | ✓ | ✓ | ✓ | Personal Access Token / bearer token. If omitted, auth falls back to OAuth. | |
| 68 | +| `auth_type` | `str` | `None` | ✓ | ✓ | ✓ | Auth flow selector: `databricks-oauth` (U2M/M2M) or `azure-oauth` (Microsoft Entra ID). Defaults to Databricks OAuth when no token or cert is given. | |
| 69 | +| `oauth_client_id` | `str` | built-in | ✓ | ✓ | ✓ | Custom OAuth client ID. Defaults to the connector's built-in `databricks-sql-python` client. | |
| 70 | +| `oauth_redirect_port` | `int` | `None` | ✓ | ✓ | ✓ | Localhost redirect port for the U2M browser flow. Required when a custom `oauth_client_id` is set. | |
| 71 | +| `oauth_client_secret` | `str` | `None` | ✗ | ✗ | ✓ | OAuth M2M client secret. Only honored on the Kernel path today; the Thrift/SEA connector does not consume it. | |
| 72 | +| `oauth_scopes` | `List[str]` | `["sql", "offline_access"]` | ✗ | ✗ | ✓ | Custom OAuth scopes. Thrift/SEA always use the built-in scope set; only the Kernel path reads a custom value. | |
| 73 | +| `experimental_oauth_persistence` | `OAuthPersistence` | `None` | ✓ | ✓ | ✗ | Storage backend for persisting OAuth tokens across process restarts (beta). Kernel manages its own token lifecycle. | |
| 74 | +| `credentials_provider` | `CredentialsProvider` | `None` | ✓ | ✓ | ✓ | Custom credentials provider for external auth. | |
| 75 | +| `identity_federation_client_id` | `str` | `None` | ✓ | ✓ | ✓ | Token-federation (workload identity federation) client ID. | |
| 76 | +| `azure_client_id` | `str` | `None` | ✓ | ✓ | ✗ | Microsoft Entra ID (Azure AD) service-principal client/app ID (with `auth_type="azure-oauth"`). | |
| 77 | +| `azure_client_secret` | `str` | `None` | ✓ | ✓ | ✗ | Azure service-principal client secret. | |
| 78 | +| `azure_tenant_id` | `str` | `None` | ✓ | ✓ | ✗ | Azure AD tenant ID. | |
| 79 | +| `azure_workspace_resource_id` | `str` | `None` | ✓ | ✓ | ✗ | Azure workspace resource ID. | |
| 80 | +| `_use_cert_as_auth` | `bool` | `False` | ✓ | ✓ | ✗ | Authenticate with a TLS client certificate instead of a token/OAuth. Internal. | |
| 81 | +| `username` / `password` | `str` | `None` | ✗ | ✗ | ✗ | **Removed.** Basic auth is no longer supported; passing either raises `ValueError`. | |
| 82 | +| **TLS / SSL** | | | | | | | |
| 83 | +| `_tls_no_verify` | `bool` | `False` | ✓ | ✓ | ✓ | Disable all TLS verification (cert **and** hostname). Dangerous — testing only. | |
| 84 | +| `_tls_verify_hostname` | `bool` | `True` | ✓ | ✓ | ✓ | Verify the server hostname matches the certificate (cert still verified). | |
| 85 | +| `_tls_trusted_ca_file` | `str` | `None` | ✓ | ✓ | ✓ | Path to a CA bundle for server-cert verification. Defaults to the system trust store. | |
| 86 | +| `_tls_client_cert_file` | `str` | `None` | ✓ | ✓ | ✓ | Path to a client certificate (mutual TLS). | |
| 87 | +| `_tls_client_cert_key_file` | `str` | `None` | ✓ | ✓ | ✓ | Path to the client certificate's private key. | |
| 88 | +| `_tls_client_cert_key_password` | `str` | `None` | ✓ | ✓ | ✓ | Password for an encrypted client-key file. | |
| 89 | +| **Session setup** | | | | | | | |
| 90 | +| `http_headers` | `List[Tuple[str, str]]` | `None` | ✓ | ✓ | ✓ | Extra `(key, value)` HTTP headers sent on every request. | |
| 91 | +| `session_configuration` | `Dict[str, Any]` | `None` | ✓ | ✓ | ✓ | Spark/SQL session parameters (e.g. `{"ansi_mode": "true"}`). Run `SET -v` for the full list. | |
| 92 | +| `catalog` | `str` | `None` | ✓ | ✓ | ✓ | Initial catalog for the session (DBR 9.0+). | |
| 93 | +| `schema` | `str` | `None` | ✓ | ✓ | ✓ | Initial schema for the session (DBR 9.0+). | |
| 94 | +| `query_tags` | `Dict[str, Optional[str]]` | `None` | ✓ | ✓ | ✓ | Key/value tags serialized into the `QUERY_TAGS` session config. (Per-*statement* query tags are not yet supported on Kernel.) | |
| 95 | +| `enable_metric_view_metadata` | `bool` | `False` | ✓ | ✓ | ✓ | Sets `spark.sql.thriftserver.metadata.metricview.enabled` so `cursor.tables()`/`cursor.columns()` surface metric-view metadata. | |
| 96 | +| `user_agent_entry` | `str` | `None` | ✓ | ✓ | ✓ | Custom tag appended to the `User-Agent` header (used by partners to identify their app). | |
| 97 | +| `_user_agent_entry` | `str` | `None` | ✓ | ✓ | ✓ | **Deprecated** alias for `user_agent_entry`; emits a warning. | |
| 98 | +| **Backend selection** | | | | | | | |
| 99 | +| `use_sea` | `bool` | `False` | — | — | — | Route through the pure-Python SEA backend. | |
| 100 | +| `use_kernel` | `bool` | `False` | — | — | — | Route through the Rust kernel backend. Mutually exclusive with `use_sea`. | |
| 101 | +| `use_hybrid_disposition` | `bool` | `False` | ✗ | ✓ | ✗ | SEA only: use the hybrid result disposition instead of inline. | |
| 102 | +| **Result format & data types** | | | | | | | |
| 103 | +| `use_cloud_fetch` | `bool` | `True` | ✓ | ✓ | ✗ | Download large result sets in parallel from cloud storage. Kernel manages result transport internally. | |
| 104 | +| `max_download_threads` | `int` | `10` | ✓ | ✓ | ✗ | Worker threads for cloud-fetch downloads. | |
| 105 | +| `enable_query_result_lz4_compression` | `bool` | `True` | ✓ | ✓ | ✗ | LZ4-compress result payloads. | |
| 106 | +| `_disable_pandas` | `bool` | `False` | ✓ | ✓ | ✗ | Skip the pandas-based Arrow deserialization path (fallback for pandas edge cases). | |
| 107 | +| `_use_arrow_native_complex_types` | `bool` | `True` | ✓ | ✓ | ✓ | Return `ARRAY`/`MAP`/`STRUCT` as native Arrow types instead of JSON strings. | |
| 108 | +| `_use_arrow_native_decimals` | `bool` | `True` | ✓ | ✗ | ✗ | Return `DECIMAL` as a native Arrow type instead of a string. | |
| 109 | +| `_use_arrow_native_timestamps` | `bool` | `True` | ✓ | ✗ | ✗ | Return `TIMESTAMP` as a native Arrow type instead of a string. | |
| 110 | +| **Query parameters & transactions** | | | | | | | |
| 111 | +| `use_inline_params` | `bool` \| `"silent"` | `False` | ✓ | ✓ | ✗ | Render parameters inline into the SQL text (legacy) instead of native bound parameters (DBR 14.1+). `"silent"` suppresses the deprecation warning. | |
| 112 | +| `ignore_transactions` | `bool` | `True` | ✓ | ✓ | ✓ | When `True`: `commit()` is a no-op, `rollback()` raises `NotSupportedError`, and setting `autocommit` is a no-op. | |
| 113 | +| `fetch_autocommit_from_server` | `bool` | `False` | ✓ | ✓ | ✓ | Query the server (`SET AUTOCOMMIT`) for autocommit state instead of returning the cached value. | |
| 114 | +| **Volume staging (`PUT`/`GET`)** | | | | | | | |
| 115 | +| `staging_allowed_local_path` | `str` \| `List[str]` | `None` | ✓ | ✓ | ✗ | Local path(s) permitted for Unity Catalog Volume `PUT`/`GET` staging operations. Kernel has no Volume API yet. | |
| 116 | +| **Networking / connection pool** | | | | | | | |
| 117 | +| `_socket_timeout` | `float` | `900` (Thrift) | ✓ | ✗ | ✗ | Socket send/recv/connect timeout, in seconds. | |
| 118 | +| `_pool_connections` | `int` | `10` | ✓ | ✓ | ✓ | Number of urllib3 connection pools on the shared HTTP client. | |
| 119 | +| `_pool_maxsize` | `int` | `20` | ✓ | ✓ | ✓ | Max connections per pool on the shared HTTP client. | |
| 120 | +| `_proxy_auth_method` | `str` | `None` | ✓ | ✓ | ✓ | Proxy authentication scheme: `basic` or `negotiate` (Kerberos). See [`docs/proxy.md`](proxy.md). | |
| 121 | +| **Retry policy** | | | | | | | |
| 122 | +| `_retry_stop_after_attempts_count` | `int` | `30` | ✓ | ✓ | ✓ | Max attempts in a retry sequence. Bounded to `[1, 60]` on Thrift. | |
| 123 | +| `_retry_stop_after_attempts_duration` | `float` | `900` | ✓ | ✓ | ✓ | Max total wall-clock seconds spent retrying. | |
| 124 | +| `_retry_delay_min` | `float` | `1` | ✓ | ✓ | ✓ | Minimum backoff delay between retries (seconds). | |
| 125 | +| `_retry_delay_max` | `float` | `60` | ✓ | ✓ | ✓ | Maximum backoff delay between retries (seconds). | |
| 126 | +| `_retry_delay_default` | `float` | `5` | ✓ | ✓ | ✗ | Delay used when a poll fails due to a TCP/OS error. | |
| 127 | +| `_retry_dangerous_codes` | `List[int]` | `[]` | ✓ | ✓ | ✗ | HTTP status codes for which even non-idempotent commands (e.g. `ExecuteStatement`) are retried. | |
| 128 | +| `_respect_server_retry_after_header` | `bool` | `False` | ✓ | ✓ | ✗ | Honor the server's `Retry-After` header. | |
| 129 | +| `_retry_max_redirects` | `int` | `None` | ✓ | ✓ | ✗ | Max HTTP redirects to follow (must be ≤ `_retry_stop_after_attempts_count`). | |
| 130 | +| `_enable_v3_retries` | `bool` | `True` | ✓ | ✓ | ✗ | Use the urllib3-based v3 retry policy. Setting `False` selects the deprecated legacy policy. | |
| 131 | +| **Telemetry** | | | | | | | |
| 132 | +| `enable_telemetry` | `bool` | `True` | ✓ | ✓ | ✓ | Enable client telemetry collection. | |
| 133 | +| `force_enable_telemetry` | `bool` | `False` | ✓ | ✓ | ✓ | Force telemetry on regardless of the server-side feature flag. | |
| 134 | +| `telemetry_batch_size` | `int` | `100` | ✓ | ✓ | ✓ | Number of telemetry events buffered before a flush. | |
| 135 | +| `_telemetry_circuit_breaker_enabled` | `bool` | `True` | ✓ | ✓ | ✓ | Enable the telemetry circuit breaker. | |
| 136 | + |
| 137 | +## Notes |
| 138 | + |
| 139 | +- **Underscore-prefixed parameters are advanced/internal** and not part of the |
| 140 | + stable public API. They can change between releases. |
| 141 | +- **`_socket_timeout`** governs the Thrift transport socket. The SEA and |
| 142 | + Kernel HTTP layers manage their own timeouts and do not read it. (It is also |
| 143 | + passed to the auth HTTP client, which applies its own retry defaults.) |
| 144 | +- **Retry defaults for auth requests differ.** The values above are the |
| 145 | + defaults for *query* traffic (from the backend retry policy). The internal |
| 146 | + auth HTTP client reuses the same `_retry_*` kwargs but applies its own |
| 147 | + built-in defaults when they are unset. |
| 148 | +- A couple of flags are declared in the code but currently unused (`_enable_ssl`, |
| 149 | + `_skip_routing_headers`); they are omitted from the table above. |
| 150 | + |
| 151 | +*Generated against connector version `4.4.0`. When in doubt, the source of |
| 152 | +truth is `Connection.__init__` in [`src/databricks/sql/client.py`](../src/databricks/sql/client.py) |
| 153 | +and the per-backend clients under `src/databricks/sql/backend/`.* |
0 commit comments