From 6b0457caa6cd1467e72a3bffbd171e3cf22914a4 Mon Sep 17 00:00:00 2001 From: Piyush Dubey Date: Sat, 13 Sep 2025 00:01:21 -0400 Subject: [PATCH 01/17] Docs: Add REST catalog authentication properties (addresses - #13209) --- docs/docs/configuration.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/docs/docs/configuration.md b/docs/docs/configuration.md index 433da6ad0ddf..a1979dc3932b 100644 --- a/docs/docs/configuration.md +++ b/docs/docs/configuration.md @@ -143,6 +143,23 @@ The properties can be manually constructed or passed in from a compute engine li Spark uses its session properties as catalog properties, see more details in the [Spark configuration](spark-configuration.md#catalog-configuration) section. Flink passes in catalog properties through `CREATE CATALOG` statement, see more details in the [Flink](flink.md#adding-catalogs) section. +### Catalog REST auth properties + +The following catalog properties configure authentication for the REST catalog. +They support Basic, OAuth2, and SigV4 authentication, in addition to the default none. + +| Property | Default | Description | +| --------------------------------- | ------------------ | ------------------------------------------------------ | +| rest.auth.type | none | Authentication mechanism for REST catalog access. Supported values: `none`, `basic`, `oauth2`, `sigv4`. | +| rest.auth.basic.username | null | Username for Basic authentication. Required if `rest.auth.type` = `basic`. | +| rest.auth.basic.password | null | Password for Basic authentication. Required if `rest.auth.type` is `basic`. | +| rest.auth.credential | null | Credential string (client_id:client_secret) exchanged for a token in the OAuth2 client-credentials flow. Required if `rest.auth.type` = `oauth2`. | +| rest.auth.oauth2-server-uri | null | OAuth2 token endpoint URI. Required if the REST catalog is not the OAuth2 authentication server. Required if `rest.auth.type` = `oauth2`. | +| rest.auth.token-expires-in-ms | 3600000 (1 hour) | Time in milliseconds after which a bearer token is considered expired. Used to decide when to refresh or re-exchange a token. | +| rest.auth.token-refresh-enabled | true | Determines whether tokens are automatically refreshed when expiration details are available. | +| rest.auth.scope | catalog | Additional scope for OAuth2. | + + ### Lock catalog properties Here are the catalog properties related to locking. They are used by some catalog implementations to control the locking behavior during commits. From 6699e0cc666eeae14a4d12657376c68ed9317a52 Mon Sep 17 00:00:00 2001 From: Piyush Dubey Date: Mon, 15 Sep 2025 21:07:21 -0400 Subject: [PATCH 02/17] Docs: Add REST catalog authentication properties. Add Google Auth Type (addresses - #13209) --- docs/docs/configuration.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/docs/configuration.md b/docs/docs/configuration.md index a1979dc3932b..4fbdf63a38b4 100644 --- a/docs/docs/configuration.md +++ b/docs/docs/configuration.md @@ -146,18 +146,18 @@ Flink passes in catalog properties through `CREATE CATALOG` statement, see more ### Catalog REST auth properties The following catalog properties configure authentication for the REST catalog. -They support Basic, OAuth2, and SigV4 authentication, in addition to the default none. +They support Basic, OAuth2, SigV4, and Google authentication, in addition to the default none. -| Property | Default | Description | -| --------------------------------- | ------------------ | ------------------------------------------------------ | -| rest.auth.type | none | Authentication mechanism for REST catalog access. Supported values: `none`, `basic`, `oauth2`, `sigv4`. | -| rest.auth.basic.username | null | Username for Basic authentication. Required if `rest.auth.type` = `basic`. | -| rest.auth.basic.password | null | Password for Basic authentication. Required if `rest.auth.type` is `basic`. | +| Property | Default | Description | +| --------------------------------- | ------------------ |---------------------------------------------------------------------------------------------------------------------------------------------------| +| rest.auth.type | none | Authentication mechanism for REST catalog access. Supported values: `none`, `basic`, `oauth2`, `sigv4`, `google`. | +| rest.auth.basic.username | null | Username for Basic authentication. Required if `rest.auth.type` = `basic`. | +| rest.auth.basic.password | null | Password for Basic authentication. Required if `rest.auth.type` is `basic`. | | rest.auth.credential | null | Credential string (client_id:client_secret) exchanged for a token in the OAuth2 client-credentials flow. Required if `rest.auth.type` = `oauth2`. | -| rest.auth.oauth2-server-uri | null | OAuth2 token endpoint URI. Required if the REST catalog is not the OAuth2 authentication server. Required if `rest.auth.type` = `oauth2`. | -| rest.auth.token-expires-in-ms | 3600000 (1 hour) | Time in milliseconds after which a bearer token is considered expired. Used to decide when to refresh or re-exchange a token. | -| rest.auth.token-refresh-enabled | true | Determines whether tokens are automatically refreshed when expiration details are available. | -| rest.auth.scope | catalog | Additional scope for OAuth2. | +| rest.auth.oauth2-server-uri | null | OAuth2 token endpoint URI. Required if the REST catalog is not the OAuth2 authentication server. Required if `rest.auth.type` = `oauth2`. | +| rest.auth.token-expires-in-ms | 3600000 (1 hour) | Time in milliseconds after which a bearer token is considered expired. Used to decide when to refresh or re-exchange a token. | +| rest.auth.token-refresh-enabled | true | Determines whether tokens are automatically refreshed when expiration details are available. | +| rest.auth.scope | catalog | Additional scope for OAuth2. | ### Lock catalog properties From 3a381a59e15facc0fc729ce7f55eafd10e864618 Mon Sep 17 00:00:00 2001 From: Piyush Dubey Date: Mon, 22 Sep 2025 15:46:39 -0400 Subject: [PATCH 03/17] Fix param names and add missing oauth2 params --- docs/docs/configuration.md | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/docs/docs/configuration.md b/docs/docs/configuration.md index 4fbdf63a38b4..ea94779a3cff 100644 --- a/docs/docs/configuration.md +++ b/docs/docs/configuration.md @@ -148,16 +148,23 @@ Flink passes in catalog properties through `CREATE CATALOG` statement, see more The following catalog properties configure authentication for the REST catalog. They support Basic, OAuth2, SigV4, and Google authentication, in addition to the default none. -| Property | Default | Description | -| --------------------------------- | ------------------ |---------------------------------------------------------------------------------------------------------------------------------------------------| -| rest.auth.type | none | Authentication mechanism for REST catalog access. Supported values: `none`, `basic`, `oauth2`, `sigv4`, `google`. | -| rest.auth.basic.username | null | Username for Basic authentication. Required if `rest.auth.type` = `basic`. | -| rest.auth.basic.password | null | Password for Basic authentication. Required if `rest.auth.type` is `basic`. | -| rest.auth.credential | null | Credential string (client_id:client_secret) exchanged for a token in the OAuth2 client-credentials flow. Required if `rest.auth.type` = `oauth2`. | -| rest.auth.oauth2-server-uri | null | OAuth2 token endpoint URI. Required if the REST catalog is not the OAuth2 authentication server. Required if `rest.auth.type` = `oauth2`. | -| rest.auth.token-expires-in-ms | 3600000 (1 hour) | Time in milliseconds after which a bearer token is considered expired. Used to decide when to refresh or re-exchange a token. | -| rest.auth.token-refresh-enabled | true | Determines whether tokens are automatically refreshed when expiration details are available. | -| rest.auth.scope | catalog | Additional scope for OAuth2. | +### REST auth properties (from `AuthProperties`) + +| Property | Default | Description | +|--------------------------------------|------------------|---------------------------------------------------------------------------------------------------------------------------------------------------| +| `rest.auth.type` | none | Authentication mechanism for REST catalog access. Supported values: `none`, `basic`, `oauth2`, `sigv4`, `google`. | +| `rest.auth.basic.username` | null | Username for Basic authentication. Required if `rest.auth.type` = `basic`. | +| `rest.auth.basic.password` | null | Password for Basic authentication. Required if `rest.auth.type` is `basic`. | +| `rest.auth.credential` | null | Credential string (client_id:client_secret) exchanged for a token in the OAuth2 client-credentials flow. Required if `rest.auth.type` = `oauth2`. | +| `rest.auth.sigv4.delegate-auth-type` | `oauth2` | Auth type to delegate to after SigV4 signing. Defaults to `oauth2`. | +| `oauth2-server-uri` | null | OAuth2 token endpoint URI. Required if the REST catalog is not the OAuth2 authentication server. Required if `rest.auth.type` = `oauth2`. | +| `token-expires-in-ms` | 3600000 (1 hour) | Time in milliseconds after which a bearer token is considered expired. Used to decide when to refresh or re-exchange a token. | +| `token-refresh-enabled` | true | Determines whether tokens are automatically refreshed when expiration details are available. | +| `token-exchange-enabled` | true | Determines whether to exchange tokens to acquire new tokens. | +| `scope` | null | Additional scope for `oauth2`. | +| `audience` | null | Optional param `audience` for `oauth2`. | +| `resource` | null | Optional param `resource` for `oauth2`. | +| `catalog` | null | Scope string(s) used when requesting `oauth2` tokens for the Iceberg REST catalog. | ### Lock catalog properties From 531d5ec93c4a595a46d08718dfdf27a7136ef0e2 Mon Sep 17 00:00:00 2001 From: "piyushdubey@microsoft.com" Date: Tue, 23 Sep 2025 13:59:27 -0400 Subject: [PATCH 04/17] Add Google Auth and separate auth mechanisms into separate tables --- docs/docs/configuration.md | 35 +++++++++++++++++++++++++---------- 1 file changed, 25 insertions(+), 10 deletions(-) diff --git a/docs/docs/configuration.md b/docs/docs/configuration.md index ea94779a3cff..01596525b5fc 100644 --- a/docs/docs/configuration.md +++ b/docs/docs/configuration.md @@ -148,23 +148,38 @@ Flink passes in catalog properties through `CREATE CATALOG` statement, see more The following catalog properties configure authentication for the REST catalog. They support Basic, OAuth2, SigV4, and Google authentication, in addition to the default none. -### REST auth properties (from `AuthProperties`) +### REST auth properties | Property | Default | Description | |--------------------------------------|------------------|---------------------------------------------------------------------------------------------------------------------------------------------------| | `rest.auth.type` | none | Authentication mechanism for REST catalog access. Supported values: `none`, `basic`, `oauth2`, `sigv4`, `google`. | | `rest.auth.basic.username` | null | Username for Basic authentication. Required if `rest.auth.type` = `basic`. | | `rest.auth.basic.password` | null | Password for Basic authentication. Required if `rest.auth.type` is `basic`. | -| `rest.auth.credential` | null | Credential string (client_id:client_secret) exchanged for a token in the OAuth2 client-credentials flow. Required if `rest.auth.type` = `oauth2`. | | `rest.auth.sigv4.delegate-auth-type` | `oauth2` | Auth type to delegate to after SigV4 signing. Defaults to `oauth2`. | -| `oauth2-server-uri` | null | OAuth2 token endpoint URI. Required if the REST catalog is not the OAuth2 authentication server. Required if `rest.auth.type` = `oauth2`. | -| `token-expires-in-ms` | 3600000 (1 hour) | Time in milliseconds after which a bearer token is considered expired. Used to decide when to refresh or re-exchange a token. | -| `token-refresh-enabled` | true | Determines whether tokens are automatically refreshed when expiration details are available. | -| `token-exchange-enabled` | true | Determines whether to exchange tokens to acquire new tokens. | -| `scope` | null | Additional scope for `oauth2`. | -| `audience` | null | Optional param `audience` for `oauth2`. | -| `resource` | null | Optional param `resource` for `oauth2`. | -| `catalog` | null | Scope string(s) used when requesting `oauth2` tokens for the Iceberg REST catalog. | + +### OAuth2 Properties +Required and Optional properties to include while using OAuth2 authentication + +| Property | Default | Description | +|-------------------------|------------------|-------------------------------------------------------------------------------------------------------------------------------------------------| +| `token` | null | A Bearer token to interact with the server. Required if `rest.auth.type` = `oauth2`. | +| `credential` | null | Credential string (client_id:client_secret) to exchange a token in the OAuth2 client credentials flow. Required if `rest.auth.type` = `oauth2`. | +| `oauth2-server-uri` | null | OAuth2 token endpoint URI. Required if the REST catalog is not the OAuth2 authentication server. Required if `rest.auth.type` = `oauth2`. | +| `token-expires-in-ms` | 3600000 (1 hour) | Time in milliseconds after which a bearer token is considered expired. Used to decide when to refresh or re-exchange a token. | +| `token-refresh-enabled` | true | Determines whether tokens are automatically refreshed when expiration details are available. | +| `token-exchange-enabled`| true | Determines whether to exchange tokens to acquire new tokens. | +| `scope` | null | Additional scope for `oauth2`. | +| `audience` | null | Optional param `audience` for `oauth2`. | +| `resource` | null | Optional param `resource` for `oauth2`. | +| `catalog` | null | Scope string(s) used when requesting `oauth2` tokens for the Iceberg REST catalog. | + +### Google auth properties +Required and Optional properties to include while using Google authentication + +| Property | Default | Description | +|------------------------------|------------------|---------------------------------------------------------------------------------------------------------------| +| `"gcp.auth.credentials-path"`| null | Path to a service account JSON key file. If not set, Application Default Credentials will be used. | +| `gcp.auth.scopes` | null | Comma-separated list of OAuth scopes to request. Defaults to `https://www.googleapis.com/auth/cloud-platform`. | ### Lock catalog properties From 36b1a4b3287f067386f912c774e2eb68afdde09b Mon Sep 17 00:00:00 2001 From: "piyushdubey@microsoft.com" Date: Tue, 23 Sep 2025 14:04:08 -0400 Subject: [PATCH 05/17] Fix formatting --- docs/docs/configuration.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/docs/configuration.md b/docs/docs/configuration.md index 01596525b5fc..2e6d24e17aa4 100644 --- a/docs/docs/configuration.md +++ b/docs/docs/configuration.md @@ -157,7 +157,7 @@ They support Basic, OAuth2, SigV4, and Google authentication, in addition to the | `rest.auth.basic.password` | null | Password for Basic authentication. Required if `rest.auth.type` is `basic`. | | `rest.auth.sigv4.delegate-auth-type` | `oauth2` | Auth type to delegate to after SigV4 signing. Defaults to `oauth2`. | -### OAuth2 Properties +### OAuth2 auth properties Required and Optional properties to include while using OAuth2 authentication | Property | Default | Description | @@ -181,7 +181,6 @@ Required and Optional properties to include while using Google authentication | `"gcp.auth.credentials-path"`| null | Path to a service account JSON key file. If not set, Application Default Credentials will be used. | | `gcp.auth.scopes` | null | Comma-separated list of OAuth scopes to request. Defaults to `https://www.googleapis.com/auth/cloud-platform`. | - ### Lock catalog properties Here are the catalog properties related to locking. They are used by some catalog implementations to control the locking behavior during commits. From 9a98544c3735695697b065ce57216e0644202bc8 Mon Sep 17 00:00:00 2001 From: "piyushdubey@microsoft.com" Date: Tue, 23 Sep 2025 14:10:17 -0400 Subject: [PATCH 06/17] Minor description fixes --- docs/docs/configuration.md | 44 +++++++++++++++++++------------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/docs/docs/configuration.md b/docs/docs/configuration.md index 2e6d24e17aa4..ec5b900c6bc8 100644 --- a/docs/docs/configuration.md +++ b/docs/docs/configuration.md @@ -150,36 +150,36 @@ They support Basic, OAuth2, SigV4, and Google authentication, in addition to the ### REST auth properties -| Property | Default | Description | -|--------------------------------------|------------------|---------------------------------------------------------------------------------------------------------------------------------------------------| -| `rest.auth.type` | none | Authentication mechanism for REST catalog access. Supported values: `none`, `basic`, `oauth2`, `sigv4`, `google`. | -| `rest.auth.basic.username` | null | Username for Basic authentication. Required if `rest.auth.type` = `basic`. | -| `rest.auth.basic.password` | null | Password for Basic authentication. Required if `rest.auth.type` is `basic`. | -| `rest.auth.sigv4.delegate-auth-type` | `oauth2` | Auth type to delegate to after SigV4 signing. Defaults to `oauth2`. | +| Property | Default | Description | +|--------------------------------------|------------------|-------------------------------------------------------------------------------------------------------------------| +| `rest.auth.type` | none | Authentication mechanism for REST catalog access. Supported values: `none`, `basic`, `oauth2`, `sigv4`, `google`. | +| `rest.auth.basic.username` | null | Username for Basic authentication. Required if `rest.auth.type` = `basic`. | +| `rest.auth.basic.password` | null | Password for Basic authentication. Required if `rest.auth.type` = `basic`. | +| `rest.auth.sigv4.delegate-auth-type` | `oauth2` | Auth type to delegate to after SigV4 signing. Defaults to `oauth2`. | ### OAuth2 auth properties Required and Optional properties to include while using OAuth2 authentication -| Property | Default | Description | -|-------------------------|------------------|-------------------------------------------------------------------------------------------------------------------------------------------------| -| `token` | null | A Bearer token to interact with the server. Required if `rest.auth.type` = `oauth2`. | -| `credential` | null | Credential string (client_id:client_secret) to exchange a token in the OAuth2 client credentials flow. Required if `rest.auth.type` = `oauth2`. | -| `oauth2-server-uri` | null | OAuth2 token endpoint URI. Required if the REST catalog is not the OAuth2 authentication server. Required if `rest.auth.type` = `oauth2`. | -| `token-expires-in-ms` | 3600000 (1 hour) | Time in milliseconds after which a bearer token is considered expired. Used to decide when to refresh or re-exchange a token. | -| `token-refresh-enabled` | true | Determines whether tokens are automatically refreshed when expiration details are available. | -| `token-exchange-enabled`| true | Determines whether to exchange tokens to acquire new tokens. | -| `scope` | null | Additional scope for `oauth2`. | -| `audience` | null | Optional param `audience` for `oauth2`. | -| `resource` | null | Optional param `resource` for `oauth2`. | -| `catalog` | null | Scope string(s) used when requesting `oauth2` tokens for the Iceberg REST catalog. | +| Property | Default | Description | +|-------------------------|------------------|-----------------------------------------------------------------------------------------------------------------------------| +| `token` | null | A Bearer token to interact with the server. Required. | +| `credential` | null | Credential string (client_id:client_secret) to exchange a token in the OAuth2 client credentials flow. Required. | +| `oauth2-server-uri` | null | OAuth2 token endpoint URI. Required if the REST catalog is not the OAuth2 authentication server. Required. | +| `token-expires-in-ms` | 3600000 (1 hour) | Time in milliseconds after which a bearer token is considered expired. Used to decide when to refresh or re-exchange a token. | +| `token-refresh-enabled` | true | Determines whether tokens are automatically refreshed when expiration details are available. | +| `token-exchange-enabled`| true | Determines whether to exchange tokens to acquire new tokens. | +| `scope` | null | Additional scope for `oauth2`. | +| `audience` | null | Optional param to specify token `audience` | +| `resource` | null | Optional param to specify `resource` | +| `catalog` | null | Scope string(s) used when requesting `oauth2` tokens for the Iceberg REST catalog. | ### Google auth properties Required and Optional properties to include while using Google authentication -| Property | Default | Description | -|------------------------------|------------------|---------------------------------------------------------------------------------------------------------------| -| `"gcp.auth.credentials-path"`| null | Path to a service account JSON key file. If not set, Application Default Credentials will be used. | -| `gcp.auth.scopes` | null | Comma-separated list of OAuth scopes to request. Defaults to `https://www.googleapis.com/auth/cloud-platform`. | +| Property | Default | Description | +|----------------------------|------------------|---------------------------------------------------------------------------------------------------------------| +| `gcp.auth.credentials-path`| null | Path to a service account JSON key file. If not set, Application Default Credentials will be used. | +| `gcp.auth.scopes` | null | Comma-separated list of OAuth scopes to request. Defaults to `https://www.googleapis.com/auth/cloud-platform`. | ### Lock catalog properties From bf35b2fe43704c0269d2d5375aa4c439c2696f83 Mon Sep 17 00:00:00 2001 From: "piyushdubey@microsoft.com" Date: Tue, 23 Sep 2025 15:40:56 -0400 Subject: [PATCH 07/17] Minor description fixes and param cleanup --- docs/docs/configuration.md | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/docs/docs/configuration.md b/docs/docs/configuration.md index ec5b900c6bc8..f5000db7ca70 100644 --- a/docs/docs/configuration.md +++ b/docs/docs/configuration.md @@ -160,18 +160,17 @@ They support Basic, OAuth2, SigV4, and Google authentication, in addition to the ### OAuth2 auth properties Required and Optional properties to include while using OAuth2 authentication -| Property | Default | Description | -|-------------------------|------------------|-----------------------------------------------------------------------------------------------------------------------------| -| `token` | null | A Bearer token to interact with the server. Required. | -| `credential` | null | Credential string (client_id:client_secret) to exchange a token in the OAuth2 client credentials flow. Required. | -| `oauth2-server-uri` | null | OAuth2 token endpoint URI. Required if the REST catalog is not the OAuth2 authentication server. Required. | -| `token-expires-in-ms` | 3600000 (1 hour) | Time in milliseconds after which a bearer token is considered expired. Used to decide when to refresh or re-exchange a token. | -| `token-refresh-enabled` | true | Determines whether tokens are automatically refreshed when expiration details are available. | -| `token-exchange-enabled`| true | Determines whether to exchange tokens to acquire new tokens. | -| `scope` | null | Additional scope for `oauth2`. | -| `audience` | null | Optional param to specify token `audience` | -| `resource` | null | Optional param to specify `resource` | -| `catalog` | null | Scope string(s) used when requesting `oauth2` tokens for the Iceberg REST catalog. | +| Property | Default | Description | +|-------------------------|-------------------|-----------------------------------------------------------------------------------------------------------------------------| +| `token` | null | A Bearer token to interact with the server. Required. | +| `credential` | null | Credential string (client_id:client_secret) to exchange a token in the OAuth2 client credentials flow. Required. | +| `oauth2-server-uri` | `v1/oauth/tokens` | OAuth2 token endpoint URI. Required if the REST catalog is not the OAuth2 authentication server. Required. | +| `token-expires-in-ms` | 3600000 (1 hour) | Time in milliseconds after which a bearer token is considered expired. Used to decide when to refresh or re-exchange a token. | +| `token-refresh-enabled` | true | Determines whether tokens are automatically refreshed when expiration details are available. | +| `token-exchange-enabled`| true | Determines whether to exchange tokens to acquire new tokens. | +| `scope` | `catalog` | Additional scope for `oauth2`. | +| `audience` | null | Optional param to specify token `audience` | +| `resource` | null | Optional param to specify `resource` | ### Google auth properties Required and Optional properties to include while using Google authentication From 5aaddd856b1e2b2870e822384743adb0fd4dd227 Mon Sep 17 00:00:00 2001 From: "piyushdubey@microsoft.com" Date: Wed, 24 Sep 2025 02:51:37 -0400 Subject: [PATCH 08/17] Rephrase descriptions as per PR feedback --- docs/docs/configuration.md | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/docs/docs/configuration.md b/docs/docs/configuration.md index f5000db7ca70..86c783fabe20 100644 --- a/docs/docs/configuration.md +++ b/docs/docs/configuration.md @@ -155,30 +155,30 @@ They support Basic, OAuth2, SigV4, and Google authentication, in addition to the | `rest.auth.type` | none | Authentication mechanism for REST catalog access. Supported values: `none`, `basic`, `oauth2`, `sigv4`, `google`. | | `rest.auth.basic.username` | null | Username for Basic authentication. Required if `rest.auth.type` = `basic`. | | `rest.auth.basic.password` | null | Password for Basic authentication. Required if `rest.auth.type` = `basic`. | -| `rest.auth.sigv4.delegate-auth-type` | `oauth2` | Auth type to delegate to after SigV4 signing. Defaults to `oauth2`. | +| `rest.auth.sigv4.delegate-auth-type` | `oauth2` | Auth type to delegate to after `sigv4` signing. | ### OAuth2 auth properties Required and Optional properties to include while using OAuth2 authentication -| Property | Default | Description | -|-------------------------|-------------------|-----------------------------------------------------------------------------------------------------------------------------| -| `token` | null | A Bearer token to interact with the server. Required. | -| `credential` | null | Credential string (client_id:client_secret) to exchange a token in the OAuth2 client credentials flow. Required. | -| `oauth2-server-uri` | `v1/oauth/tokens` | OAuth2 token endpoint URI. Required if the REST catalog is not the OAuth2 authentication server. Required. | -| `token-expires-in-ms` | 3600000 (1 hour) | Time in milliseconds after which a bearer token is considered expired. Used to decide when to refresh or re-exchange a token. | -| `token-refresh-enabled` | true | Determines whether tokens are automatically refreshed when expiration details are available. | -| `token-exchange-enabled`| true | Determines whether to exchange tokens to acquire new tokens. | -| `scope` | `catalog` | Additional scope for `oauth2`. | -| `audience` | null | Optional param to specify token `audience` | -| `resource` | null | Optional param to specify `resource` | +| Property | Default | Description | +|-------------------------|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `token` | null | A Bearer token to interact with the server. Either `token` or `credential` is required. | +| `credential` | null | Credential string in the form of `client_id:client_secret` to exchange for a token in the OAuth2 client credentials flow. Either `token` or `credential` is required. | +| `oauth2-server-uri` | `v1/oauth/tokens` | OAuth2 token endpoint URI. Required if the REST catalog is not the OAuth2 authentication server. | +| `token-expires-in-ms` | 3600000 (1 hour) | Time in milliseconds after which a bearer token is considered expired. Used to decide when to refresh or re-exchange a token. | +| `token-refresh-enabled` | true | Determines whether tokens are automatically refreshed when expiration details are available. | +| `token-exchange-enabled`| true | Determines whether to use the token exchange flow to acquire new tokens. Disabling this will allow fallback to the client credential flow. | +| `scope` | `catalog` | Additional scope for `oauth2`. | +| `audience` | null | Optional param to specify token `audience` | +| `resource` | null | Optional param to specify `resource` | ### Google auth properties Required and Optional properties to include while using Google authentication -| Property | Default | Description | -|----------------------------|------------------|---------------------------------------------------------------------------------------------------------------| -| `gcp.auth.credentials-path`| null | Path to a service account JSON key file. If not set, Application Default Credentials will be used. | -| `gcp.auth.scopes` | null | Comma-separated list of OAuth scopes to request. Defaults to `https://www.googleapis.com/auth/cloud-platform`. | +| Property | Default | Description | +|----------------------------|--------------------------------------------------|--------------------------------------------------| +| `gcp.auth.credentials-path`| Default Application Credentials | Path to a service account JSON key file. | +| `gcp.auth.scopes` | `https://www.googleapis.com/auth/cloud-platform` | Comma-separated list of OAuth scopes to request. | ### Lock catalog properties From 8b9f1ec29c91381681798976ddd70633e949e4ef Mon Sep 17 00:00:00 2001 From: Piyush Dubey Date: Wed, 24 Sep 2025 03:28:51 -0400 Subject: [PATCH 09/17] Update docs/docs/configuration.md fix phrasing Co-authored-by: Eduard Tudenhoefner --- docs/docs/configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/configuration.md b/docs/docs/configuration.md index 86c783fabe20..58fc3b57e325 100644 --- a/docs/docs/configuration.md +++ b/docs/docs/configuration.md @@ -173,7 +173,7 @@ Required and Optional properties to include while using OAuth2 authentication | `resource` | null | Optional param to specify `resource` | ### Google auth properties -Required and Optional properties to include while using Google authentication +Required and optional properties to include while using `google` authentication | Property | Default | Description | |----------------------------|--------------------------------------------------|--------------------------------------------------| From dd6ed54f893fdfff5b4111348e36417c008ce0a8 Mon Sep 17 00:00:00 2001 From: Piyush Dubey Date: Wed, 24 Sep 2025 03:29:25 -0400 Subject: [PATCH 10/17] Update docs/docs/configuration.md fix phrasing Co-authored-by: Eduard Tudenhoefner --- docs/docs/configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/configuration.md b/docs/docs/configuration.md index 58fc3b57e325..7f59e6b68f4d 100644 --- a/docs/docs/configuration.md +++ b/docs/docs/configuration.md @@ -177,7 +177,7 @@ Required and optional properties to include while using `google` authentication | Property | Default | Description | |----------------------------|--------------------------------------------------|--------------------------------------------------| -| `gcp.auth.credentials-path`| Default Application Credentials | Path to a service account JSON key file. | +| `gcp.auth.credentials-path`| Application Default Credentials (ADC) | Path to a service account JSON key file. | | `gcp.auth.scopes` | `https://www.googleapis.com/auth/cloud-platform` | Comma-separated list of OAuth scopes to request. | ### Lock catalog properties From ab30d670f107144726b3ff4e2bb4cf3e291d9fcb Mon Sep 17 00:00:00 2001 From: Piyush Dubey Date: Wed, 24 Sep 2025 03:30:15 -0400 Subject: [PATCH 11/17] Update docs/docs/configuration.md Co-authored-by: Eduard Tudenhoefner --- docs/docs/configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/configuration.md b/docs/docs/configuration.md index 7f59e6b68f4d..8916b340e4bd 100644 --- a/docs/docs/configuration.md +++ b/docs/docs/configuration.md @@ -152,7 +152,7 @@ They support Basic, OAuth2, SigV4, and Google authentication, in addition to the | Property | Default | Description | |--------------------------------------|------------------|-------------------------------------------------------------------------------------------------------------------| -| `rest.auth.type` | none | Authentication mechanism for REST catalog access. Supported values: `none`, `basic`, `oauth2`, `sigv4`, `google`. | +| `rest.auth.type` | `none` | Authentication mechanism for REST catalog access. Supported values: `none`, `basic`, `oauth2`, `sigv4`, `google`. | | `rest.auth.basic.username` | null | Username for Basic authentication. Required if `rest.auth.type` = `basic`. | | `rest.auth.basic.password` | null | Password for Basic authentication. Required if `rest.auth.type` = `basic`. | | `rest.auth.sigv4.delegate-auth-type` | `oauth2` | Auth type to delegate to after `sigv4` signing. | From 01a6cfd890f0199d22ac8481f6ff3e972b6f844e Mon Sep 17 00:00:00 2001 From: Piyush Dubey Date: Wed, 24 Sep 2025 03:30:26 -0400 Subject: [PATCH 12/17] Update docs/docs/configuration.md Co-authored-by: Eduard Tudenhoefner --- docs/docs/configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/configuration.md b/docs/docs/configuration.md index 8916b340e4bd..0bf56d163d8e 100644 --- a/docs/docs/configuration.md +++ b/docs/docs/configuration.md @@ -146,7 +146,7 @@ Flink passes in catalog properties through `CREATE CATALOG` statement, see more ### Catalog REST auth properties The following catalog properties configure authentication for the REST catalog. -They support Basic, OAuth2, SigV4, and Google authentication, in addition to the default none. +They support Basic, OAuth2, SigV4, and Google authentication. ### REST auth properties From a1c1b865309f8299012b02a0ad36cb258d799e6d Mon Sep 17 00:00:00 2001 From: Piyush Dubey Date: Wed, 24 Sep 2025 03:30:38 -0400 Subject: [PATCH 13/17] Update docs/docs/configuration.md Co-authored-by: Eduard Tudenhoefner --- docs/docs/configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/configuration.md b/docs/docs/configuration.md index 0bf56d163d8e..9543d1bd707f 100644 --- a/docs/docs/configuration.md +++ b/docs/docs/configuration.md @@ -158,7 +158,7 @@ They support Basic, OAuth2, SigV4, and Google authentication. | `rest.auth.sigv4.delegate-auth-type` | `oauth2` | Auth type to delegate to after `sigv4` signing. | ### OAuth2 auth properties -Required and Optional properties to include while using OAuth2 authentication +Required and optional properties to include while using `oauth2` authentication | Property | Default | Description | |-------------------------|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------| From 49e83396dea2381a2b6d5bddd2406d0aef263a11 Mon Sep 17 00:00:00 2001 From: Piyush Dubey Date: Wed, 24 Sep 2025 09:40:27 -0400 Subject: [PATCH 14/17] Update docs/docs/configuration.md Co-authored-by: Eduard Tudenhoefner --- docs/docs/configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/configuration.md b/docs/docs/configuration.md index 9543d1bd707f..135e8be46c52 100644 --- a/docs/docs/configuration.md +++ b/docs/docs/configuration.md @@ -152,7 +152,7 @@ They support Basic, OAuth2, SigV4, and Google authentication. | Property | Default | Description | |--------------------------------------|------------------|-------------------------------------------------------------------------------------------------------------------| -| `rest.auth.type` | `none` | Authentication mechanism for REST catalog access. Supported values: `none`, `basic`, `oauth2`, `sigv4`, `google`. | +| `rest.auth.type` | `none` | Authentication mechanism for REST catalog access. Supported values: `none`, `basic`, `oauth2`, `sigv4`, `google`. | | `rest.auth.basic.username` | null | Username for Basic authentication. Required if `rest.auth.type` = `basic`. | | `rest.auth.basic.password` | null | Password for Basic authentication. Required if `rest.auth.type` = `basic`. | | `rest.auth.sigv4.delegate-auth-type` | `oauth2` | Auth type to delegate to after `sigv4` signing. | From 9c8d8988ab5386eacf338b84453982e75ca7b2c8 Mon Sep 17 00:00:00 2001 From: "piyushdubey@microsoft.com" Date: Wed, 24 Sep 2025 09:43:34 -0400 Subject: [PATCH 15/17] Fix table alignment --- docs/docs/configuration.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/docs/configuration.md b/docs/docs/configuration.md index 135e8be46c52..9151ac9d9a2e 100644 --- a/docs/docs/configuration.md +++ b/docs/docs/configuration.md @@ -157,7 +157,7 @@ They support Basic, OAuth2, SigV4, and Google authentication. | `rest.auth.basic.password` | null | Password for Basic authentication. Required if `rest.auth.type` = `basic`. | | `rest.auth.sigv4.delegate-auth-type` | `oauth2` | Auth type to delegate to after `sigv4` signing. | -### OAuth2 auth properties +#### OAuth2 auth properties Required and optional properties to include while using `oauth2` authentication | Property | Default | Description | @@ -172,12 +172,12 @@ Required and optional properties to include while using `oauth2` authentication | `audience` | null | Optional param to specify token `audience` | | `resource` | null | Optional param to specify `resource` | -### Google auth properties +#### Google auth properties Required and optional properties to include while using `google` authentication | Property | Default | Description | |----------------------------|--------------------------------------------------|--------------------------------------------------| -| `gcp.auth.credentials-path`| Application Default Credentials (ADC) | Path to a service account JSON key file. | +| `gcp.auth.credentials-path`| Application Default Credentials (ADC) | Path to a service account JSON key file. | | `gcp.auth.scopes` | `https://www.googleapis.com/auth/cloud-platform` | Comma-separated list of OAuth scopes to request. | ### Lock catalog properties From b588b98e8c93ef1eebe6317fb2eb26cdb57f99c2 Mon Sep 17 00:00:00 2001 From: "piyushdubey@microsoft.com" Date: Wed, 24 Sep 2025 09:44:57 -0400 Subject: [PATCH 16/17] Make Rest Auth properties sub section of Catalog REST properties --- docs/docs/configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/configuration.md b/docs/docs/configuration.md index 9151ac9d9a2e..a7b9fbf816d0 100644 --- a/docs/docs/configuration.md +++ b/docs/docs/configuration.md @@ -148,7 +148,7 @@ Flink passes in catalog properties through `CREATE CATALOG` statement, see more The following catalog properties configure authentication for the REST catalog. They support Basic, OAuth2, SigV4, and Google authentication. -### REST auth properties +#### REST auth properties | Property | Default | Description | |--------------------------------------|------------------|-------------------------------------------------------------------------------------------------------------------| From 698b69c70abdf8060257db5bb5075370c9a29388 Mon Sep 17 00:00:00 2001 From: Eduard Tudenhoefner Date: Thu, 25 Sep 2025 07:51:39 +0200 Subject: [PATCH 17/17] Update configuration.md --- docs/docs/configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/configuration.md b/docs/docs/configuration.md index a7b9fbf816d0..bb6a1e9a8bd3 100644 --- a/docs/docs/configuration.md +++ b/docs/docs/configuration.md @@ -143,7 +143,7 @@ The properties can be manually constructed or passed in from a compute engine li Spark uses its session properties as catalog properties, see more details in the [Spark configuration](spark-configuration.md#catalog-configuration) section. Flink passes in catalog properties through `CREATE CATALOG` statement, see more details in the [Flink](flink.md#adding-catalogs) section. -### Catalog REST auth properties +### REST Catalog auth properties The following catalog properties configure authentication for the REST catalog. They support Basic, OAuth2, SigV4, and Google authentication.