SQL: Expand virtual tables reference, add SHOW/DESCRIBE for new objects #590
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| = Information Schema | ||
| :description: Information schema views in Redpanda SQL expose database object metadata and grants through SQL-standard views. | ||
| :page-topic-type: reference | ||
|
|
||
| The `information_schema` namespace provides SQL-standard views over database object metadata. Use these views to inspect catalogs, tables, schemas, columns, and privilege grants without querying the underlying PostgreSQL system catalogs directly. | ||
|
|
||
| == Available views | ||
|
|
||
| Standard views follow the SQL specification, with semantics consistent with https://www.postgresql.org/docs/current/information-schema.html[PostgreSQL's `information_schema`^]. Redpanda SQL also adds one extension view for per-relation EXTERNAL SOURCE grants. | ||
|
|
||
| [cols="<40%,<60%",options="header"] | ||
| |=== | ||
| |View |Description | ||
|
|
||
| |`information_schema.character_sets` | ||
| |Character sets available in the current database. | ||
|
|
||
| |`information_schema.columns` | ||
| |Columns of tables and views visible to the current role. | ||
|
|
||
| |`information_schema.key_column_usage` | ||
| |Columns constrained as keys (primary keys, unique constraints, foreign keys). | ||
|
|
||
| |`information_schema.referential_constraints` | ||
| |Foreign-key constraints visible to the current role. | ||
|
|
||
| |`information_schema.role_external_relation_grants` | ||
| |Per-relation EXTERNAL SOURCE grants. Redpanda SQL extension that captures grants whose stored relation pattern is anything other than `\*` (catalog-level grants surface in `information_schema.role_table_grants` instead). | ||
|
|
||
| |`information_schema.role_table_grants` | ||
| |Privileges granted on tables and table-like objects to the current role. | ||
|
|
||
| |`information_schema.role_usage_grants` | ||
| |USAGE privileges granted to the current role. | ||
|
|
||
| |`information_schema.table_constraints` | ||
| |Table constraints (primary key, unique, foreign key, check) visible to the current role. | ||
|
|
||
| |`information_schema.tables` | ||
| |Tables and views visible to the current role. | ||
| |=== | ||
|
|
||
| == information_schema.role_external_relation_grants | ||
|
|
||
| Per-relation EXTERNAL SOURCE grants. Rows whose stored relation pattern is anything other than `\*`. Catalog-level EXTERNAL SOURCE grants (stored pattern `\*`) surface in `information_schema.role_table_grants`, like foreign-table grants in PostgreSQL. | ||
|
|
||
| When you `GRANT ... ON EXTERNAL SOURCE catalog_name => 'pattern'`, the pattern is stored verbatim. Redpanda SQL does not normalize or canonicalize it. Patterns are either a fully-qualified relation name like `ns1.ns2.my_table` or a name with a trailing wildcard like `ns1.public_tables*`. See xref:reference:sql/sql-statements/grant.adoc[GRANT] for the full grant syntax and xref:sql:manage/manage-access.adoc[Manage access to Redpanda SQL] for the access model. | ||
|
|
||
| Visibility mirrors `information_schema.role_table_grants`. A regular role sees rows where it is the grantee. A superuser sees grants for every non-superuser. | ||
|
|
||
| Use this view to find the exact stored relation pattern when you need to xref:reference:sql/sql-statements/revoke.adoc[revoke] a grant. `REVOKE` requires the pattern argument to match a stored grant exactly. Revoking a pattern that was never granted returns an error, so query this view first to see what's currently registered. | ||
|
|
||
| [cols="<30%,<15%,<10%,<45%",options="header"] | ||
| |=== | ||
| |Column |Type |Nullable |Description | ||
|
|
||
| |`grantor` |`text` |No |Role that granted the privilege. | ||
| |`grantee` |`text` |No |Role that holds the privilege. | ||
| |`database_name` |`text` |No |Database the grant applies to. | ||
| |`schema_name` |`text` |No |Schema the grant applies to. | ||
| |`source_name` |`text` |No |External source (catalog or storage) the grant applies to. | ||
| |`relation_pattern` |`text` |No |Relation-name pattern the grant matches. Wildcards (`\*`) are allowed in the pattern. | ||
| |`privilege_type` |`text` |No |Privilege granted, such as `SELECT`. | ||
| |`is_grantable` |`text` |No |`YES` if the grantee can grant this privilege to other roles; `NO` otherwise. | ||
| |=== | ||
|
|
||
| [source,sql] | ||
| ---- | ||
| SELECT grantee, source_name, relation_pattern, privilege_type | ||
| FROM information_schema.role_external_relation_grants | ||
| WHERE grantee = 'analyst'; | ||
| ---- | ||
|
|
||
| == Suggested reading | ||
|
|
||
| * xref:reference:sql/sql-statements/grant.adoc[GRANT] | ||
| * xref:reference:sql/sql-statements/revoke.adoc[REVOKE] | ||
| * xref:sql:manage/manage-access.adoc[Manage access to Redpanda SQL] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,27 +1,67 @@ | ||
| = ALTER REDPANDA CATALOG | ||
| :description: The ALTER REDPANDA CATALOG statement modifies connection properties of an existing Redpanda catalog, including the link to an Iceberg catalog. | ||
| :page-topic-type: reference | ||
|
|
||
| The `ALTER REDPANDA CATALOG` statement modifies connection properties of an existing Redpanda catalog. You can also use it to link the Redpanda catalog to an Iceberg catalog (or detach an existing link) so that queries against the Redpanda catalog return both live records and Iceberg-translated history (records previously written from the topic into the linked Iceberg catalog). See xref:sql:query-data/query-iceberg-topics.adoc[] for details. | ||
|
Contributor
There was a problem hiding this comment. Reminder about this link not working |
||
|
|
||
| == Syntax | ||
|
|
||
| [source,sql] | ||
| ---- | ||
| ALTER REDPANDA CATALOG [IF EXISTS] catalog_name | ||
| WITH (option = 'value' [, ...]); | ||
|
|
||
| ALTER REDPANDA CATALOG [IF EXISTS] catalog_name | ||
| USING CATALOG [schema.]iceberg_catalog_name | ||
| [WITH (option = 'value' [, ...])]; | ||
|
|
||
| ALTER REDPANDA CATALOG [IF EXISTS] catalog_name | ||
| USING CATALOG NULL | ||
| [WITH (option = 'value' [, ...])]; | ||
| ---- | ||
|
|
||
| * `catalog_name`: Name of the Redpanda catalog to modify. | ||
| * `IF EXISTS`: Optional. Prevents an error if the catalog does not exist. | ||
| * `option = 'value'`: One or more connection options to update. See xref:reference:sql/sql-statements/create-redpanda-catalog.adoc[CREATE REDPANDA CATALOG] for the full list of options. | ||
| * `USING CATALOG iceberg_catalog_name`: Links the Redpanda catalog to an existing Iceberg catalog. The Iceberg catalog must already exist. You can qualify the Iceberg catalog name with a schema prefix. | ||
| * `USING CATALOG NULL`: Detaches the Redpanda catalog from any linked Iceberg catalog. | ||
|
|
||
| == Examples | ||
|
|
||
| === Update broker address | ||
|
|
||
| [source,sql] | ||
| ---- | ||
| ALTER REDPANDA CATALOG production_redpanda | ||
| WITH (initial_brokers = 'new-broker:9092'); | ||
| ---- | ||
|
|
||
| === Link to an Iceberg catalog | ||
|
|
||
| To link a Redpanda catalog to an existing Iceberg catalog so that queries against the Redpanda catalog return both live records and Iceberg-translated history: | ||
|
|
||
| [source,sql] | ||
| ---- | ||
| ALTER REDPANDA CATALOG production_redpanda | ||
| USING CATALOG lakehouse_catalog; | ||
| ---- | ||
|
|
||
| === Relink to a different Iceberg catalog | ||
|
|
||
| To change which Iceberg catalog a Redpanda catalog is linked to, run `ALTER REDPANDA CATALOG ... USING CATALOG <new-iceberg-cat>`: | ||
|
|
||
| [source,sql] | ||
| ---- | ||
| ALTER REDPANDA CATALOG production_redpanda | ||
| USING CATALOG new_lakehouse_catalog; | ||
| ---- | ||
|
|
||
| === Detach from an Iceberg catalog | ||
|
|
||
| To remove an existing link to an Iceberg catalog so that queries against the Redpanda catalog return only live records: | ||
|
|
||
| [source,sql] | ||
| ---- | ||
| ALTER REDPANDA CATALOG production_redpanda | ||
| USING CATALOG NULL; | ||
| ---- | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| = SHOW CATALOGS | ||
| :description: The SHOW CATALOGS statement lists catalogs in the cluster. Variants list only Redpanda catalogs or only Iceberg catalogs. | ||
| :page-topic-type: reference | ||
|
|
||
| The `SHOW CATALOGS` statement lists catalogs in the cluster. Variants filter by catalog type: Redpanda catalogs or Iceberg catalogs. | ||
|
|
||
| [NOTE] | ||
| ==== | ||
| `SHOW CATALOGS` and its variants only display catalogs in schemas where the user has the `USAGE` grant. | ||
| ==== | ||
|
|
||
| == Syntax | ||
|
|
||
| [source,sql] | ||
| ---- | ||
| SHOW CATALOGS; | ||
| SHOW REDPANDA CATALOGS; | ||
| SHOW KAFKA CATALOGS; | ||
| SHOW ICEBERG CATALOGS; | ||
| ---- | ||
|
|
||
| `SHOW KAFKA CATALOGS` is a synonym for `SHOW REDPANDA CATALOGS`. | ||
|
|
||
| == Examples | ||
|
|
||
| === List all catalogs | ||
|
|
||
| To list all catalogs (both Redpanda and Iceberg) visible to the caller: | ||
|
|
||
| [source,sql] | ||
| ---- | ||
| SHOW CATALOGS; | ||
| ---- | ||
|
|
||
| For SQL-queryable access to the same information (with `WHERE`, `ORDER BY`, and `LIMIT` support), see xref:reference:sql/system-virtual-tables.adoc#system-catalogs[`system.catalogs`]. | ||
|
|
||
| === List only Redpanda catalogs | ||
|
|
||
| [source,sql] | ||
| ---- | ||
| SHOW REDPANDA CATALOGS; | ||
| ---- | ||
|
|
||
| === List only Iceberg catalogs | ||
|
|
||
| [source,sql] | ||
| ---- | ||
| SHOW ICEBERG CATALOGS; | ||
| ---- | ||
|
|
||
| For each Iceberg catalog's REST endpoint, warehouse, and authentication type, see xref:reference:sql/system-virtual-tables.adoc#system-iceberg-catalogs[`system.iceberg_catalogs`] or xref:reference:sql/sql-statements/describe.adoc[`DESCRIBE ICEBERG CATALOG`]. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| = SHOW QUERIES | ||
| :description: The SHOW QUERIES statement displays currently running queries on the cluster. | ||
| :page-topic-type: reference | ||
|
|
||
| The `SHOW QUERIES` statement displays currently running queries on the cluster. This statement is only available to the superuser. | ||
|
|
||
| [NOTE] | ||
| ==== | ||
| This statement is not case-sensitive. `show queries`, `Show Queries`, and `SHOW QUERIES` all produce the same result. | ||
| ==== | ||
|
|
||
| == Examples | ||
|
|
||
| [source,sql] | ||
| ---- | ||
| SHOW QUERIES; | ||
| ---- | ||
|
|
||
| NOTE: `SHOW QUERIES` is equivalent to `SELECT * FROM system.queries`. You can query the xref:reference:sql/system-virtual-tables.adoc#system-queries[`system.queries`] table directly to filter or sort results using `WHERE` and `ORDER BY` clauses. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,52 +1,75 @@ | ||
| = SHOW TABLES | ||
| :description: The SHOW TABLES statement retrieves information about existing tables, including tables mapped from Redpanda topics and Iceberg tables registered in the local catalog. | ||
| :page-topic-type: reference | ||
|
|
||
| The `SHOW TABLES` statement retrieves information about existing tables in a catalog. Use one of the typed variants (`SHOW REDPANDA TABLES`, `SHOW ICEBERG TABLES`) or qualify with a catalog name (`SHOW TABLES FROM catalog_name`). The unqualified `SHOW TABLES` form is not supported and returns an error. | ||
|
|
||
| [NOTE] | ||
| ==== | ||
| `SHOW TABLES` variants only display tables in schemas where the user has the `USAGE` grant. | ||
| ==== | ||
|
|
||
| == Syntax | ||
|
|
||
| [source,sql] | ||
| ---- | ||
| SHOW TABLES FROM catalog_name; | ||
|
kbatuigas marked this conversation as resolved.
|
||
|
|
||
| SHOW REDPANDA TABLES; | ||
| SHOW REDPANDA TABLES catalog_name; | ||
| SHOW REDPANDA TABLES schema.catalog_name; | ||
|
|
||
| SHOW KAFKA TABLES; | ||
| SHOW KAFKA TABLES catalog_name; | ||
| SHOW KAFKA TABLES schema.catalog_name; | ||
|
|
||
| SHOW ICEBERG TABLES; | ||
| SHOW ICEBERG TABLES catalog_name; | ||
| SHOW ICEBERG TABLES schema.catalog_name; | ||
| ---- | ||
|
|
||
| * `catalog_name`: Required after `SHOW TABLES FROM`; optional after `SHOW REDPANDA TABLES` and `SHOW ICEBERG TABLES`. Filter by a specific Redpanda or Iceberg catalog. | ||
| * `schema.catalog_name`: Optional. Filter by both schema and catalog. Useful when catalogs in different schemas share a name. | ||
|
|
||
| `SHOW KAFKA TABLES` is a synonym for `SHOW REDPANDA TABLES`. | ||
|
|
||
| == Examples | ||
|
|
||
| === List tables from a Redpanda catalog | ||
|
|
||
| To list tables mapped through a specific Redpanda catalog: | ||
|
|
||
| [source,sql] | ||
| ---- | ||
| SHOW REDPANDA TABLES default_redpanda_catalog; | ||
| ---- | ||
|
|
||
| To list all Redpanda-catalog tables across all connections: | ||
|
|
||
| [source,sql] | ||
| ---- | ||
| SHOW REDPANDA TABLES; | ||
| ---- | ||
|
|
||
| === List Iceberg tables | ||
|
|
||
| To list all Iceberg tables registered in the local catalog: | ||
|
|
||
| [source,sql] | ||
| ---- | ||
| SHOW ICEBERG TABLES; | ||
| ---- | ||
|
|
||
| To list Iceberg tables in a specific catalog: | ||
|
|
||
| [source,sql] | ||
| ---- | ||
| SHOW ICEBERG TABLES lakehouse_catalog; | ||
| ---- | ||
|
|
||
| To list Iceberg tables in a specific schema and catalog (here, schema `analytics`, catalog `lakehouse_catalog`): | ||
|
|
||
| [source,sql] | ||
| ---- | ||
| SHOW ICEBERG TABLES analytics.lakehouse_catalog; | ||
| ---- | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@kbatuigas should this intro just include one link to Postgres doc, instead of so many throughout the table?