From 1a193b52a55f2b44b7898665a8cfab8c663e8bea Mon Sep 17 00:00:00 2001 From: Eunbin Son Date: Thu, 5 Nov 2026 09:10:25 +0900 Subject: [PATCH] [docs] Fix stale Flink action examples, Spark bucketed join setup and JDBC lock backends - Use --warehouse/--database/--table in the action jar compact example; --path has been rejected by ActionFactory since #4728. - Use --database/--table in the expire_snapshots action example and drop the unused --version argument. - Add spark.paimon.scan.preserve-data-grouping to the Spark bucketed join examples, which is required since #10076. - List PostgreSQL among the databases supporting the JDBC catalog lock. - Fix broken external links in the Variant, filesystem and PyPaimon SQL docs. Generated-by: Claude Code --- docs/docs/append-table/bucketed.mdx | 5 +++-- docs/docs/flink/action-jars.md | 4 +++- docs/docs/flink/sql-ddl.md | 2 +- docs/docs/learn-paimon/scenario-guide.mdx | 2 +- docs/docs/maintenance/filesystems.mdx | 2 +- docs/docs/maintenance/manage-snapshots.mdx | 4 ++-- docs/docs/multimodal-table/variant.mdx | 2 +- docs/docs/primary-key-table/query-performance.md | 4 +++- docs/docs/pypaimon/sql.md | 2 +- docs/docs/spark/catalogs.md | 2 +- 10 files changed, 17 insertions(+), 12 deletions(-) diff --git a/docs/docs/append-table/bucketed.mdx b/docs/docs/append-table/bucketed.mdx index cf3378630644..54b209129475 100644 --- a/docs/docs/append-table/bucketed.mdx +++ b/docs/docs/append-table/bucketed.mdx @@ -91,10 +91,11 @@ set of buckets through this optimization. ## Bucketed Join -Spark can use compatible bucket distributions to avoid a shuffle in a batch join. Enable V2 bucketing and join on the -distribution keys: +Spark can use compatible bucket distributions to avoid a shuffle in a batch join. Enable +`scan.preserve-data-grouping` and V2 bucketing, then join on the distribution keys: ```sql +SET spark.paimon.scan.preserve-data-grouping = true; SET spark.sql.sources.v2.bucketing.enabled = true; CREATE TABLE fact_table (order_id INT, f1 STRING) USING paimon diff --git a/docs/docs/flink/action-jars.md b/docs/docs/flink/action-jars.md index bd87f51815c8..8bac3c9b2821 100644 --- a/docs/docs/flink/action-jars.md +++ b/docs/docs/flink/action-jars.md @@ -44,7 +44,9 @@ The following command is used to compact a table. /bin/flink run \ /path/to/paimon-flink-action-@@VERSION@@.jar \ compact \ - --path + --warehouse \ + --database \ + --table ``` Use ` --help` to inspect its arguments. Partition specifications use comma-separated diff --git a/docs/docs/flink/sql-ddl.md b/docs/docs/flink/sql-ddl.md index 0fde76e63865..81ec17bcbd1d 100644 --- a/docs/docs/flink/sql-ddl.md +++ b/docs/docs/flink/sql-ddl.md @@ -123,7 +123,7 @@ You can set location in the properties of table/database by the config of `locat By using the Paimon JDBC catalog, changes to the catalog will be directly stored in relational databases such as SQLite, MySQL, postgres, etc. -Currently, lock configuration is only supported for MySQL and SQLite. If you are using a different type of database for catalog storage, please do not configure `lock.enabled`. +Currently, lock configuration is only supported for MySQL, PostgreSQL, and SQLite. If you are using a different type of database for catalog storage, please do not configure `lock.enabled`. Paimon JDBC Catalog in Flink needs to correctly add the corresponding jar package for connecting to the database. You should first download JDBC connector bundled jar and add it to classpath. such as MySQL, postgres diff --git a/docs/docs/learn-paimon/scenario-guide.mdx b/docs/docs/learn-paimon/scenario-guide.mdx index 337b37ffeab7..e085dd1f0777 100644 --- a/docs/docs/learn-paimon/scenario-guide.mdx +++ b/docs/docs/learn-paimon/scenario-guide.mdx @@ -227,7 +227,7 @@ An equality or `IN` predicate on the complete bucket key can prune unrelated buc not guarantee reading exactly one eighth of the bytes: bucket sizes can be skewed, and matching buckets still contain other keys. A composite bucket key requires constraints on every key column. -Spark can exploit compatible bucket distributions in joins with V2 bucketing enabled. Confirm +Spark can exploit compatible bucket distributions in joins with `scan.preserve-data-grouping` and V2 bucketing enabled. Confirm shuffle removal with `EXPLAIN`; see [Bucketed Join](../append-table/bucketed#bucketed-join). ### Ordered Streaming {#scenario-7-queue-like-ordered-streaming-bucketed-append} diff --git a/docs/docs/maintenance/filesystems.mdx b/docs/docs/maintenance/filesystems.mdx index 74f8b72bd50e..382f6380d2c3 100644 --- a/docs/docs/maintenance/filesystems.mdx +++ b/docs/docs/maintenance/filesystems.mdx @@ -306,7 +306,7 @@ SELECT COUNT(1) FROM test_table; From version 0.8, paimon-trino uses trino filesystem as basic file read and write system. We strongly recommend you to use jindo-sdk in trino. -You can find [How to config jindo sdk on trino](https://github.com/aliyun/alibabacloud-jindodata/blob/master/docs/user/4.x/4.6.x/4.6.12/oss/presto/jindosdk_on_presto) here. +You can find [How to config jindo sdk on trino](https://github.com/aliyun/alibabacloud-jindodata/blob/master/docs/user/4.x/4.6.x/4.6.12/oss/presto/jindosdk_on_presto.md) here. Please note that: * Use paimon to replace hive-hadoop2 when you decompress the plugin jar and find location to put in. * You can specify the `core-site.xml` in `paimon.properties` on configuration [hive.config.resources](https://trino.io/docs/current/connector/hive.html#hdfs-configuration). diff --git a/docs/docs/maintenance/manage-snapshots.mdx b/docs/docs/maintenance/manage-snapshots.mdx index d8d2e6d5b78f..c44737236b89 100644 --- a/docs/docs/maintenance/manage-snapshots.mdx +++ b/docs/docs/maintenance/manage-snapshots.mdx @@ -280,9 +280,9 @@ Run the following command: /path/to/paimon-flink-action-@@VERSION@@.jar \ expire_snapshots \ --warehouse \ - --identifier \ + --database \ + --table \ --older_than \ - --version \ --max_deletes \ --retain_max \ --retain_min \ diff --git a/docs/docs/multimodal-table/variant.mdx b/docs/docs/multimodal-table/variant.mdx index bcce4b644aa0..2e0a9e32edf6 100644 --- a/docs/docs/multimodal-table/variant.mdx +++ b/docs/docs/multimodal-table/variant.mdx @@ -32,7 +32,7 @@ suitable for event attributes, model output, tool parameters, and other data tha frequently to model every field as a table column. Paimon stores `VARIANT` with the -[Parquet Variant binary encoding](https://github.com/apache/parquet-format/blob/master/VariantEncoding), +[Parquet Variant binary encoding](https://github.com/apache/parquet-format/blob/master/VariantEncoding.md), instead of storing its JSON text. The data is parsed when it is converted to `VARIANT`, and queries can extract a path as a requested SQL type. diff --git a/docs/docs/primary-key-table/query-performance.md b/docs/docs/primary-key-table/query-performance.md index c702b3c6bf91..37c5ea41f845 100644 --- a/docs/docs/primary-key-table/query-performance.md +++ b/docs/docs/primary-key-table/query-performance.md @@ -103,6 +103,7 @@ coverage rules, especially for Vector and Full Text search. Spark can use compatible fixed-bucket layouts to avoid a join shuffle. For example: ```sql +SET spark.paimon.scan.preserve-data-grouping = true; SET spark.sql.sources.v2.bucketing.enabled = true; CREATE TABLE fact_table (order_id INT, f1 STRING) USING paimon @@ -115,6 +116,7 @@ SELECT * FROM fact_table AS fact JOIN dim_table AS dim ON fact.order_id = dim.order_id; ``` -The setting enables Spark to use partitioning reported by V2 data sources. The join keys, bucket +These settings let Paimon report its bucket layout to Spark (see +[Scan Layout and Storage Partition Joins](../spark/sql-query#scan-layout-and-storage-partition-joins)). The join keys, bucket keys, and bucket counts must be compatible; the same bucket count alone is not sufficient. Check `EXPLAIN` to confirm that Spark avoids the shuffle for the actual query. diff --git a/docs/docs/pypaimon/sql.md b/docs/docs/pypaimon/sql.md index 89ddb7572dcc..7013e08533aa 100644 --- a/docs/docs/pypaimon/sql.md +++ b/docs/docs/pypaimon/sql.md @@ -118,7 +118,7 @@ batches = ctx.sql("SELECT * FROM paimon.default.my_temp") ## Supported SQL Syntax -The SQL engine is powered by Apache DataFusion, which supports a rich set of SQL syntax. For the full SQL reference, see the [paimon-rust SQL documentation](https://paimon-rust.apache.org/sql.html) which covers: +The SQL engine is powered by Apache DataFusion, which supports a rich set of SQL syntax. For the full SQL reference, see the [paimon-rust SQL documentation](https://paimon.apache.org/docs/rust/sql/) which covers: - **DDL**: `CREATE SCHEMA`, `CREATE TABLE` (with `PARTITIONED BY`, `PRIMARY KEY`, `WITH` options), `DROP TABLE`, `ALTER TABLE`, `CREATE TEMPORARY TABLE/VIEW` - **DML**: `INSERT INTO`, `INSERT OVERWRITE` (dynamic/static partitions), `UPDATE`, `DELETE`, `MERGE INTO`, `TRUNCATE TABLE` diff --git a/docs/docs/spark/catalogs.md b/docs/docs/spark/catalogs.md index 11c92435d140..ee57c3e04905 100644 --- a/docs/docs/spark/catalogs.md +++ b/docs/docs/spark/catalogs.md @@ -94,7 +94,7 @@ If you want to see a partitioned table in Hive and also synchronize newly create The JDBC catalog stores catalog metadata in a relational database such as SQLite, MySQL, or PostgreSQL. -Currently, lock configuration is only supported for MySQL and SQLite. If you are using a different type of database for catalog storage, please do not configure `lock.enabled`. +Currently, lock configuration is only supported for MySQL, PostgreSQL, and SQLite. If you are using a different type of database for catalog storage, please do not configure `lock.enabled`. Add the JDBC driver for your database to Spark's classpath along with the Paimon connector.