Merge branch 'main' into update_commit_message_version

Author: lszskye

.asf.yaml

@@ -18,7 +18,7 @@
 #
 
 # The format of this file is documented at
-# https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features
+# https://github.com/apache/infrastructure-asfyaml
 
 github:
   description: "Paimon-cpp is a high-performance C++ implementation of Apache Paimon."
@@ -29,6 +29,11 @@ github:
     - paimon
     - apache
     - cpp
+  collaborators:
+    - zjw1111
+    - lxy-9602
+    - lucasfang
+    - lszskye
   enabled_merge_buttons:
     merge: false
     squash: true

.github/workflows/release_candidate.yaml

@@ -78,9 +78,13 @@ jobs:
             --output-dir release/ci
 
       - name: Audit source archive
+        env:
+          # The creator must ignore caller-provided gzip defaults.
+          GZIP: "-9"
         run: |
           scripts/releasing/verify_release_candidate.sh \
             --allow-unsigned \
+            --git-ref HEAD \
             --skip-build \
             "release/ci/apache-paimon-cpp-${RELEASE_VERSION}-src.tgz"
 

docs/source/user_guide/data_types.rst

@@ -188,15 +188,14 @@ and `Arrow DataTypes <https://arrow.apache.org/docs/format/Columnar.html#data-ty
 
    * - ``MAP<kt, vt>``
      - Map
-     - Data type of an associative array that maps keys to values (including NULL). A map cannot contain duplicate keys; each key can map to at most one value.
+     - Data type of an associative array that maps keys (including NULL) to values (including NULL). A map cannot contain duplicate keys; each key can map to at most one value.
 
        There is no restriction of element types; it is the responsibility of the user to ensure uniqueness.
 
        The type can be declared using ``MAP<kt, vt>`` where kt is the data type of the key elements and vt is the data type of the value elements.
 
-       **Note:** In Paimon C++, map keys must be explicitly marked as ``NOT NULL``.
-       Apache Arrow does not support nullable map keys. If the key type is not
-       marked as ``NOT NULL`` in the schema, parsing will fail with an error.
+       **Note:** Paimon C++ accepts nullable map key declarations for schema compatibility.
+       A query fails only if the data actually contains a NULL map key.
 
    * - ``MULTISET<t>``
      - Not Supported

docs/source/user_guide/schema.rst

@@ -84,35 +84,11 @@ DataField represents a column of the table.
 3. ``type``: data type, very similar to SQL type string.
 4. ``description``: string.
 
-Limitations
------------
+Nullable MAP Keys
+-----------------
 
-MAP Key Must Be NOT NULL
-^^^^^^^^^^^^^^^^^^^^^^^^
-
-Apache Arrow does not support nullable map keys. When defining a ``MAP`` type in the schema,
-the key must be explicitly marked as ``NOT NULL``. If the key is not marked as ``NOT NULL``,
-schema parsing will fail with an error.
-
-For example, the following is **valid**:
-
-.. code-block:: json
-
-   {
-     "type": "MAP",
-     "key": "TINYINT NOT NULL",
-     "value": "SMALLINT"
-   }
-
-The following is **invalid** and will be rejected:
-
-.. code-block:: json
-
-   {
-     "type": "MAP",
-     "key": "TINYINT",
-     "value": "SMALLINT"
-   }
+Paimon C++ accepts nullable ``MAP`` key declarations for compatibility with existing schemas.
+A query fails only when the data actually contains a NULL map key.
 
 Update Schema
 -------------

scripts/releasing/README.md

@@ -34,16 +34,25 @@ Before starting a release:
 - obtain an ASF code-signing key, publish it through the ASF account system,
   and make sure it is present in
   [Paimon KEYS](https://downloads.apache.org/paimon/KEYS);
-- install `git`, `gpg`, `svn`, `gh`, `python3`, `curl` or `wget`, Java, CMake,
-  Ninja, and the toolchain needed by `ci/scripts/build_paimon.sh` (Java is
-  required by Apache RAT);
+- install `git`, GNU `gzip`, `gpg`, `svn`, `gh`, `python3`, `curl` or `wget`,
+  Java, CMake, Ninja, and the toolchain needed by
+  `ci/scripts/build_paimon.sh` (Java is required by Apache RAT);
 - authenticate `gh` with access to read GitHub Actions runs in
   `apache/paimon-cpp`;
 - make sure the Apache Git remote points directly to
   `apache/paimon-cpp`;
 - prepare and merge a release-preparation PR that updates the release notes and
   all version metadata, and passes the normal and release-candidate workflows.
 
+The source archive uses GNU gzip with fixed options so that macOS and Linux
+produce the same bytes. On macOS, install Homebrew gzip and either put it first
+on `PATH` or select it explicitly:
+
+```bash
+brew install gzip
+export PAIMON_GZIP="$(brew --prefix gzip)/bin/gzip"
+```
+
 For example, update all version locations and review the diff:
 
 ```bash
@@ -80,10 +89,13 @@ The release scripts use `vVERSION-rcRC` for release-candidate tags and
 
 Start from the exact clean commit approved for the candidate. Before publishing,
 the wrapper fetches the release branch and requires `HEAD` to be contained in
-its current history. It then creates and verifies a signed RC tag, creates the
-source archive and its checksum/signature, performs the full source-release
-verification, pushes the tag, waits for the tag-triggered release-candidate
-workflow to succeed, and imports the artifacts into ASF `dist/dev`:
+its current history. It then creates and verifies a signed RC tag, pushes the
+tag, and waits for the tag-triggered release-candidate workflow. That workflow
+creates the canonical source archive and checksum, builds and tests the same
+archive with GCC and Clang, and uploads it as a workflow artifact. The wrapper
+downloads those exact bytes, confirms that they are reproducible from the tag,
+signs the archive locally, performs the full source-release verification, and
+imports the three files into ASF `dist/dev`:
 
 ```bash
 scripts/releasing/release_rc.sh \
@@ -96,12 +108,13 @@ scripts/releasing/release_rc.sh \
 The release branch defaults to `main`; use `--release-branch NAME` for a
 maintenance release from another Apache branch.
 
-Use `--prepare-only` to create and verify artifacts without pushing the tag or
-uploading to ASF infrastructure. This local-only mode does not require `HEAD`
-to match the remote release branch. Use `--dry-run` to print identifiers
-without making changes. A resumed run reuses an existing local tag or complete
-artifact set only after validating it. A prepare-only run does not print a vote
-email and must not be used to start a vote.
+Use `--prepare-only` to create and verify preview artifacts without pushing the
+tag or uploading to ASF infrastructure. This local-only mode does not require
+`HEAD` to match the remote release branch. A preview is not authoritative: a
+published run downloads the workflow artifact and rejects an existing local
+archive if its bytes differ. Use `--dry-run` to print identifiers without
+making changes. A prepare-only run does not print a vote email and must not be
+used to start a vote.
 
 The candidate directory contains:
 
@@ -164,13 +177,16 @@ The verifier checks:
 - installation plus compilation and execution of an external CMake consumer.
 
 Pass `--git-ref v0.3.0-rc1` when the Git repository is available to regenerate
-the archive from the signed tag and compare it byte-for-byte.
+the archive from the signed tag with GNU gzip and compare it byte-for-byte.
+This check requires GNU gzip on every platform; it intentionally rejects the
+macOS system gzip instead of treating different compressed bytes as equivalent.
 
 `--allow-unsigned`, `--skip-rat`, `--skip-build`, and `--skip-install` exist for
 CI or local development of the release process. They are not a substitute for
 the corresponding checks when voting. The release-candidate workflow creates
-an unsigned archive for deterministic CI validation; official artifacts must
-always be signed by the release manager.
+the unsigned canonical archive for deterministic CI validation. The release
+manager downloads and signs that exact archive; the private signing key remains
+only on the release manager's machine.
 
 ## Publish an approved release
 
@@ -207,8 +223,9 @@ than ASF's general one-hour minimum.
 
 - `bump_version.py`: consistently check or update CMake and documentation
   version metadata.
-- `create_source_release.sh`: deterministically create an archive, SHA-512
-  checksum, and optional detached signature from an immutable Git ref.
+- `create_source_release.sh`: deterministically create an archive with GNU
+  gzip, a SHA-512 checksum, and an optional detached signature from an immutable
+  Git ref.
 - `validate_source_archive.py`: reject unsafe or non-portable tar members and
   compiled files.
 - `verify_release_candidate.sh`: perform voter-facing integrity, license,

scripts/releasing/create_source_release.sh

@@ -49,6 +49,9 @@ The script creates:
   apache-paimon-cpp-VERSION-src.tgz.asc  (when --signing-key is provided)
 
 Existing artifacts are never overwritten.
+
+GNU gzip is required so macOS and Linux produce the same compressed bytes.
+Set PAIMON_GZIP to an explicit GNU gzip executable when it is not on PATH.
 EOF
 }
 
@@ -68,6 +71,36 @@ calculate_sha512() {
     fi
 }
 
+find_gnu_gzip() {
+    local candidate
+    local resolved
+    local version_line
+    local -a candidates
+
+    if [[ -n "${PAIMON_GZIP:-}" ]]; then
+        candidates=("${PAIMON_GZIP}")
+    else
+        candidates=(gzip ggzip)
+    fi
+
+    for candidate in "${candidates[@]}"; do
+        if [[ -x "${candidate}" ]]; then
+            resolved=${candidate}
+        elif resolved=$(command -v "${candidate}" 2>/dev/null); then
+            :
+        else
+            continue
+        fi
+        version_line=$("${resolved}" --version 2>/dev/null | sed -n '1p' || true)
+        if [[ "${version_line}" =~ ^gzip[[:space:]][0-9] ]]; then
+            printf '%s\n' "${resolved}"
+            return 0
+        fi
+    done
+
+    fail "GNU gzip is required for reproducible source archives; on macOS run 'brew install gzip' and set PAIMON_GZIP to the Homebrew gzip executable"
+}
+
 while [[ $# -gt 0 ]]; do
     case "$1" in
         --version)
@@ -128,6 +161,8 @@ DOCS_VERSION=$(
 [[ "${DOCS_VERSION}" == "${RELEASE_VERSION}" ]] ||
     fail "documentation version ${DOCS_VERSION:-<missing>} does not match ${RELEASE_VERSION}"
 
+GZIP_BIN=$(find_gnu_gzip)
+
 ARTIFACT_NAME="apache-paimon-cpp-${RELEASE_VERSION}-src.tgz"
 ARCHIVE_ROOT="paimon-cpp-${RELEASE_VERSION}"
 
@@ -147,7 +182,10 @@ git -C "${SOURCE_ROOT}" -c tar.umask=0022 archive \
     --format=tar \
     --prefix="${ARCHIVE_ROOT}/" \
     "${GIT_REF}" |
-    gzip -n >"${TEMP_DIR}/${ARTIFACT_NAME}"
+    (
+        unset GZIP
+        "${GZIP_BIN}" --no-name --stdout -6
+    ) >"${TEMP_DIR}/${ARTIFACT_NAME}"
 
 SHA512=$(calculate_sha512 "${TEMP_DIR}/${ARTIFACT_NAME}")
 printf '%s  %s\n' "${SHA512}" "${ARTIFACT_NAME}" \