GH-49686: [C++][FlightRPC][ODBC][Release] Create signing script for Windows FlightSQL ODBC build by amoeba · Pull Request #49788 · apache/arrow

amoeba · 2026-04-18T01:20:33Z

Rationale for this change

We need a script for the release manager to run during the release to locally sign the Windows artifacts for the FlightSQL ODBC driver.

Ref: #49404

What changes are included in this PR?

New script, 07-flightsql-odbc-upload.sh
Update to release guide
Renamed other release scripts to make space in the numbering scheme:
- 07-publish-gh-release -> 08
- 08-binary-verify -> 09
- 08-vote-emai -> 10

Are these changes tested?

Not 100% but I've tested each step separately. I tested on my fork using fake tags and releases.

Are there any user-facing changes?

No.

GitHub Issue: [Release][FlightRPC][ODBC] Create release script for signing Windows DLL and MSI #49686

amoeba · 2026-04-18T01:28:42Z

The new script depends on the unmerged changes in #49603 so this PR will need a rebase after that's merged. Reviewers will note that the script in this PR refers to a workflow_dispatch trigger in #49603 that's not on main yet.

kou

Could you add new scripts to lint targets?

arrow/.pre-commit-config.yaml

Lines 286 to 360 in 6dd07b1

    
             - repo: https://github.com/koalaman/shellcheck-precommit 
        
               rev: v0.10.0 
        
               hooks: 
        
                 - id: shellcheck 
        
                   alias: shell 
        
                   # TODO: Remove this when we fix all lint failures 
        
                   files: >- 
        
                     ( 
        
                     ?^c_glib/test/run-test\.sh$| 
        
                     ?^ci/scripts/c_glib_build\.sh$| 
        
                     ?^ci/scripts/c_glib_test\.sh$| 
        
                     ?^ci/scripts/ccache_setup\.sh$| 
        
                     ?^ci/scripts/cpp_test\.sh$| 
        
                     ?^ci/scripts/download_tz_database\.sh$| 
        
                     ?^ci/scripts/install_azurite\.sh$| 
        
                     ?^ci/scripts/install_ccache\.sh$| 
        
                     ?^ci/scripts/install_ceph\.sh$| 
        
                     ?^ci/scripts/install_chromedriver\.sh$| 
        
                     ?^ci/scripts/install_cmake\.sh$| 
        
                     ?^ci/scripts/install_conda\.sh$| 
        
                     ?^ci/scripts/install_dask\.sh$| 
        
                     ?^ci/scripts/install_emscripten\.sh$| 
        
                     ?^ci/scripts/install_gcs_testbench\.sh$| 
        
                     ?^ci/scripts/install_iwyu\.sh$| 
        
                     ?^ci/scripts/install_minio\.sh$| 
        
                     ?^ci/scripts/install_ninja\.sh$| 
        
                     ?^ci/scripts/install_numba\.sh$| 
        
                     ?^ci/scripts/install_numpy\.sh$| 
        
                     ?^ci/scripts/install_pandas\.sh$| 
        
                     ?^ci/scripts/install_python\.sh$| 
        
                     ?^ci/scripts/install_sccache\.sh$| 
        
                     ?^ci/scripts/install_spark\.sh$| 
        
                     ?^ci/scripts/install_vcpkg\.sh$| 
        
                     ?^ci/scripts/integration_arrow_build\.sh$| 
        
                     ?^ci/scripts/integration_arrow\.sh$| 
        
                     ?^ci/scripts/integration_dask\.sh$| 
        
                     ?^ci/scripts/integration_hdfs\.sh$| 
        
                     ?^ci/scripts/integration_spark\.sh$| 
        
                     ?^ci/scripts/matlab_build\.sh$| 
        
                     ?^ci/scripts/msys2_setup\.sh$| 
        
                     ?^ci/scripts/msys2_system_clean\.sh$| 
        
                     ?^ci/scripts/msys2_system_upgrade\.sh$| 
        
                     ?^ci/scripts/nanoarrow_build\.sh$| 
        
                     ?^ci/scripts/python_build_emscripten\.sh$| 
        
                     ?^ci/scripts/python_build\.sh$| 
        
                     ?^ci/scripts/python_sdist_build\.sh$| 
        
                     ?^ci/scripts/python_sdist_test\.sh$| 
        
                     ?^ci/scripts/python_wheel_unix_test\.sh$| 
        
                     ?^ci/scripts/python_test_type_annotations\.sh$| 
        
                     ?^ci/scripts/r_build\.sh$| 
        
                     ?^ci/scripts/r_revdepcheck\.sh$| 
        
                     ?^ci/scripts/release_test\.sh$| 
        
                     ?^ci/scripts/ruby_test\.sh$| 
        
                     ?^ci/scripts/rust_build\.sh$| 
        
                     ?^ci/scripts/util_download_apache\.sh$| 
        
                     ?^ci/scripts/util_enable_core_dumps\.sh$| 
        
                     ?^ci/scripts/util_free_space\.sh$| 
        
                     ?^ci/scripts/util_log\.sh$| 
        
                     ?^cpp/build-support/build-lz4-lib\.sh$| 
        
                     ?^cpp/build-support/build-zstd-lib\.sh$| 
        
                     ?^cpp/build-support/get-upstream-commit\.sh$| 
        
                     ?^cpp/build-support/update-thrift\.sh$| 
        
                     ?^cpp/examples/minimal_build/run\.sh$| 
        
                     ?^cpp/examples/tutorial_examples/run\.sh$| 
        
                     ?^cpp/src/arrow/flight/sql/odbc/install/unix/install_odbc\.sh$| 
        
                     ?^dev/release/05-binary-upload\.sh$| 
        
                     ?^dev/release/08-binary-verify\.sh$| 
        
                     ?^dev/release/binary-recover\.sh$| 
        
                     ?^dev/release/post-03-binary\.sh$| 
        
                     ?^dev/release/post-08-docs\.sh$| 
        
                     ?^dev/release/post-09-python\.sh$| 
        
                     ?^dev/release/setup-rhel-rebuilds\.sh$| 
        
                     ?^dev/release/utils-generate-checksum\.sh$| 
        
                     ?^swift/gen-protobuffers\.sh$| 
        
                     )

arrow/.pre-commit-config.yaml

Lines 374 to 383 in 6dd07b1

    
                   files: >- 
        
                     ( 
        
                     ?^ci/scripts/python_test_type_annotations\.sh$| 
        
                     ?^cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc\.sh$| 
        
                     ?^dev/release/05-binary-upload\.sh$| 
        
                     ?^dev/release/binary-recover\.sh$| 
        
                     ?^dev/release/post-03-binary\.sh$| 
        
                     ?^dev/release/post-08-docs\.sh$| 
        
                     ?^dev/release/post-09-python\.sh$| 
        
                     )

kou · 2026-04-18T11:28:56Z

+  echo "[4/9] Removing unsigned DLL from GitHub Release..."
+  gh release delete-asset "${tag}" \
+    --repo "${GITHUB_REPOSITORY}" \
+    --yes \
+    "${dll_unsigned}"


Why do we need to remove unsigned DLL?

Just to reduce confusion and artifact usage, we don't need to. I don't know if I have a principled reason either way. If we did jsign signing entirely in CI, we'd probably build the unsigned DLL, sign it, and only upload the signed version. Happy to change this though.

I see.

How about keeping the unsigned DLL while we use this workflow (building the unsigned DLL in CI, signing the unsigned DLL on local and uploading the signed DLL from local) so that we may be able to debug something or retry signing when the signed DLL has any problem.

I like that idea. I removed the step in ac39888.

kou · 2026-04-18T11:31:12Z

+  run_url=$(gh workflow run cpp_extra.yml \
+    --repo "${GITHUB_REPOSITORY}" \
+    --ref "${tag}" \
+    --field odbc_release_step=true 2>&1 | grep -oE 'https://[^ ]+')
+  run_id=${run_url##*/} # Extract the run ID from the URL (the part after the last slash)


How about using --json and --jq like we did in

arrow/dev/release/utils-watch-gh-workflow.sh

Lines 37 to 43 in 6dd07b1

RUN_ID=$(gh run list \

--branch "${TAG}" \

--jq '.[].databaseId' \

--json databaseId \

--limit 1 \

--repo "${REPOSITORY}" \

--workflow "${WORKFLOW}")

?

workflow run doesn't have a --json output arg and the command just prints,

[4/9] Triggering odbc_release_step in cpp_extra.yml workflow... ✓ Created workflow_dispatch event for cpp_extra.yml at AMOEBA-arrow-99.9.9-rc0 https://github.com/amoeba/arrow/actions/runs/24583481107 To see the created workflow run, try: gh run view 24583481107 To see runs for this workflow, try: gh run list --workflow="cpp_extra.yml"

Doing it this way isn't ideal but it seems slightly better than the alternative which would be to use gh run list with the right filters and assume the first result is the correct run. I guess that's a pretty safe assumption.

Do you have a preference?

Oh, sorry. I misunderstood the used command...

Doing it this way isn't ideal but it seems slightly better than the alternative which would be to use gh run list with the right filters and assume the first result is the correct run. I guess that's a pretty safe assumption.

Do you have a preference?

If it works, I prefer it. Parsing free format text may be fragile. gh workflow run output format may be changed.

If it doesn't work, let's parsing free format text.

That's a good point. I think we can use utils-watch-gha-workflow.sh for this then. It already includes --interval.

Changed in df22a8e.

Copilot

Pull request overview

Adds release automation scripts under dev/release/ to support Windows FlightSQL ODBC artifact signing and GitHub Release/RC verification steps as part of the Arrow release process.

Changes:

Add a new 07-flightsqlodbc-upload.sh script intended to download, sign, and re-upload the FlightSQL ODBC DLL/MSI to the RC GitHub Release.
Add 08-publish-gh-release.sh to publish (undraft) an RC GitHub Release.
Add 09-binary-verify.sh to rerun the verify_rc.yml workflow for an RC tag.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 8 comments.

File	Description
dev/release/07-flightsqlodbc-upload.sh	New local signing + upload workflow for FlightSQL ODBC Windows DLL/MSI artifacts.
dev/release/08-publish-gh-release.sh	New script to mark the RC GitHub Release as non-draft.
dev/release/09-binary-verify.sh	New script to rerun the GitHub Actions RC verification workflow.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Sutou Kouhei <kou@cozmixng.org>

amoeba · 2026-04-18T18:20:55Z

The pre-commit failure in https://github.com/apache/arrow/actions/runs/24610839359/job/71964834631?pr=49788#step:5:192 looks like it's caused by an unrelated issue. The fs package recently made a breaking change and it now requires libuv-dev.

I'm going to test a fix in ff033b6 but will revert it and put it into another PR. See #49594 for more context.

This reverts commit ff033b6.

amoeba · 2026-04-18T18:31:56Z

That seems to fix it and now pre-commit is green in CI. I filed #49791. Reverted the test change in 7b99cd6.

amoeba · 2026-04-18T18:32:44Z

Thanks for the review @kou. I think I've addressed everything so this is ready for another review.

…kage (#49791) ### Rationale for this change Pre-commit in CI just started to fail trying to install the R `fs` package. See https://github.com/apache/arrow/actions/runs/24610839359. I think we didn't see this until now because we were using cached pre-commit and a recent PR of mine just invalidated the cache incidentally. See also #49594. The `fs` package recently made a change that requires we install libuv development headers to install it. ### What changes are included in this PR? - Updates dev.yml to Install libuv1-dev ### Are these changes tested? Yes. See my other PR: #49788 (comment). ### Are there any user-facing changes? No. Authored-by: Bryce Mecum <petridish@gmail.com> Signed-off-by: Bryce Mecum <petridish@gmail.com>

kou

+1

kou · 2026-04-19T05:53:00Z

+if [ -z "${ESIGNER_STOREPASS:-}" ]; then
+  echo "ERROR: ESIGNER_STOREPASS is not set" >&2
+  exit 1
+fi
+if [ -z "${ESIGNER_KEYPASS:-}" ]; then
+  echo "ERROR: ESIGNER_KEYPASS is not set" >&2
+  exit 1
+fi


Could you move them after utils-env.sh?

Whoops. Fixed in 81b1e88.

kou · 2026-04-19T06:02:42Z

          ?^dev/release/05-binary-upload\.sh$|
-          ?^dev/release/08-binary-verify\.sh$|
+          ?^dev/release/07-flightsqlodbc-upload\.sh$|
+          ?^dev/release/09-binary-verify\.sh$|


Could you also add 08-publishgh-release.sh if it doesn't have any lint failures?
If it has any lint failures, it's out of scope of this PR.

Could you also add 07-flightsqlodbc-upload.sh to

arrow/.pre-commit-config.yaml

Lines 365 to 383 in 7190a7c

rev: v3.10.0-1

hooks:

- id: shfmt

alias: shell

args:

# The default args is "--write --simplify" but we don't use

# "--simplify". Because it's conflicted will ShellCheck.

- "--write"

# TODO: Remove this when we fix all lint failures

files: >-

(

?^ci/scripts/python_test_type_annotations\.sh$|

?^cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc\.sh$|

?^dev/release/05-binary-upload\.sh$|

?^dev/release/binary-recover\.sh$|

?^dev/release/post-03-binary\.sh$|

?^dev/release/post-08-docs\.sh$|

?^dev/release/post-09-python\.sh$|

)

?

I checked 08-publishgh-release.sh locally, and it has one failure. I'll file a separate PR.

I added the script to the shfmt step in fa646fc.

Copilot

Pull request overview

Copilot reviewed 4 out of 6 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Create new signing script

817b479

amoeba requested review from assignUser, jonkeane, kou and raulcd as code owners April 18, 2026 01:20

amoeba requested a review from alinaliBQ April 18, 2026 01:20

remove the unsigned dll from the release

61e7503

github-actions bot added the awaiting review Awaiting review label Apr 18, 2026

amoeba mentioned this pull request Apr 18, 2026

[Docs][Release][FlightRPC][ODBC] Document release process for Windows artifacts #49687

Open

amoeba mentioned this pull request Apr 18, 2026

[C++][CI][Packaging][FlightPRC] Manual ODBC Windows MSI installer signing #49404

Open

kou reviewed Apr 18, 2026

View reviewed changes

github-actions bot removed the awaiting review Awaiting review label Apr 18, 2026

kou requested a review from Copilot April 18, 2026 11:35

github-actions bot added the awaiting changes Awaiting changes label Apr 18, 2026

Copilot started reviewing on behalf of kou April 18, 2026 11:35 View session

Copilot AI reviewed Apr 18, 2026

View reviewed changes

Update dev/release/07-flightsqlodbc-upload.sh

0f1f391

Co-authored-by: Sutou Kouhei <kou@cozmixng.org>

github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Apr 18, 2026

amoeba added 6 commits April 18, 2026 10:46

remove old scripts

f120e3c

add step to release.rst

12b503b

Update pre-commit

31a99fb

Put env vars in .env.example

62265bd

Use fixed local temp dir

92a8ae8

use --interval

5b01a0f

github-actions bot added the Component: Documentation label Apr 18, 2026

amoeba added 3 commits April 18, 2026 10:53

fix shellcheck issues in script

6717e34

fix latent bug in is_signed

33866c7

Fix bug with filename

47ec038

amoeba added 2 commits April 18, 2026 11:14

make it so the grep doesn't fail the script early

7f0e6e0

testing, should revert: add libuv1-dev

ff033b6

amoeba mentioned this pull request Apr 18, 2026

MINOR: [Dev][CI] Install libuv1-dev in dev.yml lint step for R fs package #49791

Merged

Revert "testing, should revert: add libuv1-dev"

7b99cd6

This reverts commit ff033b6.

amoeba requested a review from kou April 18, 2026 18:32

github-actions bot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Apr 19, 2026

kou approved these changes Apr 19, 2026

View reviewed changes

kou requested a review from Copilot April 19, 2026 06:03

github-actions bot added awaiting merge Awaiting merge and removed awaiting changes Awaiting changes labels Apr 19, 2026

Copilot started reviewing on behalf of kou April 19, 2026 06:03 View session

Copilot AI reviewed Apr 19, 2026

View reviewed changes

Comment thread dev/release/07-flightsqlodbc-upload.sh Outdated

Comment thread dev/release/07-flightsqlodbc-upload.sh Outdated

Comment thread docs/source/developers/release.rst Outdated

Comment thread docs/source/developers/release.rst

amoeba added 7 commits April 20, 2026 08:36

move env var checks after utils-env

81b1e88

add 07 script to shfmt

fa646fc

use :=

402b75f

use 'rc-number' for consistency

d051f0b

rename 09-vote-email to 10-vote-email

07a4328

remove step deleting unsigned dll

ac39888

Use watch-gh-workflow script

df22a8e

	- repo: https://github.com/koalaman/shellcheck-precommit
	rev: v0.10.0
	hooks:
	- id: shellcheck
	alias: shell
	# TODO: Remove this when we fix all lint failures
	files: >-
	(
	?^c_glib/test/run-test\.sh$\|
	?^ci/scripts/c_glib_build\.sh$\|
	?^ci/scripts/c_glib_test\.sh$\|
	?^ci/scripts/ccache_setup\.sh$\|
	?^ci/scripts/cpp_test\.sh$\|
	?^ci/scripts/download_tz_database\.sh$\|
	?^ci/scripts/install_azurite\.sh$\|
	?^ci/scripts/install_ccache\.sh$\|
	?^ci/scripts/install_ceph\.sh$\|
	?^ci/scripts/install_chromedriver\.sh$\|
	?^ci/scripts/install_cmake\.sh$\|
	?^ci/scripts/install_conda\.sh$\|
	?^ci/scripts/install_dask\.sh$\|
	?^ci/scripts/install_emscripten\.sh$\|
	?^ci/scripts/install_gcs_testbench\.sh$\|
	?^ci/scripts/install_iwyu\.sh$\|
	?^ci/scripts/install_minio\.sh$\|
	?^ci/scripts/install_ninja\.sh$\|
	?^ci/scripts/install_numba\.sh$\|
	?^ci/scripts/install_numpy\.sh$\|
	?^ci/scripts/install_pandas\.sh$\|
	?^ci/scripts/install_python\.sh$\|
	?^ci/scripts/install_sccache\.sh$\|
	?^ci/scripts/install_spark\.sh$\|
	?^ci/scripts/install_vcpkg\.sh$\|
	?^ci/scripts/integration_arrow_build\.sh$\|
	?^ci/scripts/integration_arrow\.sh$\|
	?^ci/scripts/integration_dask\.sh$\|
	?^ci/scripts/integration_hdfs\.sh$\|
	?^ci/scripts/integration_spark\.sh$\|
	?^ci/scripts/matlab_build\.sh$\|
	?^ci/scripts/msys2_setup\.sh$\|
	?^ci/scripts/msys2_system_clean\.sh$\|
	?^ci/scripts/msys2_system_upgrade\.sh$\|
	?^ci/scripts/nanoarrow_build\.sh$\|
	?^ci/scripts/python_build_emscripten\.sh$\|
	?^ci/scripts/python_build\.sh$\|
	?^ci/scripts/python_sdist_build\.sh$\|
	?^ci/scripts/python_sdist_test\.sh$\|
	?^ci/scripts/python_wheel_unix_test\.sh$\|
	?^ci/scripts/python_test_type_annotations\.sh$\|
	?^ci/scripts/r_build\.sh$\|
	?^ci/scripts/r_revdepcheck\.sh$\|
	?^ci/scripts/release_test\.sh$\|
	?^ci/scripts/ruby_test\.sh$\|
	?^ci/scripts/rust_build\.sh$\|
	?^ci/scripts/util_download_apache\.sh$\|
	?^ci/scripts/util_enable_core_dumps\.sh$\|
	?^ci/scripts/util_free_space\.sh$\|
	?^ci/scripts/util_log\.sh$\|
	?^cpp/build-support/build-lz4-lib\.sh$\|
	?^cpp/build-support/build-zstd-lib\.sh$\|
	?^cpp/build-support/get-upstream-commit\.sh$\|
	?^cpp/build-support/update-thrift\.sh$\|
	?^cpp/examples/minimal_build/run\.sh$\|
	?^cpp/examples/tutorial_examples/run\.sh$\|
	?^cpp/src/arrow/flight/sql/odbc/install/unix/install_odbc\.sh$\|
	?^dev/release/05-binary-upload\.sh$\|
	?^dev/release/08-binary-verify\.sh$\|
	?^dev/release/binary-recover\.sh$\|
	?^dev/release/post-03-binary\.sh$\|
	?^dev/release/post-08-docs\.sh$\|
	?^dev/release/post-09-python\.sh$\|
	?^dev/release/setup-rhel-rebuilds\.sh$\|
	?^dev/release/utils-generate-checksum\.sh$\|
	?^swift/gen-protobuffers\.sh$\|
	)

	RUN_ID=$(gh run list \
	--branch "${TAG}" \
	--jq '.[].databaseId' \
	--json databaseId \
	--limit 1 \
	--repo "${REPOSITORY}" \
	--workflow "${WORKFLOW}")

	rev: v3.10.0-1
	hooks:
	- id: shfmt
	alias: shell
	args:
	# The default args is "--write --simplify" but we don't use
	# "--simplify". Because it's conflicted will ShellCheck.
	- "--write"
	# TODO: Remove this when we fix all lint failures
	files: >-
	(
	?^ci/scripts/python_test_type_annotations\.sh$\|
	?^cpp/src/arrow/flight/sql/odbc/install/mac/install_odbc\.sh$\|
	?^dev/release/05-binary-upload\.sh$\|
	?^dev/release/binary-recover\.sh$\|
	?^dev/release/post-03-binary\.sh$\|
	?^dev/release/post-08-docs\.sh$\|
	?^dev/release/post-09-python\.sh$\|
	)

Conversation

amoeba commented Apr 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

amoeba commented Apr 18, 2026

Uh oh!

kou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kou Apr 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

amoeba commented Apr 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

amoeba commented Apr 18, 2026

Uh oh!

amoeba commented Apr 18, 2026

Uh oh!

kou left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

amoeba commented Apr 18, 2026 •

edited

Loading

kou Apr 19, 2026 •

edited

Loading

amoeba commented Apr 18, 2026 •

edited

Loading