[Content Addressable] Add support for Packaging Single-ABI Gems by OughtPuts · Pull Request #62 · Shopify/cibuildgem

OughtPuts · 2026-09-01T16:22:50Z

https://github.com/shop/issues-rails-infrastructure/issues/1747

TL;DR

This PR adds an optional include-single-abi flag to cibuildgem so native-extension maintainers can ask the workflow to build single-Ruby-ABI, content-addressable gems in addition to the existing multi-ABI/platform gems.

Summary

This PR keeps cibuildgem as the workflow/wiring layer. When include-single-abi: true is passed to the GitHub Action, cibuildgem passes that intent through to the rake process and sets Rake::ExtensionTask#content_addressable = true when supported.

The actual single-ABI gem creation now lives in the upstream tooling:

rake-compiler owns compiling/packaging one native gem per Ruby ABI.
Gem::PackageTask owns content-addressable packaging and SHA-based gem names.
cibuildgem only exposes the workflow flag and hands it off.

The default remains unchanged: existing workflows continue building only the current multi-ABI/platform gems unless they explicitly opt in.

Tophatting

Tophat was run locally against the dummy native gem using local branches for:

Shopify/rubygems#9897
rake-compiler/rake-compiler#257
this cibuildgem branch

Checked scenarios:

Default/no flag
cibuildgem package built only the existing Ruby/source gem and multi-ABI platform gem.
Opt-in happy path
cibuildgem package --include-single-abi built:
- the Ruby/source gem
- the existing multi-ABI platform gem
- one SHA-named content-addressable gem per Ruby ABI
All generated gems were emitted under pkg/*.gem.
Old rake-compiler compatibility
With old rake-compiler, cibuildgem warned that content-addressable packaging is unsupported and continued building the existing gems without crashing.
Old RubyGems compatibility
With new rake-compiler but old RubyGems, rake-compiler warned that RubyGems does not support content-addressable package tasks and continued building the existing gems without crashing.
Single-ABI install validation
The e2e install logic found the content-addressable gem matching the current Ruby ABI by inspecting gemspec metadata and successfully installed it.
Release dry-run
cibuildgem release --glob "pkg/*" selected all built .gem artifacts for push.

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

+      if options["package-mode"] == "single-ABI"
+        compilation_task.ruby_versions.each do |ruby_version|
+          ENV["RUBY_CC_VERSION"] = ruby_version.to_s
+          run_rake_tasks!("cibuildgem:setup", :cross, :native, :gem)


+      cibuildgem will create one gem per Ruby version supported by the gem.
    MSG
    method_option "gemspec", type: "string", required: false, desc: "The gemspec to use. Defaults to the gemspec from the current working directory."
+    method_option "package-mode", type: "string", required: false, desc: "The package mode to use when compiling multi-ABI or single-ABI gems. Defaults to multi-ABI."


      assert_predicate($CHILD_STATUS, :success?)
    end

+    def test_package_single_abi_runs_rake_tasks_for_each_ruby_version


OughtPuts · 2026-09-18T11:50:56Z

I need to have another look at backwards compatibility.

I've run these scenarios locally:

Happy path: new RubyGems + new rake-compiler + include-single-abi: true — passed; built two
traditional and two content-addressed gems.
Backwards compatibility: old RubyGems + one Ruby version — passed; built traditional gems.
include-single-abi: false: new toolchain — passed; built only traditional multi-ABI gems.
include-single-abi: true + old RubyGems — found a remaining issue: Rake crashes with the staging-
directory InvalidSpecificationException before our current post-build compatibility check runs.

This afternoon I'll do a bit more digging into this error and put forward some solutions.

Fix #258 ## TLDR Implements some small changes in order to support the build and packaging of single ABI gems, which are content addressed. ## Description This PR adds an opt-in `content_addressable` option to `Rake::ExtensionTask`: 1. When `content_addressable` is enabled, in addition to the usual multi-ABI gem, a single Ruby ABI gem is built for each Ruby version in `RUBY_CC_VERSION` via new `native:<gem>:<platform>:<ruby abi>` tasks. `native:<platform>` / `native:<gem>` depend on all of them. 2. Single ABI gems use a pessimistic `required_ruby_version` (`~> X.Y.0`) to align with the RubyGems content addressable gem expectations. 3. `Gem::PackageTask#content_addressable = true` is set for single ABI gems. RubyGems scopes the staging directory by Ruby ABI, so all gems land in `pkg/`. Each packaging task depends on the staged binaries directly. 4. If the running RubyGems doesn't support content addressable gems (`Gem::PackageTask#content_addressable=` is missing), a warning is shown and only the multi-ABI gem is built. 5. README documents the option (requires RubyGems 4.1.0+). ## Testing Tests added to `extensiontask_spec`. ## Top Hat This has been top-hatted in the flow as a whole, see the [cibuildgem CA support PR](Shopify/cibuildgem#62 (comment)) for more information. --------- Co-authored-by: Jenny Shen <jenny.shen@shopify.com> Co-authored-by: Gira Chawda <64598936+girachawda@users.noreply.github.com>

Co-authored-by: Gira Chawda <64598936+girachawda@users.noreply.github.com>

girachawda · 2026-10-09T01:45:24Z

      ENV["RUBY_CC_VERSION"] ||= compilation_task.ruby_cc_version
-
-      run_rake_tasks!("cibuildgem:setup", :cross, :native, :gem)
+      run_rake_tasks!(


@OughtPuts I know we discussed explicitly running the rake task twice here, once for the fat and then the skinny. But it seems that rake compiler already schedules that order here. So calling run rake task here with the single abi flag we would essentially trigger multi ABI, multi ABI + skinny. Did you have a specific reason in mind as to specifically calling the run rake task twice here? Maybe to make sure multi ABI executes first? I just think it is going to take extra time to run rake task twice for multi ABI twice. WDYT?

Yeah I had another look at the rake-compiler implementation this morning and we're all good, how we've set things up here will result in the correct gems (single and multi) being built with rake-compiler without any additional iteration. I just couldn't remember how rake-compiler worked when we paired! Sorry for any confusion.

Edouard-chin

The code LGTM. If it works . You can try to run this branch on the canary action (it builds a real gem).

head there https://github.com/Shopify/cibuildgem/actions/workflows/date-gem-compile.yml
Click on "run workflow", select this branch.
You can then check the artifact that was uploaded during the run, decompress them and confirm the right binaries (fat/skiny) got compiled correctly.

OughtPuts · 2026-10-12T13:37:54Z

+          @extension_task.content_addressable = true
+        else
+          warn("WARNING: content-addressable packaging was requested, but rake-compiler does not support it. " \
+            "Upgrade rake-compiler to build single-ABI gems.")


As mentioned in Slack, it would be even better to provide a rake-compiler version here to give the user more of a steer of what they need.

OughtPuts · 2026-10-12T14:23:38Z

 module Cibuildgem
  class CompilationTasks
-    attr_reader :gemspec, :native, :create_packaging_task, :extension_task
+    CONTENT_ADDRESSABLE_ENV = "CIBUILDGEM_CONTENT_ADDRESSABLE"


Are you definitely convinced we need this assignment? It feels a bit overcomplicated and hard to quickly understand to me.

OughtPuts · 2026-10-12T14:29:42Z

      ENV["RUBY_CC_VERSION"] ||= compilation_task.ruby_cc_version
-
-      run_rake_tasks!("cibuildgem:setup", :cross, :native, :gem)
+      run_rake_tasks!(


Yeah I had another look at the rake-compiler implementation this morning and we're all good, how we've set things up here will result in the correct gems (single and multi) being built with rake-compiler without any additional iteration. I just couldn't remember how rake-compiler worked when we paired! Sorry for any confusion.

OughtPuts · 2026-10-12T14:35:15Z

+      assert_nil(ENV[TEST_CONTENT_ADDRESSABLE_ENV])
+    end
+
+    def test_package_defaults_to_multi_abi_and_preserves_ruby_cc_version_from_environment


I think this naming might be out of date. We want to check if the package call defaults to false right rather than multi_abi.

OughtPuts · 2026-10-12T14:36:25Z

+      )
+    end
+
+    def test_compilation_task_enables_content_addressable_extension_task


This seems a little clunky - can we combine this test with the first test or does it definitely have to be separate?

OughtPuts · 2026-10-12T14:41:00Z

Thanks for helping with this one @girachawda ! I've left some comments - what do you think?

If you're in agreement and happy to make those small tweaks, afterwards perhaps we could arrange for one of us to test this as Edouard suggests above before the sync on Tues? And then we're pretty much ready for the Rubydex test. Let me know what you reckon!

~~I'll also update the PR description as it's out of date now with the new functionality.~~ EDIT: this looks already done, my apologies!

OughtPuts · 2026-10-12T15:49:11Z

    - name: "Install gems"
      if: "${{ inputs.step == 'install' }}"
      shell: bash
      run: gem install pkg/*.gem


Just remembered too about your question on this one. Yeah I see the problem - if we have multiple single abi gems in the pkg directory and then here we do a smoke test to install all the gems, the 3.2 ABI gem will get rejected by a 3.4 job. I reckon we get round this by just installing the multi ABI gem, but we'd have to basically filter it out of all the other gems in the directory.

OughtPuts marked this pull request as draft September 1, 2026 16:23

OughtPuts requested a balanced review from Copilot September 1, 2026 16:23

Copilot started reviewing on behalf of OughtPuts September 1, 2026 16:23 View session

Copilot AI reviewed Sep 1, 2026

OughtPuts requested a balanced review from Copilot September 1, 2026 16:26

Copilot started reviewing on behalf of OughtPuts September 1, 2026 16:27 View session

Copilot AI reviewed Sep 1, 2026

View reviewed changes

OughtPuts force-pushed the ho/build-skinny-ca-gems branch from 89aab5c to 1bf41d9 Compare September 9, 2026 16:27

OughtPuts force-pushed the ho/build-skinny-ca-gems branch from 1bf41d9 to 9b06afb Compare September 18, 2026 11:29

OughtPuts mentioned this pull request Sep 18, 2026

[Content Addressable] Add content addressable / single Ruby ABI gem support in rake-compiler rake-compiler/rake-compiler#257

Merged

OughtPuts marked this pull request as ready for review September 18, 2026 11:48

OughtPuts marked this pull request as draft September 18, 2026 11:48

OughtPuts mentioned this pull request Sep 18, 2026

Support content addressable gems in Gem::PackageTask ruby/rubygems#9897

Merged

github-actions Bot added the cla-needed label Oct 8, 2026

girachawda force-pushed the ho/build-skinny-ca-gems branch from 54fefd3 to 4459c6a Compare October 8, 2026 00:08

github-actions Bot removed the cla-needed label Oct 8, 2026

girachawda force-pushed the ho/build-skinny-ca-gems branch 2 times, most recently from 65c193b to a9602c2 Compare October 8, 2026 00:35

Add support for packaging single-ABI gems

13d2b7e

Co-authored-by: Gira Chawda <64598936+girachawda@users.noreply.github.com>

girachawda force-pushed the ho/build-skinny-ca-gems branch from a9602c2 to 13d2b7e Compare October 9, 2026 01:32

girachawda reviewed Oct 9, 2026

View reviewed changes

girachawda marked this pull request as ready for review October 9, 2026 01:45

girachawda requested a review from Edouard-chin October 9, 2026 01:48

Edouard-chin approved these changes Oct 12, 2026

View reviewed changes

OughtPuts commented Oct 12, 2026

View reviewed changes

Conversation

OughtPuts commented Sep 1, 2026 • edited by girachawda Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TL;DR

Summary

Tophatting

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

OughtPuts commented Sep 18, 2026

Uh oh!

girachawda Oct 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

OughtPuts Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

Edouard-chin left a comment

Choose a reason for hiding this comment

Uh oh!

OughtPuts Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

OughtPuts Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

OughtPuts Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

OughtPuts Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

OughtPuts Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

OughtPuts commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

OughtPuts Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

OughtPuts commented Sep 1, 2026 •

edited by girachawda

Loading

girachawda Oct 9, 2026 •

edited

Loading

OughtPuts commented Oct 12, 2026 •

edited

Loading