rubygems: Add PQC ML-DSA support for cryptographically signed gems workflow by junaruga · Pull Request #9697 · ruby/rubygems

junaruga · 2026-07-15T10:53:03Z

This PR is related to #9542, and to add PQC ML-DSA support for cryptographically signed gems workflow.
The PR has 6 commits. The 1st - 5th commits are not related to PQC, but for preparation to implement the 6th commit, the main commit of this PR. The PR's 6th commit is in draft phase. I want to share my early phase of the implementation to get feedback and adjust the direction. I hope #9678 will be reviewed and merged before this PR.

Proof of concept

I prepared proof-of-concept scripts for signed gems workflow.

The document is here.
The script is here.
The CI result is here.

Commit message

These changes enable the full PQC ML-DSA cryptographically signed
gems workflow: gem cert --build (key/cert generation), gem build
(package signing), and gem install -P HighSecurity (signature
verification).

The gem cert -A accepts ML-DSA-44, ML-DSA-65, or ML-DSA-87 to generate
ML-DSA based cert and key.

$ gem cert --build your@email.com -A ML-DSA-44
$ gem cert --build your@email.com -A ML-DSA-65
$ gem cert --build your@email.com -A ML-DSA-87

The reason why it accepts all 3 ML-DSA parameter sets rather than one of them
is because all 3 ML-DSA parameter sets suit different workflows.
While ML-DSA-65 can be used commonly, a high security environment requires
ML-DSA-87.

ML-DSA-44: NIST security strength category 2, signature size 2420 bytes
ML-DSA-65: NIST security strength category 3, signature size 3309 bytes
ML-DSA-87: NIST security strength category 5, signature size 4627 bytes

See NIST FIPS 204 Section 4 (Parameter Sets).
https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.204.pdf

See Security (Evaluation Criteria) - 4.A.5 Security Strength Categories.
https://csrc.nist.gov/projects/post-quantum-cryptography/post-quantum-cryptography-standardization/evaluation-criteria/security-(evaluation-criteria)

The workflow also includes

gem cert -C cert.pem
gem cert -K private_key.pem
gem build gemname.gemspec with s.cert_chain and s.signing_key to build
signed gem.
gem install gemname-version.gem -P HighSecurity # or -P MediumSecurity
gem update gemname-version.gem -P HighSecurity # or -P MediumSecurity

The changes come from:

OpenSSL::PKey::RSA, OpenSSL::PKey::DSA, OpenSSL::PKey::EC for RSA, DSA, EC
cases, vs OpenSSL::PKey::PKey for ML-DSA-NN cases. OpenSSL::PKey::PKey
doesn't have the methods #to_pem, #private?.
Ruby OpenSSL methods #sign (signing) and #verify don't accept digest
algorithm, in ML-DSA cases because ML-DSA has a built-in digest
Unify tool/create_certs.rb with tool/create_encrypted_key.rb
to generate both RSA and ML-DSA-65 certificates/keys.
ML-DSA-65 related .pem files generated by tool/create_certs.rb.
Adding ML-DSA tests. We need lots of tests because of differences between
OpenSSL::PKey::RSA/DSA/EC and OpenSSL::PKey::PKey mentioned above.

The method chains for the changes are below.

gem cert --build (key/cert generation)
- lib/rubygems/security.rb self.sign
  - chain: execute -> build -> build_cert ->
    Gem::Security.create_cert_email -> create_cert_self_signed ->
    sign
- lib/rubygems/security.rb self.write
  - chain: execute -> build -> build_key -> Gem::Security.write
gem cert -K (for --sign or --build)
- lib/rubygems/commands/cert_command.rb open_private_key
  - chain: open_private_key
gem build (package signing)
- lib/rubygems/security/signer.rb sign
  - chain: execute -> build_gem -> build_package ->
    Gem::Package.build -> Package#build -> add_metadata ->
    TarWriter#add_file_signed -> signer.sign
gem install -P HighSecurity (signature verification)
- lib/rubygems/security/policy.rb check_trust
  - chain: execute -> install_gems -> install_gem ->
    RequestSet#install -> Installer#install ->
    pre_install_checks -> verify_spec -> Package#spec ->
    Package#verify -> Policy#verify_signatures ->
    Policy#verify -> check_trust
- lib/rubygems/security/policy.rb check_data
  - chain: (same as above up to Policy#verify) -> check_data

See also https://guides.rubygems.org/security/ for the signed gems workflows.

Assisted-by: Claude:claude-opus-4-6[1m]

Notes & challenges

Differences between `OpenSSL::PKey::RSA/DSA/EC` and `OpenSSL::PKey::PKey`

OpenSSL::PKey::RSA, OpenSSL::PKey::DSA, OpenSSL::PKey::EC for RSA, DSA, EC cases, vs OpenSSL::PKey::PKey for ML-DSA-NN cases. OpenSSL::PKey::PKey doesn't have the methods #to_pem, #private?.

The above situation makes the changes complicated. Ideally I want the new feature OpenSSL::PKey::ML-DSA class or OpenSSL::PKey::ML-DSA-NN classes, or OpenSSL::PKey::PKey to implement #to_pem and #private? methods in Ruby OpenSSL. I am thinking to implement the following classes in the meantime.

Gem::OpenSSL::PKey::ML_DSA class which is child class of OpenSSL::PKey::PKey for the compatibility with OpenSSL::PKey::RSA and etc. The class can have #to_pem and #private?.
Gem::OpenSSL::PKey::PKey class to implement self.generate_key which is wrapper of OpenSSL::PKey.generate_key, but can return Gem::OpenSSL::PKey::ML_DSA.

After OpenSSL::PKey::ML-DSA is implemented, we can use use OpenSSL::PKey::ML-DSA if OpenSSL::PKey::ML-DSA is defined. Otherwise can use Gem::OpenSSL::PKey::ML_DSA.

or

Gem::OpenSSL::PKey::PKey class which is child class of OpenSSL::PKey::PKey for the compatibility with OpenSSL::PKey::RSA and etc. The class can have #to_pem and #private?.
Gem::OpenSSL::PKey::PKey class to implement self.generate_key which is wrapper of OpenSSL::PKey.generate_key, but can return Gem::OpenSSL::PKey::PKey.

Constant management in test files

test/rubygems/helper.rb defines KEY/CERT constants.

However, for example, the following files also defines such constants. I also saw duplicated constants from the ones in helper.rb.

test/rubygems/test_gem_commands_cert_command.rb
test/rubygems/test_gem_security.rb
test/rubygems/test_gem_security_policy.rb
test/rubygems/test_gem_security_signer.rb

These constants can be managed in one place.

omit_if_support_pqc

I added omit_if_support_pqc to be used in non-PQC specific tests. However, so far it is not used in any places. I will delete the method, minimizing the code changes if it is not used eventually. I used omit_if_support_pqc for an error case in non-PQC tests. However, it was hard to handle OpenSSL versions and Ruby OpenSSL versions in the error case. So, I deleted the non-PQC tests.

What was the end-user or developer problem that led to this PR?

Users cannot use ML-DSA for signed gem workflow.
https://guides.rubygems.org/security/

What is your fix for the problem, implemented in this PR?

Added the implementation to add PQC ML-DSA support for cryptographically signed gems workflow.

Make sure the following tasks are checked

Describe the problem / feature
Write tests for features and bug fixes
Write code to solve the problem
Make sure you follow the current code style and write meaningful commit messages without tags

The following help text and error message in lib/rubygems/security.rb show that the order of algorithms is RSA, DSA, EC. Align the case branches in create_key with this order. ``` -A, --key-algorithm ALGORITHM Select key algorithm for --build from RSA, DSA, or EC. Defaults to RSA. ``` ``` "#{algorithm} algorithm not found. RSA, DSA, and EC algorithms are supported." ``` Assisted-by: Claude:claude-opus-4-6[1m]

The create_certs.rb created test/rubygems/expired_cert_32.pem which was not used and managed in git repository. When not_before and not_after are the same between the regular and 32-bit certificates, don't create 32-bit certificate. Because it is redundant. Assisted-by: Claude:claude-opus-4-6[1m]

* Move omit_unless_support_pqc from test_gem_remote_fetcher_local_ssl_server.rb to helper.rb so that all rubygems test files can use it * Create test/rubygems/pqc_utilities.rb to manage PQC utilities * Update local_ssl_server_utilities.rb to require and include pqc_utilities.rb Assisted-by: Claude:claude-opus-4-6[1m]

* Rename test/rubygems/private_ec_key.pem to test/rubygems/ec_private_key.pem to align with the pattern <algorithm>_key_name.pem (e.g., mldsa65_ssl_key.pem) * Rename EC_KEY to EC_PRIVATE_KEY in test_gem_security.rb * Rename PRIVATE_EC_KEY_FILE to EC_PRIVATE_KEY_FILE in test_gem_commands_cert_command.rb Assisted-by: Claude:claude-opus-4-6[1m]

…rkflow These changes enable the full PQC ML-DSA cryptographically signed gems workflow: `gem cert --build` (key/cert generation), `gem build` (package signing), and `gem install -P HighSecurity` (signature verification). The `gem cert -A` accepts `ML-DSA-44`, `ML-DSA-65`, or `ML-DSA-87` to generate ML-DSA based cert and key. ``` $ gem cert --build your@email.com -A ML-DSA-44 $ gem cert --build your@email.com -A ML-DSA-65 $ gem cert --build your@email.com -A ML-DSA-87 ``` The reason why it accepts all 3 ML-DSA parameter sets rather than one of them is because all 3 ML-DSA parameter sets suit different workflows. While ML-DSA-65 can be used commonly, a high security environment requires ML-DSA-87. ML-DSA-44: NIST security strength category 2, signature size 2420 bytes ML-DSA-65: NIST security strength category 3, signature size 3309 bytes ML-DSA-87: NIST security strength category 5, signature size 4627 bytes See NIST FIPS 204 Section 4 (Parameter Sets). https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.204.pdf See Security (Evaluation Criteria) - 4.A.5 Security Strength Categories. https://csrc.nist.gov/projects/post-quantum-cryptography/post-quantum-cryptography-standardization/evaluation-criteria/security-(evaluation-criteria) The workflow also includes * `gem cert -C cert.pem` * `gem cert -K private_key.pem` * `gem build gemname.gemspec` with `s.cert_chain` and `s.signing_key` to build signed gem. * `gem install gemname-version.gem -P HighSecurity` # or `-P MediumSecurity` * `gem update gemname-version.gem -P HighSecurity` # or `-P MediumSecurity` The changes come from: * `OpenSSL::PKey::RSA`, `OpenSSL::PKey::DSA`, `OpenSSL::PKey::EC` for RSA, DSA, EC cases, vs `OpenSSL::PKey::PKey` for ML-DSA-NN cases. `OpenSSL::PKey::PKey` doesn't have the methods `#to_pem`, `#private?`. * Ruby OpenSSL methods `#sign` (signing) and `#verify` don't accept digest algorithm, in ML-DSA cases because ML-DSA has a built-in digest * Unify tool/create_certs.rb with tool/create_encrypted_key.rb to generate both RSA and ML-DSA-65 certificates/keys. * ML-DSA-65 related .pem files generated by tool/create_certs.rb. * Adding ML-DSA tests. We need lots of tests because of differences between `OpenSSL::PKey::RSA/DSA/EC` and `OpenSSL::PKey::PKey` mentioned above. The method chains for the changes are below. * gem cert --build (key/cert generation) * lib/rubygems/security.rb self.sign * chain: execute -> build -> build_cert -> Gem::Security.create_cert_email -> create_cert_self_signed -> sign * lib/rubygems/security.rb self.write * chain: execute -> build -> build_key -> Gem::Security.write * gem cert -K (for --sign or --build) * lib/rubygems/commands/cert_command.rb open_private_key * chain: open_private_key * gem build (package signing) * lib/rubygems/security/signer.rb sign * chain: execute -> build_gem -> build_package -> Gem::Package.build -> Package#build -> add_metadata -> TarWriter#add_file_signed -> signer.sign * gem install -P HighSecurity (signature verification) * lib/rubygems/security/policy.rb check_trust * chain: execute -> install_gems -> install_gem -> RequestSet#install -> Installer#install -> pre_install_checks -> verify_spec -> Package#spec -> Package#verify -> Policy#verify_signatures -> Policy#verify -> check_trust * lib/rubygems/security/policy.rb check_data * chain: (same as above up to Policy#verify) -> check_data See also https://guides.rubygems.org/security/ for the signed gems workflows. Assisted-by: Claude:claude-opus-4-6[1m]

junaruga · 2026-07-15T13:01:56Z

I added "Proof of concept" section to the first comment.

junaruga · 2026-07-15T13:18:14Z

Differences between OpenSSL::PKey::RSA/DSA/EC and OpenSSL::PKey::PKey

For the above topic, I opened ruby/openssl#1085 for feature request.

rhenium · 2026-07-15T16:06:36Z

    save_cert = OpenSSL::X509::Certificate.new File.read path
-    save_dgst = digester.digest save_cert.public_key.to_pem
+    save_pkey = save_cert.public_key
+    save_pkey_str = Gem::Security.pemmable_to_pem(save_pkey)
+    save_dgst = digester.digest(save_pkey_str)

-    pkey_str = root.public_key.to_pem
-    cert_dgst = digester.digest pkey_str
+    root_pkey = root.public_key
+    root_pkey_str = Gem::Security.pemmable_to_pem(root_pkey)
+    cert_dgst = digester.digest(root_pkey_str)

    raise Gem::Security::Exception,
          "trusted root certificate #{root.subject} checksum " \
          "does not match signing root certificate checksum" unless
      save_dgst == cert_dgst


Can this be simplified like this?

However this is clearly checking the subject public key, not the certificate itself. (I haven't read the rest of the code, so I don't know what this implies.)

raise Gem::Security::Exception, "trusted root certificate #{root.subject} checksum " \ "does not match signing root certificate checksum" unless save_cert.public_key.to_der == root.public_key.to_der

rhenium · 2026-07-15T16:07:18Z

-    raise Gem::OptionParser::InvalidArgument,
-      "#{key_file}: private key not found" unless key.private?
+    # ML-DSA keys (OpenSSL::PKey::PKey) do not have private? method.
+    # Try private_to_pem instead to check if the key has private data.
+    if key.respond_to?(:private?)
+      raise Gem::OptionParser::InvalidArgument,
+        "#{key_file}: private key not found" unless key.private?
+    else
+      begin
+        key.private_to_pem
+      rescue OpenSSL::PKey::PKeyError
+        raise Gem::OptionParser::InvalidArgument,
+          "#{key_file}: private key not found"
+      end
+    end


I wonder if we can just remove this and instead check for an exception that may be raised by OpenSSL::X509::Certificate#sign later.

This may be possible, by checking an exception raised at a later step, and raise new exception with actionable error message.

junaruga · 2026-07-16T10:55:26Z

+        rescue OpenSSL::PKey::PKeyError
+          # public_to_pem doesn't have cipher and passphrase arguments
+          pemmable.public_to_pem
+        end


@rhenium Note this logic begin .. rescue OpenSSL::PKey::PKeyError .. end is because OpenSSL::PKey::PKey doesn't have #private?.

Is a public pkey ever passed to this method?

Yes, test_class_pemmable_to_pem_rsa_public_key in test/rubygems/test_gem_security.rb is to test Gem::Security.pemmable_to_pem with a public key.

The method self.pemmable_to_pem that you mentioned is what I created in this PR. The reason why I created this method is mainly to deal with the current implementation of self.write in lib/rubygems/security.rb. The pemmable can be key OpenSSL::PKey::{RSA,DSA,EC} or certificate OpenSSL::X509::Certificate. But when I investigated a bit in the past, there might be no case that pemmable was public key for the self.write.

rubygems/lib/rubygems/security.rb

Lines 592 to 604 in 7288145

def self.write(pemmable, path, permissions = 0o600, passphrase = nil, cipher = KEY_CIPHER)

path = File.expand_path path

File.open path, "wb", permissions do |io|

if passphrase && cipher

io.write pemmable.to_pem cipher, passphrase

else

io.write pemmable.to_pem

end

end

path

end

I think I need to refactor the following things in in ruby/rubygems.

Split Gem::Security.write to Gem::Security.write_private_key and Gem::Security.write_certificate.

Replace OpenSSL::PKey::{RSA,DSA,EC}'s #to_pem with #public_to_pem or #private_to_pem. Continue to use OpenSSL::X509::Certificate#to_pem.

Remove #private? to deal with OpenSSL::PKey::PKey which doesn't have #private?.

I'm not sure Gem::Security.write is considered a public API of RubyGems or not, but I think the current method signature is questionable, since the passphrase and cipher arguments are only meaningful when exporting private pkeys.

Split Gem::Security.write to Gem::Security.write_private_key and Gem::Security.write_certificate.

So this makes a lot of sense to me.

Replace OpenSSL::PKey::{RSA,DSA,EC}'s #to_pem with #public_to_pem or #private_to_pem. Continue to use OpenSSL::X509::Certificate#to_pem.

I also think this is a good idea. I think it's an acceptable change, but it's perhaps worth noting that a_rsa_priv_key.to_pem and a_rsa_priv_key.private_to_pem produce different formats, with the latter using a more common format (PKCS#8). OpenSSL::PKey.read has supported both formats since the beginning.

junaruga · 2026-07-17T18:13:59Z

Differences between OpenSSL::PKey::RSA/DSA/EC and OpenSSL::PKey::PKey

For the above challenge, I sent the PR #9701 based on the discussion on this PR. I think the PR #9701 can be a solution for the challenge. I hope the PR #9701 is reviewed before this PR. Then I can rebase this PR on the latest master branch.

junaruga force-pushed the wip/rubygems-pqc-signed-gem branch from 8157611 to a558720 Compare July 15, 2026 11:49

junaruga mentioned this pull request Jul 15, 2026

RubyGems: Adding post-quantum cryptography (PQC) support #9542

Open

junaruga changed the title ~~Add PQC ML-DSA support for cryptographically signed gems workflow~~ rubygems: Add PQC ML-DSA support for cryptographically signed gems workflow Jul 15, 2026

junaruga force-pushed the wip/rubygems-pqc-signed-gem branch from a558720 to 4116303 Compare July 15, 2026 12:04

junaruga added 5 commits July 15, 2026 13:23

security.rb: Improve self.write code comment.

c5adfbb

junaruga force-pushed the wip/rubygems-pqc-signed-gem branch from 4116303 to acf46a0 Compare July 15, 2026 12:29

junaruga force-pushed the wip/rubygems-pqc-signed-gem branch from acf46a0 to 23850b4 Compare July 15, 2026 12:46

junaruga mentioned this pull request Jul 15, 2026

Feature request: OpenSSL::PKey::PKey, OpenSSL::PKey::ML-DSA, or OpenSSL::PKey::ML-DSA-NN to implement #private? and #to_pem ruby/openssl#1085

Open

rhenium reviewed Jul 15, 2026

View reviewed changes

junaruga commented Jul 16, 2026

View reviewed changes

junaruga mentioned this pull request Jul 17, 2026

rubygems: Replace OpenSSL::PKey::{RSA,DSA,EC}#to_pem and remove #private? #9701

Open

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

rubygems: Add PQC ML-DSA support for cryptographically signed gems workflow#9697

rubygems: Add PQC ML-DSA support for cryptographically signed gems workflow#9697
junaruga wants to merge 6 commits into
ruby:masterfrom
junaruga:wip/rubygems-pqc-signed-gem

junaruga commented Jul 15, 2026 •

edited

Loading

Uh oh!

junaruga commented Jul 15, 2026 •

edited

Loading

Uh oh!

junaruga commented Jul 15, 2026

Uh oh!

rhenium Jul 15, 2026

Uh oh!

rhenium Jul 15, 2026

Uh oh!

junaruga Jul 16, 2026

Uh oh!

junaruga Jul 16, 2026 •

edited

Loading

Uh oh!

rhenium Jul 16, 2026

Uh oh!

junaruga Jul 16, 2026 •

edited

Loading

Uh oh!

rhenium Jul 16, 2026

Uh oh!

junaruga commented Jul 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	def self.write(pemmable, path, permissions = 0o600, passphrase = nil, cipher = KEY_CIPHER)
	path = File.expand_path path

	File.open path, "wb", permissions do \|io\|
	if passphrase && cipher
	io.write pemmable.to_pem cipher, passphrase
	else
	io.write pemmable.to_pem
	end
	end

	path
	end

Uh oh!

Uh oh!

Conversation

junaruga commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proof of concept

Commit message

Notes & challenges

Differences between OpenSSL::PKey::RSA/DSA/EC and OpenSSL::PKey::PKey

Constant management in test files

omit_if_support_pqc

What was the end-user or developer problem that led to this PR?

What is your fix for the problem, implemented in this PR?

Make sure the following tasks are checked

Uh oh!

junaruga commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

junaruga commented Jul 15, 2026

Uh oh!

rhenium Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

rhenium Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

junaruga Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

junaruga Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rhenium Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

junaruga Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rhenium Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

junaruga commented Jul 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

junaruga commented Jul 15, 2026 •

edited

Loading

Differences between `OpenSSL::PKey::RSA/DSA/EC` and `OpenSSL::PKey::PKey`

junaruga commented Jul 15, 2026 •

edited

Loading

junaruga Jul 16, 2026 •

edited

Loading

junaruga Jul 16, 2026 •

edited

Loading