Repository navigation
Conversation
rhenium
force-pushed
the
ky/ssl-callback-errors
branch
2 times, most recently
from
August 31, 2026 12:32
a05a89f to
5ae433b
Compare
Simplify handling of the temporary jump tag. As a side effect, this also avoids corruption of OpenSSL's internal state if the SSLSocket object is frozen in the middle of SSL_*() calls.
Do not invoke another Ruby callback while we are waiting for a chance to do rb_jump_tag(). As far as I know, this cannot currently happen. Handle it explicitly in preparation for an upcoming patch that uses a custom BIO_METHOD backed by Ruby methods in SSLSocket. These callbacks may be called after Ruby code called from BIO_METHOD has failed, or vice versa.
Use rb_protect() to prevent exiting from SSL_accept() prematurely. Also set the callback to SSL_CTX instead of SSL, similarly to what we do for other callbacks configurable on SSLContext.
Currently, if the callback raises an exception or returns an invalid value, the handshake is resumed as if the callback has not been called. This is contrary to the documentation. Propagate exceptions raised by the callback to the originating method. If the callback returns an invalid value, raise TypeError. It is immediately caught by rb_protect() and will be re-raised in the originating method in the same way.
Prepare the Array objects for the argument inside rb_protect(). This object allocation may potentially raise an exception and corrupt OpenSSL's internal state. Also look up the callback Proc objects directly from SSLContext's instance variables to simplify.
rhenium
force-pushed
the
ky/ssl-callback-errors
branch
from
October 13, 2026 09:58
5ae433b to
11bf938
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extracted from #736
ssl: add struct ossl_ssl_data
We will need to store additional internal states in
SSLSocketin the upcoming patches. Prepare a new struct for such purposes and store it inSSL'sex_data.Refactor existing internal states that are local to C to use this new struct:
ex_datassl: do not call another callback when an exception is pending
Do not attempt to execute Ruby callback when another callback has already raised an exception and we are waiting for a chance to call
rb_jump_tag().As far as I know, this currently cannot happen. However, an upcoming patch to use custom
BIO_METHODinSSLSocketmay make it possible. Let's handle it explicitly.ssl: refactor info_callback
Set the callback to
SSL_CTXinstead ofSSLso that we do not have to configure everySSLobject, similar to all other existing callbacks.Also, wrap
rb_funcall()inrb_protect()to prevent exiting fromSSL_accept()prematurely.ssl: re-raise exceptions in client_cert_cb
Re-raise exceptions raised inside
client_cert_cbfrom the originating Ruby method, which can be any of#connect,#sysread*, or#syswrite*.ssl: simplify looking up callback procs for session_{get,new}_cb
Look up the callback procs by checking the corresponding instance variables on
SSLContextdirectly. The removed methodsSSLSocket#session_new_cband#session_get_cbare private and have not been part of the public interface of ruby/openssl.Also remove the unused method
SSLSocket#client_cert_cb.ssl: refactor session_{get,new,remove}_cb
Prepare the argument array object inside
rb_protect(). The callback proc is already called in it, but the array allocation can also potentially fail and needs to be done there.