How to override HTTP timeouts per-request in AWS S3 SDK?

[ClearlyClaire]

Hi!

In our application, we have low HTTP timeouts for S3 queries because in some code paths, we expect the operations to be fast and can't afford to let them block processing.

However, we also have code paths were it is safe for us to wait for much longer, and we do have operations that we know can be slow.

Therefore I'm looking into how to override the HTTP timeouts for these specific requests. The description of http_read_timeout in https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html makes me think this should be possible (emphasis mine), but I couldn't find how:

The default number of seconds to wait for response data. This value can safely be set per-request on the session.

Is it possible to override the http_read_timeout config on a per-request basis for an existing Aws::S3::Client, or do we have to create a new instance of a client?

[jterapin]

Hi! Based on my light investigation, I observed the following:

[1] pry(Aws)> client = Aws::S3::Client.new
=> #<Aws::S3::Client>
[2] pry(Aws)> client.config.http_open_timeout
=> 15
[3] pry(Aws)> client.put_object({bucket: 'some-bucket', key: "foo", body: 'bar'})
Session open timeout is: 15
[Aws::S3::Client 200 0.200642 0 retries] put_object(bucket:"some-bucket",key:"foo",body:"bar",checksum_algorithm:"CRC32")  
=> #<struct Aws::S3::Types::PutObjectOutput ... 
[4] pry(Aws)> client.config.http_open_timeout=10
[4] pry(Aws)> client.put_object({bucket: 'some-bucket', key: "foo", body: 'bar'})
Session open timeout is: 10
[Aws::S3::Client 200 0.165705 0 retries] put_object(bucket:"some-bucket",key:"foo",body:"bar",checksum_algorithm:"CRC32")  
=> #<struct Aws::S3::Types::PutObjectOutput .. 

This completely overrides the client config, so you'd need to create a new client instance. However, this could be a feature that we could consider adding in the next MV.

[ClearlyClaire]

Thanks for the prompt reply!

This is a bit cumbersome because we don't initialize the clients ourselves but do that through a library that does not give us a lot of control in that regard, so we'd need to reimplement a lot of it.

That being said, I did not think the config was mutable, and since that library creates thread-scoped clients, we could temporarily change the config and restore it after the query in an ensure block.

Having a clean way to do it without mutating the config or creating a new client instance from scratch would be very welcome, though. Thanks!