Fix request_signer body serialization and wildcard double-encoding - #336
Open
xluo-aws wants to merge 1 commit into
Open
Fix request_signer body serialization and wildcard double-encoding#336xluo-aws wants to merge 1 commit into
xluo-aws wants to merge 1 commit into
Conversation
xluo-aws
requested review from
VachaShah,
VijayanB,
harshavamsi,
nhtruong,
robsears and
vamshin
as code owners
July 25, 2026 02:15
Fix two bugs before 4.0.0 release: 1. request_signer body mismatch (opensearch-project#311) The signer was receiving the raw body object (Hash/nil) while the Faraday transport serialized it separately via __convert_to_json. This meant the signer computed a signature over different bytes than what was actually sent, producing 403 errors on signed requests with a body. Fix: serialize the body through the transport serializer before passing it to sign_request, mirroring what the transport does. String bodies are passed through unchanged. 2. Wildcard double-encoding in index params (opensearch-project#319) normalize_value used CGI.escape which encodes '*' -> '%2A'. The HTTP layer then percent-encodes '%' -> '%25', so OpenSearch received 'test-%2A' literally instead of the intended wildcard. Fix: switch to URI::DEFAULT_PARSER.escape with a safe set that preserves '*' and other characters meaningful in OpenSearch path patterns. Spaces are now encoded as '%20' rather than '+', which is correct for URL paths. Both fixes include updated/new tests. Signed-off-by: Xuesong Luo <lxuesong@amazon.com>
xluo-aws
force-pushed
the
fix/signer-and-wildcard-encoding
branch
from
July 25, 2026 02:37
2aaa1e6 to
4866a6a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes two bugs that should be resolved before the 4.0.0 release.
Fix 1 — request_signer body serialization mismatch (#311)
The `request_signer` feature (added in 4.0 beta via #281) was passing the raw Ruby `body` object (a Hash) to `sign_request`. However, the Faraday transport then serialized the body separately via `__convert_to_json`. When these two serializations produced different byte strings (different key order, custom serializer, etc.), the resulting signature didn't match the wire body, causing 403 errors on any signed request with a body.
Fix: serialize the body through the transport's own serializer before passing it to `sign_request`, mirroring exactly what the transport sends on the wire. String bodies are passed through unchanged.
Fix 2 — wildcard/special character double-encoding in index params (#319)
`normalize_value` used `CGI.escape` which encodes `*` → `%2A`. The HTTP layer then percent-encodes the `%` → `%25`, so OpenSearch received `test-%2A` literally instead of treating it as a wildcard, causing `index_not_found_exception` errors.
Fix: switch to `URI::DEFAULT_PARSER.escape` with a safe-character set that preserves `*` and other characters meaningful in OpenSearch path patterns. Spaces are encoded as `%20` (correct for URL paths) rather than `+`.
Issues Resolved
Fixes #311
Fixes #319
Testing
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.