Skip to content

fix: fix URL encoding for non-ASCII bytes - #738

Merged
wgtmac merged 1 commit into
apache:mainfrom
fallintoplace:fix/url-encoder-high-bit-bytes
Jun 15, 2026
Merged

fix: fix URL encoding for non-ASCII bytes#738
wgtmac merged 1 commit into
apache:mainfrom
fallintoplace:fix/url-encoder-high-bit-bytes

Conversation

@fallintoplace

Copy link
Copy Markdown
Contributor

Summary

  • Treat URL encoder input bytes as unsigned before splitting them into hex nibbles.
  • Add UTF-8 regression coverage for café encoding to caf%C3%A9.

Why

On platforms where char is signed, bytes >= 0x80 can become negative before the hex lookup. That can lead to undefined behavior or incorrect percent encoding for non-ASCII strings.

Testing

  • cmake -S . -B build -G Ninja -DICEBERG_BUILD_BUNDLE=OFF -DICEBERG_BUILD_REST=OFF -DICEBERG_BUILD_HIVE=OFF -DICEBERG_BUILD_SQL_CATALOG=OFF
  • cmake --build build --target util_test
  • ctest --test-dir build -R util_test --output-on-failure

@zhjwpku zhjwpku left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, for non-ASCII input bytes, kHexChars[c >> 4] can lead to out-of-bounds access and undefined behavior. +1 for this fix.

@wgtmac wgtmac changed the title Fix URL encoding for non-ASCII bytes fix: fix URL encoding for non-ASCII bytes Jun 15, 2026
@wgtmac
wgtmac merged commit 5eb32df into apache:main Jun 15, 2026
20 checks passed
@wgtmac

wgtmac commented Jun 15, 2026

Copy link
Copy Markdown
Member

Thanks @fallintoplace for fixing this and @zhjwpku for the review!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants