Skip to content

[fix](function) Bound parse_url's HOST, PORT and USERINFO at the authority - #68370

Open
arpitjain099 wants to merge 1 commit into
apache:masterfrom
arpitjain099:fix/parse-url-authority-bounds
Open

arpitjain099 wants to merge 1 commit into
apache:masterfrom
arpitjain099:fix/parse-url-authority-bounds

Conversation

@arpitjain099

@arpitjain099 arpitjain099 commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: none

Problem Summary:

parse_url hunts for the ':' that separates the port, and the '@' that separates the userinfo, across the whole url rather than across its authority. Anything in the path, the query or the fragment gets picked up as a separator:

select parse_url('http://example.com/a:b', 'HOST');       -- example.com/a
select parse_url('http://example.com/a:b', 'PORT');       -- b
select parse_url('http://example.com/a@b:c', 'USERINFO'); -- example.com/a
select parse_url('http://example.com?x=1', 'AUTHORITY');  -- example.com?x=1

A ':' inside a path segment is perfectly legal (RFC 3986 pchar), and Hive answers these through java.net.URL, which gives host example.com, no port and no userinfo for all four.

Both of our implementations do it, since the FE constant folding path in StringArithmetic.parseUrlRaw mirrors the BE UrlParser closely. AUTHORITY was the one already close to right, because it cut at the first '/', so HOST, PORT and USERINFO now go through that same step and it is extended to stop at '?' and '#' as well.

Release note

parse_url no longer treats a ':' or an '@' outside the authority as a port or userinfo separator, so HOST, PORT, USERINFO and AUTHORITY now agree with java.net.URL on urls whose path, query or fragment contains one of those characters.

Check List (For Author)

  • Test: Unit Test
    • Cases added to be/test/exprs/function/function_url_test.cpp and to StringArithmeticTest.java, one per affected part.
    • I do not have a full BE or FE build on this machine, so I checked the changed code on its own instead: url_parser.cpp compiled and driven standalone, and the FE parseUrl* methods extracted and run under a JDK. Both now agree with java.net.URL on every case in the new tests. CI is the real check here, and I would rather say that than imply I ran the suites.
  • Behavior changed: Yes, the four results above. Existing cases in nereids_function_p0 and fold_constant_string_arithmatic have no ':' or '@' outside the authority, so they should be unaffected.
  • Does this need documentation: No

…ority

### What problem does this PR solve?

Issue Number: none

Problem Summary: parse_url searches for the ':' that separates the port, and
the '@' that separates the userinfo, across the whole url rather than across
its authority component, so a ':' or an '@' in the path, the query or the
fragment is read as a separator. parse_url('http://example.com/a:b', 'HOST')
returns example.com/a and the PORT of the same url returns b. Both the BE
UrlParser and the FE StringArithmetic constant-folding mirror have it. HOST,
PORT and USERINFO now go through the same authority step AUTHORITY used, which
is extended to stop at '?' and '#' as well.

### Release note

parse_url no longer treats a ':' or an '@' outside the authority as a port or
userinfo separator, so HOST, PORT, USERINFO and AUTHORITY agree with
java.net.URL for urls whose path, query or fragment contains one of those
characters.

### Check List (For Author)

- Test: Unit Test
- Behavior changed: Yes, as described in the release note
- Does this need documentation: No

Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@yiguolei

Copy link
Copy Markdown
Contributor

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static review result: ready from the code-review perspective; no blocking findings.

Critical checkpoint conclusions:

  • Goal and correctness: the patch bounds AUTHORITY, HOST, PORT, and USERINFO parsing at the earliest slash, question mark, or hash. The BE runtime path and FE constant-folding path implement the same boundary and preserve matching empty-string versus NULL behavior.
  • Scope and clarity: the change is focused on a shared BE helper plus the mirrored FE helper and their direct tests; unrelated URL parts and function registration are unchanged.
  • Parallel paths and compatibility: both FE folding and BE vectorized execution were updated. The behavior change is limited to separators outside the authority; there is no FE-BE protocol, symbol, serialization, storage-format, or rolling-upgrade concern.
  • Conditions and error handling: all delimiter absence and ordering combinations were checked statically. BE negative-length StringRef slicing intentionally means the remaining full view, and returned views are copied by the caller before input lifetime ends. Invalid-part handling is unchanged.
  • Concurrency and lifecycle: this is stateless scalar parsing. It adds no threads, locks, mutable shared state, cross-TU dependency, or special resource lifecycle.
  • Configuration, persistence, transactions, and data writes: none are involved. No observability addition is needed for this local deterministic helper.
  • Performance: the added work is a bounded set of linear single-byte scans over one URL, allocation-free in BE and planner-time only in FE; no material regression was found.
  • Tests: mirrored BE and FE unit cases cover path, query, and fragment false separators, real userinfo and port values, all four affected parts, and NULL behavior. The author explicitly reports standalone checks rather than full suites. This automated review was static-only, so no build or test result was independently produced.
  • User focus: no additional focus was supplied; the full five-file change and relevant call chains were reviewed.

All normal and risk-focused review passes converged with no new valuable findings. No inline comments were submitted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants