feat(http): support https over proxy via HTTP CONNECT by ithewei · Pull Request #886 · ithewei/libhv

ithewei · 2026-09-21T15:31:27Z

What

Adds HTTPS-over-proxy via HTTP CONNECT to HttpClient, and a reusable
PROXY_PROTOCOL_HTTP_CONNECT at the io layer.

The bug this fixes

The built-in (non-curl) HttpClient advertised setHttpsProxy(), but for an
https:// target it fell back to plain absolute-URI forwarding and skipped
TLS entirely (https = IsHttps() && !IsProxy()). The request went to the proxy
in cleartext and real HTTPS proxies (which only accept CONNECT) rejected
it. http:// forwarding was fine; https:// through a proxy was broken.

Design

https over a proxy must use an HTTP CONNECT tunnel: connect to the proxy,
CONNECT origin:port, then do end-to-end TLS with the origin. This is a
transport-layer tunnel, distinct from plain-HTTP absolute-URI forwarding, so
it reuses the io-layer proxy framework added in #885 (same slot as SOCKS5,
before the SSL handshake).

connect[proxy] -> CONNECT origin:port -> [2xx] -> TLS(origin) -> request

event layer (reusable by any io client)

hloop.h: add PROXY_PROTOCOL_HTTP_CONNECT to proxy_protocol_e.
socks5.{h,c}: http_connect_build_request() — CONNECT authority-form
request + Host + optional Proxy-Authorization: Basic; proxy_conn_t.rbuf
grown to 1024 for HTTP response headers.
nio.c: http_connect_handshake (accumulate until \r\n\r\n, require a 2xx
status), dispatched by proxy_handshake_start. The shared SOCKS5 handshake
helpers were renamed socks5_* → proxy_*. Handshake bytes go through the
raw proxy_send (no upper-layer write_cb, no premature hssl_write), the
same discipline as SOCKS5.

HttpClient

HttpMessage: HttpRequest gains tunnel_proxy_{host,port,username,password}
- SetTunnelProxy()/IsTunnelProxy(), distinct from the plain-HTTP proxy bit.
make_request: https+proxy → SetTunnelProxy (CONNECT tunnel);
http+proxy → absolute-URI forward + optional Basic Proxy-Authorization.
sync: extract http_client_ssl_handshake(); add blocking
http_client_http_connect + http_client_connect_tunnel.
async: doTask connects to the proxy for tunnels; doTaskWithAddr sets
hio_set_proxy(HTTP_CONNECT) + enableSSL(origin SNI); tunnels are never
pooled (the pool is keyed by proxy peeraddr). Also fixes a latent UAF:
snapshot IsTunnelProxy() before successCallback() frees the task/req.
curl path: map the tunnel proxy to CURLOPT_PROXY(+PROXYUSERPWD); curl
issues its own CONNECT. (curl was already correct; unchanged behavior.)
add http_client_set_proxy_auth / HttpClient::setProxyAuth.

API

hv::HttpClient cli;
cli.setHttpProxy("proxy", 8080);      // plain http: absolute-URI forwarding
cli.setHttpsProxy("proxy", 8080);     // https: HTTP CONNECT tunnel
cli.setProxyAuth("user", "pass");     // optional (Basic)

Testing

End-to-end (sync + async) against a CONNECT-capable proxy and an https origin:

case	result
http via forward proxy	200
https via CONNECT proxy	200
https via CONNECT + Basic auth	200
https, wrong proxy creds	rejected (407 → error)

SOCKS5 (#885) regression re-verified: real socks5_proxy_server (ipv4 + domain
target) and a fragmenting fake proxy still deliver data.

make libhv / make socks5_client_test build clean; the changed units carry no
new warnings under -Wall -Wextra.

Scope

Protocols: SOCKS5 (existing) + HTTP CONNECT (new); enum leaves room for more.
CONNECT auth: Basic. The legacy IOCP backend is not wired (unmaintained,
superseded by wepoll), consistent with feat(event): client-side SOCKS5 proxy support at the io layer #885.

The HttpClient built-in (non-curl) path advertised setHttpsProxy but for an https target it fell back to plain absolute-URI forwarding and skipped TLS, so the request was sent in the clear (and rejected by real HTTPS proxies). Add a proper HTTP CONNECT tunnel, reusing the io-layer proxy framework from #885. event layer (reusable by any io client): - hloop.h: add PROXY_PROTOCOL_HTTP_CONNECT to proxy_protocol_e. - socks5.{h,c}: http_connect_build_request() (CONNECT authority-form request + Host + optional Basic Proxy-Authorization); proxy_conn_t rbuf grown to 1024 for HTTP response headers. - nio.c: http_connect_handshake state (accumulate until CRLFCRLF, require 2xx), dispatched by proxy_handshake_start; rename socks5_* handshake helpers to proxy_* since they are now shared. Handshake sends via raw proxy_send (no upper-layer write_cb, no premature hssl_write), matching SOCKS5. HttpClient: - HttpMessage: HttpRequest gains tunnel_proxy_host/port/username/password and SetTunnelProxy()/IsTunnelProxy(), distinct from the plain-HTTP bit. - make_request: https+proxy -> SetTunnelProxy (CONNECT tunnel, end-to-end TLS); http+proxy -> absolute-URI forward + optional Basic Proxy-Authorization. - sync: extract http_client_ssl_handshake(); add blocking http_client_http_connect + http_client_connect_tunnel (connect proxy -> CONNECT -> TLS to origin). - async: doTask connects to the proxy for tunnels; doTaskWithAddr sets hio_set_proxy(HTTP_CONNECT) + enableSSL(origin SNI); tunnels are never pooled. Fix a latent UAF: snapshot IsTunnelProxy() before successCallback() frees the task/req. - curl path: map tunnel proxy to CURLOPT_PROXY(+PROXYUSERPWD); curl issues its own CONNECT. - add http_client_set_proxy_auth / HttpClient::setProxyAuth. Verified end-to-end (sync + async) against a CONNECT-capable proxy and an https origin: http-forward, https-CONNECT, CONNECT+Basic-auth all return 200; bad proxy creds are rejected. SOCKS5 (#885) regression re-checked: real proxy (ipv4/domain target) and fragmented fake proxy still deliver data. Co-authored-by: TRAE CLI <traecli@bytedance.com>

The event layer builds with only -I. -Ibase -Issl -Ievent (core srcdirs), so including util/base64.h broke 'make examples' (fatal: base64.h not found). Inline a minimal base64 encoder in socks5.c for the Proxy-Authorization header instead of depending on util/. Co-authored-by: TRAE CLI <traecli@bytedance.com>

Copilot

Copilot review overview

🟡 Changes recommended

Unresolved critical and moderate findings remain in proxy validation, CONNECT handling, pooling, redirects, IPv6 formatting, and credential state.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 · 5

Open (6)

Synchronous CONNECT formatting mishandles IPv6 · New
CONNECT handling drops pipelined target bytes · New
Buffer too small for maximum proxy credentials · New
Tunnel requests can reuse incompatible pooled connections · New
Proxy mode state is not cleared on redirects or reuse · New
Stale libcurl proxy credentials persist across requests · New

What changed in this PR

Adds HTTPS-over-proxy support using HTTP CONNECT, with synchronous, asynchronous, cURL, and reusable I/O-layer integration.

Changes:

Adds HTTP CONNECT tunneling and Basic proxy authentication.
Integrates tunnel-then-TLS flows across client implementations.
Updates proxy state, pooling, APIs, and documentation.

File	Summary	Final findings
`http/HttpMessage.h`	Adds tunnel proxy fields and APIs.	None
`http/HttpMessage.cpp`	Initializes tunnel proxy state.	None
`http/client/HttpClient.h`	Exposes proxy authentication APIs.	None
`http/client/HttpClient.cpp`	Implements synchronous CONNECT, TLS, cURL, and proxy handling.	Critical (3 votes): Bracket IPv6 literals in CONNECT authority and `Host` headers. Moderate (3 votes): Clear stale cURL proxy credentials. Moderate (3 votes): Recompute and clear mutually exclusive proxy state.
`http/client/AsyncHttpClient.cpp`	Adds asynchronous tunnel setup and lifecycle handling.	Moderate (2 votes): Prevent plain HTTP pool connections from being reused for tunnels. Moderate (1 vote): Recompute proxy selection after redirects.
`event/socks5.h`	Declares shared CONNECT request support.	None
`event/socks5.c`	Builds CONNECT requests with authentication.	Critical (1 vote): Validate target hosts against control characters before interpolation.
`event/nio.c`	Implements nonblocking CONNECT handshakes.	Moderate (2 votes): Size the request buffer for maximum credentials. Moderate (2 votes): Preserve data received after the CONNECT response headers.
`event/hloop.h`	Adds the HTTP CONNECT protocol enum.	None
`event/hevent.c`	Accepts HTTP CONNECT proxy settings.	None
`docs/cn/HttpClient.md`	Documents HTTPS proxy tunneling and authentication.	None

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot

Copilot review overview

🟡 Changes recommended

Unresolved correctness, framing, proxy-state, and credential-handling issues remain.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 4 · 5

Open (9)

Handle partial nonblocking CONNECT request writes · New
Prevent proxy credentials from reaching tunnel origins · New
Clear stale generated Proxy-Authorization headers · New
Synchronous CONNECT formatting mishandles IPv6
Stale libcurl proxy credentials persist across requests
Proxy mode state is not cleared on redirects or reuse
Tunnel requests can reuse incompatible pooled connections
Buffer too small for maximum proxy credentials
CONNECT handling drops pipelined target bytes

Previously missed (1)

In code that hasn't changed since last review

Bracket IPv6 authorities in CONNECT requests

event/socks5.c:121

URL parsing stores IPv6 hosts without brackets, so a target such as https://[::1]:8443 is emitted as CONNECT ::1:8443 and the proxy cannot parse the authority. Format IPv6 authorities as [host]:port for both the request-target and Host header.

Address Copilot review on #886: - nio.c proxy_send: a positive short write is legal TCP behavior (and HTTP CONNECT requests are larger than SOCKS5 messages). Loop until all bytes are sent instead of treating a short write as fatal, backing off briefly on EAGAIN (bounded ~5s, portable hv_msleep rather than poll()). - HttpClient make_request: always erase any prior Proxy-Authorization before deciding the proxy mode. This keeps the credential transport-only: it never crosses a CONNECT tunnel into the origin request (redirect/https reuse), and it no longer lingers after setProxyAuth(NULL, NULL) or when a request object is reused. It is regenerated only in the plain-HTTP forward-proxy branch, where the request is sent to the proxy itself. Re-verified sync + async: http-forward, https-CONNECT, CONNECT+Basic-auth all 200; bad proxy creds rejected. Co-authored-by: TRAE CLI <traecli@bytedance.com>

Copilot

Copilot review overview

🟡 Changes recommended

Unresolved critical and moderate proxy-handling findings remain.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 3 · 5

Open (8)

Handshake sleep blocks the event loop · New
Proxy modes remain active simultaneously · New
Synchronous CONNECT formatting mishandles IPv6
Stale libcurl proxy credentials persist across requests
Proxy mode state is not cleared on redirects or reuse
Tunnel requests can reuse incompatible pooled connections
Buffer too small for maximum proxy credentials
CONNECT handling drops pipelined target bytes

Resolved since last review (3)

Clear stale generated Proxy-Authorization headers
Prevent proxy credentials from reaching tunnel origins
Handle partial nonblocking CONNECT request writes

Previously missed (2)

In code that hasn't changed since last review

Async CONNECT fails for unbracketed IPv6 authorities

event/socks5.c:121

This formats an IPv6 origin as CONNECT 2001:db8::1:443, which is not valid authority-form syntax; IPv6 literals must be enclosed in brackets. The async CONNECT path therefore cannot reach IPv6 HTTPS origins through the proxy. Format the authority as [host]:port when is_ipv6(host) is true.

Tunnel proxy changes are ignored during connection reuse

http/client/HttpClient.cpp:498

For tunnel requests cli->host and cli->port identify the origin, so the reuse condition above does not detect a change to setHttpsProxy() when the origin stays the same. The existing fd can then be reused through the old proxy, bypassing the newly selected proxy and its CONNECT/authentication. Include the tunnel proxy endpoint in the connection identity or force a reconnect when it changes.

+    void SetTunnelProxy(const char* host, int port,
+                        const char* username = NULL, const char* password = NULL) {
+        tunnel_proxy_host = host ? host : "";
+        tunnel_proxy_port = port;
+        tunnel_proxy_username = username ? username : "";


…modes) Round-3 Copilot review on #886: - IPv6 authority form: bracket IPv6 literals as [addr]:port in both the io-layer http_connect_build_request (socks5.c) and the sync http_client_http_connect (HttpClient.cpp) request-target and Host header (RFC 3986). - nio.c http_connect_handshake: MSG_PEEK to locate the \r\n\r\n terminator, then drain exactly the header bytes, leaving any post-terminator bytes in the socket. A server-first origin protocol (SMTP/IMAP/FTP greeting) arriving in the same segment as the CONNECT response is no longer swallowed. - nio.c http_connect_start: request buffer 1024 -> 2048 so a max-size (255/255) credential's Basic header cannot be rejected as truncation. - AsyncHttpClient: tunnels bypass conn_pool LOOKUP (not just insertion), so a pooled plain-HTTP connection to the same proxy addr is never reused for a tunnel (wrong-transport bug). - Proxy modes made mutually exclusive: SetProxy clears tunnel_*, SetTunnelProxy clears the proxy bit; make_request resets both modes + Proxy-Authorization before routing each request/redirect, so a reused HttpRequest can't carry a stale route across a redirect / no_proxy host / scheme switch. - curl path: reset CURLOPT_PROXY/PROXYUSERPWD each request (reused easy handle otherwise leaks prior proxy creds to a new/void proxy). - Revert proxy_send to a single send() (small handshake msg, empty sndbuf); the previous EAGAIN hv_msleep loop could stall the event-loop thread. Verified: sync + async http-forward, https-CONNECT, CONNECT+Basic-auth all 200; bad proxy creds rejected (-1015, proxy logs auth_ok=False). Co-authored-by: TRAE CLI <traecli@bytedance.com>

Copilot

Copilot review overview

🔵 Needs a closer look

Four moderate findings affect redirect routing, proxy-state rebuilding, connection reuse, and synchronous redirect handling.

Review effort: Lite
Findings: 1

Open (1)

Proxy modes remain active simultaneously

Resolved since last review (7)

Handshake sleep blocks the event loop
Synchronous CONNECT formatting mishandles IPv6
Stale libcurl proxy credentials persist across requests
Proxy mode state is not cleared on redirects or reuse
Tunnel requests can reuse incompatible pooled connections
Buffer too small for maximum proxy credentials
CONNECT handling drops pipelined target bytes

Previously missed (2)

In code that hasn't changed since last review

Redirects reuse stale proxy routing

http/client/AsyncHttpClient.cpp:189

The redirect path requeues ctx->task through doTask(), which only calls ParseUrl() and never reruns http_client_make_request(). Thus the proxy state from the original URL survives redirects: an HTTPS-tunnel request redirected to HTTP or a no-proxy host still CONNECTs through the old proxy, while an HTTP forward-proxy request redirected to HTTPS can remain a cleartext absolute-URI request. Recompute the route and clear stale proxy credentials before requeueing the redirected task.

Connection reuse ignores proxy route changes

http/client/HttpClient.cpp:509

Tunnel connections store the origin in cli->host/cli->port while cli->fd is actually connected to the proxy. The reuse check immediately above compares only the origin host and port, so changing setHttpsProxy/addNoProxy or switching direct versus tunnel and then sending the same origin can skip this branch and reuse the old route/socket. Include the proxy mode and endpoint in the connection identity, or close the existing connection whenever routing changes.

Copilot AI lite review requested due to automatic review settings September 21, 2026 15:31

Copilot started reviewing on behalf of ithewei September 21, 2026 15:31 View session

Copilot AI reviewed Sep 21, 2026

View reviewed changes

Comment thread http/client/HttpClient.cpp Outdated

Comment thread event/nio.c Outdated

Comment thread event/nio.c Outdated

Comment thread http/client/AsyncHttpClient.cpp

Comment thread http/client/HttpClient.cpp

Comment thread http/client/HttpClient.cpp

Copilot AI review requested due to automatic review settings September 21, 2026 15:41

Copilot started reviewing on behalf of ithewei September 21, 2026 15:41 View session

Copilot AI reviewed Sep 21, 2026

View reviewed changes

Comment thread event/nio.c

Comment thread http/client/HttpClient.cpp

Comment thread http/client/HttpClient.cpp

Copilot AI review requested due to automatic review settings September 21, 2026 15:58

Copilot started reviewing on behalf of ithewei September 21, 2026 15:59 View session

Copilot AI reviewed Sep 21, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings September 21, 2026 16:30

Copilot started reviewing on behalf of ithewei September 21, 2026 16:31 View session

Copilot AI reviewed Sep 21, 2026

View reviewed changes

Navigation Menu

feat(http): support https over proxy via HTTP CONNECT - #886

feat(http): support https over proxy via HTTP CONNECT#886
ithewei wants to merge 4 commits into
masterfrom
feat-http-connect-proxy

ithewei commented Sep 21, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

ithewei commented Sep 21, 2026

What

The bug this fixes

Design

event layer (reusable by any io client)

HttpClient

API

Testing

Scope

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants