Fix forwarding requests when body is cached on disk (#2482)#2525
Conversation
…framework#2482) When an inbound HTTP request body exceeds clientMaxMemoryBodySize_, the parser stores it on disk via HttpRequestImpl::cacheFilePtr_ instead of content_. HttpRequestImpl::appendToBuffer() — used by HttpClientImpl when forwarding the request through HttpAppFramework::forward() — only serialized content_, so the downstream server received an empty body and a Content-Length of 0. Fix appendToBuffer() to also pull the body bytes from cacheFilePtr_ and include them in the Content-Length header, matching what bodyView() and bodyLength() already do. Add a regression unittest that drives the cacheFilePtr_ path directly (via a test-only friend) and asserts that: - a small in-memory body is still serialized as before; - a body sitting in cacheFilePtr_ is serialized with the correct length; - the boundary case at exactly clientMaxMemoryBodySize_ works; - binary payloads with NULs and high bytes survive byte-for-byte; - passThrough_ mode (used by app().forward()) still emits the body; - custom headers and cookies are preserved alongside the cached body.
…BodyTest The CI's clang-format-17 check rewraps a short while-condition onto a single line. Apply the same formatting locally so the format job passes.
|
Hi maintainers — a quick note in case it's helpful while reviewing. I noticed #2518 tackled the same issue earlier and was closed; if there's a preferred direction (e.g. an in-flight refactor of the forward path) I'm very happy to rework or abandon this PR. Just let me know. If the change to Local verification:
CI is in |
Summary
Fixes #2482.
When an inbound HTTP request body exceeds
clientMaxMemoryBodySize_(default 64 KiB), the parser spills it to disk throughHttpRequestImpl::cacheFilePtr_instead ofcontent_. ButHttpRequestImpl::appendToBuffer()— whichHttpClientImpl::sendReq()uses to serialize a request, including the path taken byHttpAppFramework::forward()— only appendedcontent_to the output buffer. Result: forwarding a large request to a downstream server sent the headers, setcontent-length: 0, and dropped the body on the floor.The fix mirrors the logic already used by
HttpRequestImpl::bodyView()/bodyLength(): ifcacheFilePtr_is set, take its bytes viagetStringView(), count them in theContent-Lengthheader, and append them to the output buffer.The two body sources (
content_andcacheFilePtr_) are mutually exclusive by construction (seeappendToBody()andreserveBodySize()); the new asserts inappendToBuffer()document and enforce this.Changes
lib/src/HttpRequestImpl.cc—appendToBuffer()now reads the body fromcacheFilePtr_when present, includes its length inContent-Length, and writes its bytes aftercontent_.lib/src/HttpRequestImpl.h— forward declaration +friend class ::HttpRequestImplCacheFileTestAccessso the new unit test can install aCacheFiledirectly without spinning up the full framework / event loop.lib/tests/unittests/HttpRequestForwardCacheBodyTest.cc— new regression tests.lib/tests/CMakeLists.txt— register the new source.Regression coverage
The new test file covers seven cases:
BodyEmpty— sanity: no body, nocontent-length. (Regression guard.)InMemoryStillForwarded— small body incontent_still serializes correctly. (Regression guard.)CachedBodyForwarded— 200 KiB body incacheFilePtr_is serialized with the rightContent-Lengthand full byte payload. This is the HTTP request body stored in cache file is not forwarded. #2482 case.CachedBodyExactBoundary— exactlyclientMaxMemoryBodySize_(64 KiB).CachedBodyBinarySafe— binary payload with NULs and high bytes survives byte-for-byte.PassThroughKeepsBody—passThrough_mode (which is whatapp().forward()actually uses) still emits the body.CachedBodyWithCustomHeaders— extra headers and cookies are preserved alongside the cached body.Verification
BUILD_TESTING=ON.unittest.exe: 1270 assertions in 54 test cases — all pass."0" == "204800"for the missingContent-Length. So the tests genuinely guard the fix.Test plan
-DBUILD_TESTING=ON).unittest— all tests pass.