Skip to content

Fix forwarding requests when body is cached on disk (#2482)#2525

Merged
an-tao merged 2 commits into
drogonframework:masterfrom
milestone-17:bugfix/2482-forward-cached-body
May 20, 2026
Merged

Fix forwarding requests when body is cached on disk (#2482)#2525
an-tao merged 2 commits into
drogonframework:masterfrom
milestone-17:bugfix/2482-forward-cached-body

Conversation

@milestone-17

Copy link
Copy Markdown
Contributor

Summary

Fixes #2482.

When an inbound HTTP request body exceeds clientMaxMemoryBodySize_ (default 64 KiB), the parser spills it to disk through HttpRequestImpl::cacheFilePtr_ instead of content_. But HttpRequestImpl::appendToBuffer() — which HttpClientImpl::sendReq() uses to serialize a request, including the path taken by HttpAppFramework::forward() — only appended content_ to the output buffer. Result: forwarding a large request to a downstream server sent the headers, set content-length: 0, and dropped the body on the floor.

The fix mirrors the logic already used by HttpRequestImpl::bodyView() / bodyLength(): if cacheFilePtr_ is set, take its bytes via getStringView(), count them in the Content-Length header, and append them to the output buffer.

The two body sources (content_ and cacheFilePtr_) are mutually exclusive by construction (see appendToBody() and reserveBodySize()); the new asserts in appendToBuffer() document and enforce this.

Changes

  • lib/src/HttpRequestImpl.ccappendToBuffer() now reads the body from cacheFilePtr_ when present, includes its length in Content-Length, and writes its bytes after content_.
  • lib/src/HttpRequestImpl.h — forward declaration + friend class ::HttpRequestImplCacheFileTestAccess so the new unit test can install a CacheFile directly without spinning up the full framework / event loop.
  • lib/tests/unittests/HttpRequestForwardCacheBodyTest.cc — new regression tests.
  • lib/tests/CMakeLists.txt — register the new source.

Regression coverage

The new test file covers seven cases:

  1. BodyEmpty — sanity: no body, no content-length. (Regression guard.)
  2. InMemoryStillForwarded — small body in content_ still serializes correctly. (Regression guard.)
  3. CachedBodyForwarded — 200 KiB body in cacheFilePtr_ is serialized with the right Content-Length and full byte payload. This is the HTTP request body stored in cache file is not forwarded. #2482 case.
  4. CachedBodyExactBoundary — exactly clientMaxMemoryBodySize_ (64 KiB).
  5. CachedBodyBinarySafe — binary payload with NULs and high bytes survives byte-for-byte.
  6. PassThroughKeepsBodypassThrough_ mode (which is what app().forward() actually uses) still emits the body.
  7. CachedBodyWithCustomHeaders — extra headers and cookies are preserved alongside the cached body.

Verification

  • Built with MSVC 19.50 (VS2026), Ninja, vcpkg-provided dependencies, BUILD_TESTING=ON.
  • unittest.exe: 1270 assertions in 54 test cases — all pass.
  • I also reverted the production fix locally and confirmed that 5 of the new 7 cases fail without it, with errors like "0" == "204800" for the missing Content-Length. So the tests genuinely guard the fix.

Test plan

  • Build with the test enabled (-DBUILD_TESTING=ON).
  • Run unittest — all tests pass.
  • Reverse-validate: with the fix removed, the new cases fail; with it restored, they pass.

milestone added 2 commits May 19, 2026 22:24
…framework#2482)

When an inbound HTTP request body exceeds clientMaxMemoryBodySize_, the
parser stores it on disk via HttpRequestImpl::cacheFilePtr_ instead of
content_. HttpRequestImpl::appendToBuffer() — used by HttpClientImpl
when forwarding the request through HttpAppFramework::forward() — only
serialized content_, so the downstream server received an empty body
and a Content-Length of 0.

Fix appendToBuffer() to also pull the body bytes from cacheFilePtr_ and
include them in the Content-Length header, matching what bodyView() and
bodyLength() already do.

Add a regression unittest that drives the cacheFilePtr_ path directly
(via a test-only friend) and asserts that:
- a small in-memory body is still serialized as before;
- a body sitting in cacheFilePtr_ is serialized with the correct length;
- the boundary case at exactly clientMaxMemoryBodySize_ works;
- binary payloads with NULs and high bytes survive byte-for-byte;
- passThrough_ mode (used by app().forward()) still emits the body;
- custom headers and cookies are preserved alongside the cached body.
…BodyTest

The CI's clang-format-17 check rewraps a short while-condition onto a
single line. Apply the same formatting locally so the format job passes.
@milestone-17

Copy link
Copy Markdown
Contributor Author

Hi maintainers — a quick note in case it's helpful while reviewing.

I noticed #2518 tackled the same issue earlier and was closed; if there's a preferred direction (e.g. an in-flight refactor of the forward path) I'm very happy to rework or abandon this PR. Just let me know.

If the change to HttpRequestImpl::appendToBuffer() itself is OK, the testing side here tries to stay light: a single friend class for a small test access shim, with the new test source guarded by the same if(MSVC AND BUILD_SHARED_LIBS) exclusion already used by HttpFileTest / HttpMethodTest / WebsocketResponseTest. No extra shim or platform-specific source.

Local verification:

  • unittest: 1270/1270 assertions, 54/54 cases — 100 consecutive runs, 0 failures.
  • End-to-end with two drogon listeners over app().forward(), body sizes 0, 1, 16, 4K, 65535, 65536, 65537, 70K, 128K, 256K, 1M, 4M, POST and PUT — 24/24 pass.
  • Reverse-validated: with the fix reverted, sizes > 64 KiB fail (matches HTTP request body stored in cache file is not forwarded. #2482); with the fix restored, all pass.

CI is in action_required since this is my first PR here — happy to do anything that helps unblock workflow approval. Thanks!

@an-tao an-tao merged commit 5629635 into drogonframework:master May 20, 2026
34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HTTP request body stored in cache file is not forwarded.

2 participants