Pointer wraparound in body parsing + integer overflow in websocket_calc_frame_size

[ByamB4]

Summary

Two memory safety vulnerabilities were found via manual source code audit of websocket_parser.c.

Bug 1: Pointer Wraparound in Body Length Check (CWE-190/CWE-131) — High

Location: websocket_parser_execute(), line 145

case s_body:
    if(parser->require) {
        if(p + parser->require <= end) {
            EMIT_DATA_CB(frame_body, p, parser->require);

Issue: For frames with 8-byte extended length (payload length field = 127), parser->length is accumulated from 8 bytes (lines 101-107) with no upper bound check. The WebSocket RFC (section 5.2) requires the MSB to be 0, but the parser doesn't enforce this.

The check p + parser->require <= end uses pointer arithmetic. When parser->require is close to SIZE_MAX, p + parser->require wraps around (unsigned overflow), evaluating to a value less than end. The check passes incorrectly.

Example (64-bit):

• Input buffer at address 0x00007f0000001000, length 256 bytes
• p at offset 10: 0x00007f000000100A
• 8-byte length encodes 0xFFFFFF00FFFFF000 (~18 exabytes)
• p + parser->require wraps around → check passes
• Callback is invoked with (p, 0xFFFFFF00FFFFF000) — pointer to 246 bytes of actual data, claiming ~18 EB
• Any code that processes the callback data reads far past the buffer

On 32-bit: Only the low 32 bits of the 8-byte length survive (truncation during accumulation). Crafting the length bytes so the low 32 bits are ~(p - end) directly wraps the pointer.

Impact: The on_frame_body callback receives a data pointer with a fraudulently large length. If the callback copies, inspects, or forwards the data, it causes massive out-of-bounds read.

Bug 2: Integer Overflow in websocket_calc_frame_size (CWE-190) — Medium

Location: websocket_calc_frame_size(), lines 191-205

size_t websocket_calc_frame_size(websocket_flags flags, size_t data_len) {
    size_t size = data_len + 2;
    if(data_len >= 126) {
        if(data_len > 0xFFFF) {
            size += 8;
        } else {
            size += 2;
        }
    }
    if(flags & WS_HAS_MASK) {
        size += 4;
    }
    return size;
}

Issue: data_len + 2 overflows when data_len is near SIZE_MAX. For data_len = SIZE_MAX - 1: size = 0, then size += 8 → 8, size += 4 → 12. Returns 12. But actual frame size would be ~SIZE_MAX bytes. If the caller allocates 12 bytes and passes it to websocket_build_frame, the frame building writes far past the buffer.

Impact: Heap buffer overflow if caller trusts the returned size for allocation.

Suggested Fixes

Bug 1:

Validate the extended length and use subtraction instead of addition for bounds checks:

/* After accumulating length in s_length: */
if(parser->length > SIZE_MAX / 2) {
    return GET_NPARSED();  /* reject absurdly large frames */
}

/* In s_body, use subtraction to avoid wraparound: */
if(parser->require <= (size_t)(end - p)) {

Bug 2:

Check for overflow:

if(data_len > SIZE_MAX - 14) return 0; /* max overhead is 14 bytes */

Found via manual source code audit. Reported responsibly — please let me know if you need any additional information.