Skip to content

Fix out-of-bounds read when formatting overlong vector text elements - #32

Merged
AryanSuvarna merged 1 commit into
mainfrom
fix/vector-text-buffer-termination
Oct 1, 2026
Merged

AryanSuvarna merged 1 commit into
mainfrom
fix/vector-text-buffer-termination

Conversation

@AryanSuvarna

@AryanSuvarna AryanSuvarna commented Sep 28, 2026 •

Copy link
Copy Markdown

Summary

Fixes an out-of-bounds stack read in vectorParseSqliteText (libsql-sqlite3/src/vector.c).

Each TEXT vector element is copied into char valueBuf[MAX_FLOAT_CHAR_SZ + 1] (1025 bytes), where the final byte must remain the NUL terminator. The length guard used iBuf > MAX_FLOAT_CHAR_SZ, so the 1025th character overwrote the terminator before the guard fired. The 1026th character then reached the error path, which formats valueBuf with %s, reading past the end of the stack buffer.

The fix rejects an element once it reaches MAX_FLOAT_CHAR_SZ characters:

- if( iBuf > MAX_FLOAT_CHAR_SZ ){
+ if( iBuf >= MAX_FLOAT_CHAR_SZ ){

This is a Shopify fork patch for the OSS vulnerability finding tracked in shop/issues#89264 (Vault issue 81902), following the team decision to patch the fork first and consider upstreaming separately.

Changes

  • libsql-sqlite3/src/vector.c: preserve the reserved NUL terminator.
  • libsql-ffi/bundled/src/sqlite3.c and libsql-ffi/bundled/SQLite3MultipleCiphers/src/sqlite3.c: regenerated amalgamations containing the same one-line change.
  • libsql-sqlite3/test/libsql_vector.test: boundary regression tests for every vector built-in that parses TEXT (vector*, vector_extract, vector_distance_cos, vector_distance_l2), covering 1023/1024-character elements that must still parse, 1025/1026/5000-character elements that must return the length error, malformed elements at the boundary, and oversized second elements. Also adds the missing finish_test call.

Testing

Built testfixture from libsql-sqlite3 on macOS with clang and AddressSanitizer (-O1 -g -fsanitize=address -fno-omit-frame-pointer).

  • Unpatched build: the new tests reproduce the bug. ASan reports stack-buffer-overflow ... READ in sqlite3_str_vappendf from vectorParseSqliteText for 1026-character elements, and for malformed 1025-character elements.

  • Patched build: ./testfixture test/libsql_vector.test '--match=vector-1-text-length-*' passes all 86 new cases with no ASan report and no leaks.

  • Patched build: libsql_vector_index.test passes 50/50 test cases (reports pre-existing unfreed memory from the dbv2 connection the test never closes).

  • Both bundled sqlite3.c files are byte-identical to the regenerated amalgamation.

  • Plain -O2 (non-sanitized) build: the same 86 cases pass. The constructor assertions compare vector_extract(...) output rather than raw blobs, because vector8 blobs carry uninitialized alignment padding bytes that are not stable across runs (this is what made an earlier revision of the tests fail in the make-sqlite3 CI job).

Known caveats:

  • The full libsql_vector.test file has one failure, vector-1-conversion-f8, that fails byte-identically on the unpatched build in this macOS/ASan environment; it is unrelated to this change.
  • make test could not complete locally because the system Tcl is 8.5 and some session tests require Tcl 8.6.
  • Rust workspace tests were not run locally (no cargo toolchain available).
  • The Extensions Tests CI job fails during dependency resolution (rustc 1.85.0 is not supported by icu_*/wasm-encoder/wast) because libsql-sqlite3/test/rust_suite has no Cargo.lock and rust-toolchain.toml pins 1.85.0. It fails identically on main and is unrelated to this change.

Follow-ups

  • Update the POS Mobile Rust workspace pin and the libsql-server Dockerfile tag in libsql-infrastructure to a revision containing this fix.
  • Consider upstreaming to tursodatabase/libsql.

vectorParseSqliteText stores each vector element in a 1025-byte stack
buffer whose final byte must remain NUL. The length guard used `>`, so a
1025-character element overwrote that terminator before the guard
fired; the following error path then formatted the buffer with `%s`,
reading past the end of the stack buffer.

Reject the element once it reaches MAX_FLOAT_CHAR_SZ characters so the
terminator is preserved, regenerate the bundled amalgamations, and add
boundary regression tests for every vector function that parses TEXT.
@AryanSuvarna
AryanSuvarna force-pushed the fix/vector-text-buffer-termination branch from 38157ac to 553cb83 Compare September 28, 2026 20:55
@AryanSuvarna
AryanSuvarna marked this pull request as ready for review October 1, 2026 15:31
@AryanSuvarna
AryanSuvarna requested a review from a team October 1, 2026 15:31

@raphaelfeitoza raphaelfeitoza left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@AryanSuvarna
AryanSuvarna merged commit 7966594 into main Oct 1, 2026
18 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants