Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
89d2de1
Add ALTER TABLE RENAME support for vec0 virtual tables
vlasky Nov 28, 2025
12f4e05
Add cosine distance support for binary quantized vectors
vlasky Nov 28, 2025
b632185
Fix memory leak by clearing vectors and rowids on delete
vlasky Nov 28, 2025
f10e191
Fix CI/CD builds: upgrade deprecated runners and use native ARM64
vlasky Nov 28, 2025
6815d95
Add optimize command for space reclamation after deletions
vlasky Nov 28, 2025
ad04763
Add distance constraints for KNN queries
vlasky Nov 28, 2025
9dfbd88
Add package configuration files for GitHub-based installation
vlasky Nov 28, 2025
4fc6536
Fix Windows 32-bit compilation for __popcnt64 (PR #87)
vlasky Dec 1, 2025
2bd13b2
Fix Windows ARM/ARM64 __builtin_popcountl to handle u64
vlasky Dec 1, 2025
0dfec69
Add LIKE operator support for text metadata columns (Issue #197)
vlasky Dec 1, 2025
dd13eb5
Fix locale-dependent JSON float parsing (Issue #241)
vlasky Dec 1, 2025
4afbb8b
Fix compilation on musl libc (Alpine Linux)
vlasky Dec 1, 2025
2dfe885
Bump version to v0.2.1-alpha
vlasky Dec 2, 2025
4899d0e
Add GLOB operator support for text metadata columns (Issue #191)
vlasky Dec 2, 2025
12dc66a
Add support for IS/IS NOT/IS NULL/IS NOT NULL operators on metadata c…
vlasky Dec 2, 2025
b088df9
Fix all compilation warnings
vlasky Dec 2, 2025
7b72554
Bump version to v0.2.2-alpha
vlasky Dec 2, 2025
042d093
Add optimize/VACUUM integration test and docs
vlasky Dec 13, 2025
03f2b2f
Fix Linux linking by moving -lm to LDLIBS
vlasky Dec 19, 2025
e8f156b
Improve shared library build and installation
vlasky Dec 19, 2025
c6f5b56
Add Android 16KB page support via LDFLAGS
vlasky Dec 29, 2025
5c151dd
Fix incomplete documentation in KNN and Matryoshka guides
vlasky Dec 29, 2025
920b0d2
Bump version to v0.2.3-alpha
vlasky Dec 29, 2025
b2698b6
Add Lua binding with IEEE 754 compliant float serialization
vlasky Jan 3, 2026
606e000
Bump version to v0.2.4-alpha
vlasky Jan 4, 2026
fd193ae
Fix Ruby documentation example errors
vlasky Jan 5, 2026
0c1d19f
Update version to 0.2.4-alpha in language binding metadata
vlasky Jan 5, 2026
9facf1a
Add note about exclusive access requirement for optimize/VACUUM
vlasky Jan 5, 2026
623661e
Pass correct pointer to cleanup in ensure_vector_match error path
renatgalimov Jan 31, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ permissions:
contents: read
jobs:
build-linux-x86_64-extension:
runs-on: ubuntu-20.04
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: ./scripts/vendor.sh
Expand Down Expand Up @@ -55,13 +55,12 @@ jobs:
name: sqlite-vec-windows-x86_64-extension
path: dist/*
build-linux-aarch64-extension:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04-arm
steps:
- uses: actions/checkout@v4
- run: sudo apt-get install gcc-aarch64-linux-gnu
- run: ./scripts/vendor.sh
- run: make sqlite-vec.h
- run: make CC=aarch64-linux-gnu-gcc loadable static
- run: make loadable static
- uses: actions/upload-artifact@v4
with:
name: sqlite-vec-linux-aarch64-extension
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ permissions:
contents: read
jobs:
build-linux-x86_64-extension:
runs-on: ubuntu-20.04
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v3
Expand Down
10 changes: 9 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
/target
/Cargo.lock
.vscode
sift/
*.tar.gz
Expand All @@ -7,12 +8,20 @@ sift/
*.bin
*.out
venv/
.venv

vendor/
dist/

*.pyc
*.db-journal
build/
*.egg-info/
sqlite_vec.py
sqlite_vec/
vec0.so
vec0.dylib
vec0.dll

alexandria/
openai/
Expand All @@ -22,7 +31,6 @@ examples/dbpedia-openai
examples/imdb
examples/sotu

sqlite-vec.h
tmp/

poetry.lock
Expand Down
27 changes: 27 additions & 0 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,3 +120,30 @@ The third character of the block is the constraint operator. It will be one of
metadata column KNN filters.

The foruth character of the block is a `_` filler.

#### `VEC0_IDXSTR_KIND_KNN_DISTANCE_CONSTRAINT` (`'*'`)

`argv[i]` is a constraint on the `distance` column in a KNN query.

This enables filtering KNN results by distance thresholds, useful for:
- Cursor-based pagination: `WHERE embedding MATCH ? AND k = 10 AND distance > 0.21`
- Range queries: `WHERE embedding MATCH ? AND k = 100 AND distance BETWEEN 0.5 AND 1.0`

The second character of the block denotes the constraint operator. It will be one of
the values of `enum vec0_distance_constraint_operator`:

| Operator | Value | Description | SQL Example |
| -------- | ----- | ------------------------ | -------------------- |
| `GT` | `'a'` | Greater than | `distance > 0.5` |
| `GE` | `'b'` | Greater than or equal to | `distance >= 0.5` |
| `LT` | `'c'` | Less than | `distance < 1.0` |
| `LE` | `'d'` | Less than or equal to | `distance <= 1.0` |

The third and fourth characters of the block are `_` fillers.

**Note on precision:** Distance values are cast from f64 to f32 for comparison, which may
result in precision loss for very small distance differences.

**Note on pagination:** When multiple vectors have identical distances, pagination using
`distance > X` may skip some results. For stable pagination, combine distance with rowid:
`WHERE (distance > 0.5) OR (distance = 0.5 AND rowid > 123)`
162 changes: 162 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
# Changelog

All notable changes to this community fork will be documented in this file.

## [0.2.4-alpha] - 2026-01-03

### Added

- **Lua binding with IEEE 754 compliant float serialization** ([#237](https://github.com/asg017/sqlite-vec/pull/237))
- `bindings/lua/sqlite_vec.lua` provides `load()`, `serialize_f32()`, and `serialize_json()` functions
- Lua 5.1+ compatible with lsqlite3
- IEEE 754 single-precision float encoding with round-half-to-even (banker's rounding)
- Proper handling of special values: NaN, Inf, -Inf, -0.0, subnormals
- Example script and runner in `/examples/simple-lua/`

## [0.2.3-alpha] - 2025-12-29

### Added

- **Android 16KB page support** ([#254](https://github.com/asg017/sqlite-vec/pull/254))
- Added `LDFLAGS` support to Makefile for passing linker-specific flags
- Enables Android 15+ compatibility via `-Wl,-z,max-page-size=16384`
- Required for Play Store app submissions on devices with 16KB memory pages

- **Improved shared library build and installation** ([#149](https://github.com/asg017/sqlite-vec/issues/149))
- Configurable install paths via `INSTALL_PREFIX`, `INSTALL_LIB_DIR`, `INSTALL_INCLUDE_DIR`, `INSTALL_BIN_DIR`
- Hidden internal symbols with `-fvisibility=hidden`, exposing only public API
- `EXT_CFLAGS` captures user-provided `CFLAGS` and `CPPFLAGS`

- **Optimize/VACUUM integration test and documentation**
- Added test demonstrating optimize command with VACUUM for full space reclamation

### Fixed

- **Linux linking error with libm** ([#252](https://github.com/asg017/sqlite-vec/pull/252))
- Moved `-lm` flag from `CFLAGS` to `LDLIBS` at end of linker command
- Fixes "undefined symbol: sqrtf" errors on some Linux distributions
- Linker now correctly resolves math library symbols

### Documentation

- **Fixed incomplete KNN and Matryoshka guides** ([#208](https://github.com/asg017/sqlite-vec/pull/208), [#209](https://github.com/asg017/sqlite-vec/pull/209))
- Completed unfinished sentence describing manual KNN method trade-offs
- Added paper citation and Matryoshka naming explanation

## [0.2.2-alpha] - 2025-12-02

### Added

- **GLOB operator for text metadata columns** ([#191](https://github.com/asg017/sqlite-vec/issues/191))
- Standard SQL pattern matching with `*` (any characters) and `?` (single character) wildcards
- Case-sensitive matching (unlike LIKE)
- Fast path optimization for prefix-only patterns (e.g., `'prefix*'`)
- Full pattern matching with `sqlite3_strglob()` for complex patterns

- **IS/IS NOT/IS NULL/IS NOT NULL operators for metadata columns** ([#190](https://github.com/asg017/sqlite-vec/issues/190))
- **Note**: sqlite-vec metadata columns do not currently support NULL values. These operators provide syntactic compatibility within this limitation.
- `IS` behaves like `=` (all metadata values are non-NULL)
- `IS NOT` behaves like `!=` (all metadata values are non-NULL)
- `IS NULL` always returns false (no NULL values exist in metadata)
- `IS NOT NULL` always returns true (all metadata values are non-NULL)
- Works on all metadata types: INTEGER, FLOAT, TEXT, and BOOLEAN

### Fixed

- **All compilation warnings eliminated**
- Fixed critical logic bug: `metadataInIdx` type corrected from `size_t` to `int` (prevented -1 wrapping to SIZE_MAX)
- Fixed 5 sign comparison warnings with proper type casts
- Fixed 7 uninitialized variable warnings by adding initializers and default cases
- Clean compilation with `-Wall -Wextra` (zero warnings)

## [0.2.1-alpha] - 2025-12-02

### Added

- **LIKE operator for text metadata columns** ([#197](https://github.com/asg017/sqlite-vec/issues/197))
- Standard SQL pattern matching with `%` and `_` wildcards
- Case-insensitive matching (SQLite default)

### Fixed

- **Locale-dependent JSON parsing** ([#241](https://github.com/asg017/sqlite-vec/issues/241))
- Custom locale-independent float parser fixes JSON parsing in non-C locales
- No platform dependencies, thread-safe

- **musl libc compilation** (Alpine Linux)
- Removed non-portable preprocessor macros from vendored sqlite3.c

## [0.2.0-alpha] - 2025-11-28

### Added

- **Distance constraints for KNN queries** ([#166](https://github.com/asg017/sqlite-vec/pull/166))
- Support GT, GE, LT, LE operators on the `distance` column in KNN queries
- Enables cursor-based pagination: `WHERE embedding MATCH ? AND k = 10 AND distance > 0.5`
- Enables range queries: `WHERE embedding MATCH ? AND k = 100 AND distance BETWEEN 0.5 AND 1.0`
- Works with all vector types (float32, int8, bit)
- Compatible with partition keys, metadata, and auxiliary columns
- Comprehensive test coverage (15 tests)
- Fixed variable shadowing issues from original PR
- Documented precision handling and pagination caveats

- **Optimize command for space reclamation** ([#210](https://github.com/asg017/sqlite-vec/pull/210))
- New special command: `INSERT INTO vec_table(vec_table) VALUES('optimize')`
- Reclaims disk space after DELETE operations by compacting shadow tables
- Rebuilds vector chunks with only valid rows
- Updates rowid mappings to maintain data integrity

- **Cosine distance support for binary vectors** ([#212](https://github.com/asg017/sqlite-vec/pull/212))
- Added `distance_cosine_bit()` function for binary quantized vectors
- Enables cosine similarity metric on bit-packed vectors
- Useful for memory-efficient semantic search

- **ALTER TABLE RENAME support** ([#203](https://github.com/asg017/sqlite-vec/pull/203))
- Implement `vec0Rename()` callback for virtual table module
- Allows renaming vec0 tables with standard SQL: `ALTER TABLE old_name RENAME TO new_name`
- Properly renames all shadow tables and internal metadata

- **Language bindings and package configurations for GitHub installation**
- Go CGO bindings (`bindings/go/cgo/`) with `Auto()` and serialization helpers
- Python package configuration (`pyproject.toml`, `setup.py`) for `pip install git+...`
- Node.js package configuration (`package.json`) for `npm install vlasky/sqlite-vec`
- Ruby gem configuration (`sqlite-vec.gemspec`) for `gem install` from git
- Rust crate configuration (`Cargo.toml`, `src/lib.rs`) for `cargo add --git`
- All packages support installing from main branch or specific version tags
- Documentation in README with installation table for all languages

- **Python loadable extension support documentation**
- Added note about Python requiring `--enable-loadable-sqlite-extensions` build flag
- Recommended using `uv` for virtual environments (uses system Python with extension support)
- Documented workarounds for pyenv and custom Python builds

### Fixed

- **Memory leak on DELETE operations** ([#243](https://github.com/asg017/sqlite-vec/pull/243))
- Added `vec0Update_Delete_ClearRowid()` to clear deleted rowids
- Added `vec0Update_Delete_ClearVectors()` to clear deleted vector data
- Prevents memory accumulation from deleted rows
- Vectors and rowids now properly zeroed out on deletion

- **CI/CD build infrastructure** ([#228](https://github.com/asg017/sqlite-vec/pull/228))
- Upgraded deprecated ubuntu-20.04 runners to ubuntu-latest
- Added native ARM64 builds using ubuntu-24.04-arm
- Removed cross-compilation dependencies (gcc-aarch64-linux-gnu)
- Fixed macOS link flags for undefined symbols

## Original Version

This fork is based on [`asg017/sqlite-vec`](https://github.com/asg017/sqlite-vec) v0.1.7-alpha.2.

All features and functionality from the original repository are preserved.
See the [original documentation](https://alexgarcia.xyz/sqlite-vec/) for complete usage information.

---

## Notes

This is a community-maintained fork created to merge pending upstream PRs and provide
continued support while the original author is unavailable. Once development resumes
on the original repository, users are encouraged to switch back.

All original implementation credit goes to [Alex Garcia](https://github.com/asg017).
Loading