Skip to content

Upgrade to zvec v0.6.0 — FFI + ext migration (closes #170 #171 #172 #173 #174) - #184

Merged
s2x merged 4 commits into
mainfrom
issue-170/zvec-v0.6.0-bump
Jul 30, 2026
Merged

Upgrade to zvec v0.6.0 — FFI + ext migration (closes #170 #171 #172 #173 #174)#184
s2x merged 4 commits into
mainfrom
issue-170/zvec-v0.6.0-bump

Conversation

@s2x

@s2x s2x commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

What

Upgrade the upstream alibaba/zvec dependency from v0.4.0 → v0.6.0 and migrate the C++ wrappers (FFI shared library + PHP extension) to the v0.6.0 C++ API. All breaking changes are confined to the C++ layer — the PHP-facing C API in ffi/*.h is byte-identical and src/*.php is untouched (one defensive guard in toCStringArray), so the public PHP API stays backward compatible.

Closes #170, #171, #172, #173, #174, #32.

Commits

  1. 8f2e377 — bump submodule to v0.6.0 + fix link list (libzvec_db.alibzvec.a, add libsnowball.a+libutf8proc.a, DiskANN now built-in / libaio via dlopen)
  2. 85ad96c — migrate ffi/zvec_ffi.cc and php-ext/zvec_collection.cc to the v0.6.0 query structs
  3. 0c49c82 — completeness: bump embedded version constants v0.4.0→v0.6.0, remaining v0.4.0 refs in workflows/docs, defensive toCStringArray guard, regression test for FFI: output_fields became std::optional — empty vector now means 'select no fields' #172

Per-issue verification (rigorous, done after the 3rd commit)

Issue Checklist item Status Evidence
#170 zvec lib builds at v0.6.0 ✅ DONE zvec/build/.zvec_version = v0.6.0; libzvec.a 12.7MB present
#170 link list correct (new externals) ✅ DONE libsnowball.a + libutf8proc.a added; every referenced .a exists in build (full set compared: 41 produced libs, all refs resolved)
#170 diskann plugin .so gone, libaio dlopen ✅ DONE libcore_knn_diskann.a built-in, no plugin
#170 default version strings bumped ✅ DONE grep v0.4.0 across repo (excl. CHANGELOG/MIGRATION history) → 0 hits in build/docs/workflows; ffi/zvec_ffi.cc version constants = v0.6.0
#171 VectorQuerySearchQuery+QueryTarget ✅ DONE rg VectorQuery in .cc → only VectorQueryHolder (our holder name, wraps SearchQuery query); target_.field_name_, target_.set_vector(), target_.query_params_ everywhere
#171 all query entry points migrated ✅ DONE fp32/fp16/fp64/sparse/queryById/queryByFilter — both ffi & ext
#171 QueryParams::set_type() unused ✅ DONE rg set_type in ffi/ php-ext/ → 0 hits
#172 leave nullopt when no output fields ✅ DONE rg output_fields_ = {} / empty vector assign → 0 hits; setters only touch it when count>0
#172 regression test added ✅ DONE tests/test_query_output_fields_semantics.phpt (added in commit 3) — passes
#173 group_topk_topk_per_group_ ✅ DONE rg group_topk_ → 0 hits; topk_per_group_ used
#173 field/vector/params into target_ ✅ DONE confirmed in GroupByVectorQueryHolder
#173 test_groupby_query_object.phpt passes ✅ DONE green
#174 MultiQuery path verified ✅ DONE no C-side MultiQuery (PHP does per-field SearchQuery + rerank); rerank purely PHP (rg Rerank in ffi → 0)

Build & test results

  • ./build_ffi.shffi/build/libzvec_ffi.dylib (53MB) links clean
  • ./php-ext/build_ext.shphp-ext/modules/zvec.so (32MB) builds & loads
  • php -n run-tests.php -q tests/171 passed, 2 XFAIL, 0 failed (175 total)

Side effects

  • bug_0002 now passes — zvec v0.6.0 implements group-by; --XFAIL-- removed
  • test_extended_query_params expectation: 1 group → 2 groups (v0.6.0 returns even/odd correctly)
  • FP64 XFAIL comments updated v0.4.0→v0.6.0 (still unsupported upstream — separate issue)

Out of scope (separate issues, v0.6.0 milestone)

New v0.6.0 features not included — only the migration: #176 fetch outputFields, #177 random rotation, #178 includeDocId, #179 DiskAnnIndexParams split, #180 FTS, #181 minor v0.5.1 options.

Known non-blocker

Running the suite with the native zvec extension loaded fails test_vector_query_object.phpt on ZVecVectorQuery::setTopk() — a pre-existing native-ext feature gap, unrelated to this upgrade. FFI mode (CI path) is green.

Piotr Hałas added 3 commits July 30, 2026 18:43
- Update zvec submodule from v0.4.0 to v0.6.0 (commit ec8a78e)
- ffi/CMakeLists.txt: libzvec_db.a renamed to libzvec.a (upstream #481)
- Add libFastPFOR.a, libsnowball.a (FTS stemmer, #513) and libutf8proc.a (UTF-8 tokenizer, #515)
- build_ffi.sh: update pre-flight check libzvec_db.a -> libzvec.a
- build_zvec.sh / build_zvec_lib.sh / AGENTS.md: default version v0.4.0 -> v0.6.0
- DiskANN is now built-in (libcore_knn_diskann.a), libaio via dlopen (upstream #532)

FFI wrapper (ffi/zvec_ffi.cc) does not yet compile - VectorQuery->SearchQuery
migration is tracked by #171-#174. This issue only covers the build/link list.
…172 #173 #174)

- Replace removed VectorQuery with SearchQuery; field name, query vector
  and query params now live on QueryTarget (target_.field_name_,
  target_.set_vector(), target_.query_params_)
- output_fields_ is now std::optional: leave as nullopt (select all) when
  no output fields requested instead of setting an empty vector, which in
  v0.6.0 means 'select no fields' (#172)
- GroupByVectorQuery: group_topk_ renamed to topk_per_group_; vector,
  field name and query params moved into target_ (#173)
- php-ext: port zvec_collection.cc to the same SearchQuery/QueryTarget
  layout; update config.m4 link list (libzvec/libzvec_core/libzvec_turbo,
  FastPFOR/snowball/utf8proc) and build_ext.sh stamp check for v0.6.0
- bug_0002 (GroupByQuery): now fixed upstream in v0.6.0, drop XFAIL
- test_extended_query_params: group-by now returns proper groups (2)
- ffi/zvec_ffi.cc: bump embedded version constants v0.4.0 -> v0.6.0
- build_zvec_lib.sh, docker/build-zvec.sh, docs/workflow.md, .github/workflows/*.yml: remaining v0.4.0 -> v0.6.0 references
- src/ZVec.php: guard toCStringArray() for empty arrays (defensive, output_fields=[])
- tests/test_query_output_fields_semantics.phpt: regression test for #172 (nullopt = select all)
- tests/test_fp64_vectors.phpt, test_query_fp64.phpt: XFAIL comment v0.4.0 -> v0.6.0 (FP64 still unsupported upstream)
@s2x
s2x merged commit a48cc28 into main Jul 30, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bump zvec submodule to v0.6.0 and verify FFI link list

1 participant