Make USSI embeddable by a serving host - #79
Merged
Merged
Conversation
A host sharing one JVM with uSSI cannot divide processors with it at the operating system level. ProcessorAllowance lets a host lower uSSI's share at runtime through a supplier, clamped by the processors the process may run on and by the per-thread buffer table a loaded BLAS binary retains. QueryAdmission, ParallelismBudget, and the dense scorers now size from the allowance rather than from availableProcessors. The admission semaphore and the budget resize under a moment with no search running, so a multiply in flight is never torn down.
SearchThreads held a static pool sized from availableProcessors with no shutdown. The pool is now created lazily from ProcessorAllowance, resized under a moment with no search running, and released by shutdown so a host unloading uSSI or a test does not leak daemon threads.
A graduated dense index held the matrix three times: the row map, the Java chunk arrays, and the native or device copy. A scorer that copied the chunks into its own buffers no longer needs the Java arrays, so they are released and the filtered paths read the row map instead. Exposes MemoryFootprint per index and per namespace, separating on-heap from native bytes, so a host limiting memory can account for what uSSI holds.
uSSI blocked in three places and turned an interrupt into a bare IllegalStateException, so a host mapping failures to status codes had to match on message text to tell a bad query from an internal fault. InvalidQueryException covers a malformed query, SearchCancelledException covers a search cancelled by interrupt, and InternalIndexException covers an invariant the library violated. The interrupt flag is restored before a cancellation is thrown, and a search cancelled in flight waits for its work units to finish before throwing.
Consumers build with Gradle or Maven and uSSI builds with Bazel. The src_main target is now a java_export, which produces a Maven-coordinated artifact with a generated POM from the existing Bazel dependencies. Bazel remains the single source of truth, and consumers never learn that Bazel exists.
INTEGRATION.md states what USSI owns and what the host must own, the cost of a rebuild, the blocking model, the processor allowance, the search pool lifecycle, the cancellation contract, the exception types, the memory footprint estimate, the native library load, and the Maven coordinates. It is the one place an integrator reads.
Applying an allowance change re-entered the admission semaphore, which is not reentrant, so a change that moved the count a native library holds suspended every search in the process permanently. The change is now applied inline by a caller that already holds exclusivity. Every row-storing index now estimates its own row storage rather than inheriting zero, and the hybrid index sums the two it delegates to. The namespace estimate no longer claims to cover caches it does not read. Adds the tests for both, including one that fails against the deadlock.
Reading the BLAS thread maximum sets the count to a sentinel and restores it. It was registered as the live supplier for the allowance clamp, so every re-clamp set the process-global thread count twice, on the budget thread, outside the moment with no search running. That is the modification that is unsafe while a multiply is dispatching work, and it ran once per averaging window for the life of the process. The buffer table is fixed when the binary is built, so the number read at construction is registered instead.
The estimate counted the row map alone, so an inverted index reported exactly what a scan index over the same rows reported, and the structure answering metadata filters contributed nothing. It now counts the metadata structure for every row-storing index, and the inverted lists, the indexed and verification row maps, the unilateral values, and the discarded terms for an inverted index. Also removes meta-commentary and an idiom the writing standards exclude, and spells USSI as the rest of the codebase does.
The shared budget passed the socket width as the batch width and the allowance as the threads one search may use. A host lowering the allowance below one socket's cores before building its first namespace therefore violated the invariant that the batch width never exceeds the threads one search may use, and the static initializer threw, leaving the library unusable. The batch width is now the narrower of the two. The socket width is retained undivided, so raising the allowance widens the batch back to what the machine has. Also records the cost of a filtered dense search against a scorer holding its own copy, and that the memory estimate is linear in what a namespace holds.
The shard count was derived from the processors the process may run on rather than from the allowance, so an inverted index built while a host held USSI to a few processors still divided for every core. Registering the native bound from the OpenBLAS constructor let an instance built to exercise that class bound every search in the process, which made the outcome depend on the order the tests ran in. The bound is registered for the instance serving the process instead, and the clamp is a pure function the table-driven test exercises without touching the shared allowance. Resizing the search pool is package-private, reached only by the budget that already holds exclusivity. Shutting it down suspends every search first rather than documenting that the caller must.
Publishing showed the CUDA bindings and Lombok listed as runtime dependencies for every consumer. neverlink governs the Bazel runtime classpath alone, so the maven:compile-only tag is what keeps a dependency out of the POM. Adds a terms section to INTEGRATION.md so no term is used before it is defined, corrects the publish command to take any repository rather than a path under the home directory, and counts the values a merging candidate generator keeps in its inverted lists.
The rows keyed by the lists and the rows scored are the same records as the rows held, once no term is discarded and the indexed form is the identity. Those maps are also one another in that case. The estimate counted the record set three times over, reporting roughly twice the memory an index holds. Each map is now counted by identity, and a record the held map already references contributes its slot alone. Also drops the term width, which named what the existing prose calls a batch size and a socket's cores, and stops an allowance change creating a search pool that no search has asked for yet.
The allowance supplier is the host's own code and is read from the periodic work that re-derives both thread counts. A periodic task that throws is not scheduled again, so a supplier that failed once left both counts where they stood for the life of the process. A failed reading now leaves the allowance at what it last reported and the next interval takes another. Records that setting the allowance is optional, and what every count derives from when a host sets nothing.
A dense scorer holding its own copy releases the Java matrix, and every lifecycle test built a scan or inverted index, so nothing exercised rebuilding a matrix namespace from the rows it kept. Inserts through a matrix namespace until it graduates and consolidates, then asserts every row is still nearest to its own vector.
A native scorer keeps a read buffer for the products of one range and one dot product per row for each query a multiply may carry, both for the life of the scorer. Neither was counted, so a namespace of two million rows understated its heap by the larger part of a hundred megabytes. Adds the test file MemoryFootprint had none of, and covers the estimate for an index of no rows and for the record sets discarding a popular term gives an inverted index separately.
drametwally
force-pushed
the
embeddable-host
branch
from
September 22, 2026 00:37
cd366c3 to
119ab96
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Prepares USSI to be embedded by any serving host. Each item is host-neutral: USSI
states a fact about itself and offers a knob, and the host decides what to do with it.
ProcessorAllowancelets a host lower the processors USSI may use at runtime througha supplier, clamped by the processors the process may run on and by the per-thread
buffer table a loaded BLAS binary retains. Admission, the parallelism budget, the
search pool, and the dense scorers size from it rather than from
availableProcessors.SearchThreadsno longer creates a static pool with no shutdown. It is created lazilyfrom the allowance, resized under a moment with no search running, and released by
shutdown().A graduated dense index held the vectors three times: the row map, the Java chunk
arrays, and the native or device copy. A scorer that copies the chunks into its own
buffers now releases the Java arrays, and the filtered paths read the row map.
MemoryFootprintreports an estimate per index and per namespace, separating on-heapfrom native bytes.
Interrupt is the cancellation signal at all three blocking sites and raises
SearchCancelledException.InvalidQueryExceptionandInternalIndexExceptionreplace bare
IllegalArgumentExceptionandIllegalStateExceptionat the facade, soa host mapping failures to status codes does not match on message text.
src_mainis now ajava_export, producingcom.uber.ussi:ussi:0.1.0with a POMgenerated from the existing Bazel dependencies. No second build file.
INTEGRATION.mddescribes the seam: what USSI owns, what the host must own, the costof a rebuild, the blocking model, the allowance, the pool lifecycle, cancellation, the
exception contract, the footprint estimate, the native library load, and Maven
consumption.
Behaviour change: the search pool is no longer created at class load and is not
shut down by
NearestNeighborSearchIndex.close(). A host unloading USSI callsSearchThreads.shutdown().676 tests, 675 passing, one aborted on the GPU assumption as before.