Filter source membership before retrieval ranking for 26.10.6

CHANGELOG.md

@@ -2,6 +2,19 @@
 
 All notable changes to **flycanon** are documented here.
 
+## [26.10.6] - 2026-10-26
+
+### Fixed
+
+- Apply nonempty `source_ids` membership before the BM25 and dense candidate windows, Reciprocal Rank Fusion and `top_k` selection. Lower-ranked admitted sources are no longer lost behind unrelated higher-ranked candidates.
+- Intersect source membership with the authenticated tenant and workspace in both retrieval channels. A selected source with no matching chunks returns no hits; retrieval never falls back to the global source set. Hydrated filtering remains defense in depth.
+- Preserve the existing request schema and the meaning of omitted, `null` and empty-list source selectors: no additional source constraint within the authenticated scope. Knowledge metadata filters retain their hydration behavior.
+
+### Distribution
+
+- Align server and Python/Java SDK version metadata at `26.10.6`. SDK search methods, runtime dependency pins, database migrations, source intake and EDA event schemas are unchanged.
+- Retain pyfly `v26.09.04` and Agentic `26.10.1`. Upgrade the server to obtain the source-filter correction; compatible existing SDKs continue to work.
+
 ## [26.10.5] - 2026-10-17
 
 ### Fixed

README.md

@@ -19,7 +19,7 @@ over whole documents with citations — all behind a single HTTP service.
 [![agentic](https://img.shields.io/badge/genai-fireflyframework--agentic-purple)](https://github.com/fireflyframework/fireflyframework-agentic)
 [![OpenAPI](https://img.shields.io/badge/api-openapi%203.1-green)](docs/api-reference.md)
 [![vector store](https://img.shields.io/badge/vector--store-pgvector%20%7C%20qdrant%20%7C%20chroma-336791)](docs/architecture.md#retrieval-backend-pluggable-dense-store)
-[![Version](https://img.shields.io/badge/version-26.10.5-green.svg)](#)
+[![Version](https://img.shields.io/badge/version-26.10.6-green.svg)](#)
 [![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
 
 </div>

docs/api-reference.md

@@ -118,6 +118,8 @@ for private dev stacks only.
 | `POST` | `/api/v1/query/stream` | Same as `/query` but streams the answer as Server-Sent Events. |
 | `POST` | `/api/v1/query/suggest` | Suggest follow-up questions for an existing answer. Body: `AnswerRequest`. |
 
+A nonempty `source_ids` selector is intersected with the authenticated tenant and workspace before BM25 and dense candidate ranking, fusion and `top_k` selection. Sources with no matching chunks return no hits; retrieval never falls back to the global source set. Omitting `source_ids`, using `null`, or supplying an empty list retains the existing meaning of no additional source constraint within the authenticated scope. Hydrated source checks remain active after ranking as defense in depth. Other knowledge metadata filters retain their documented hydration behavior.
+
 ### Answer mode (RLM default / RAG deprecated)
 
 The answer endpoints (`/api/v1/query`, `/api/v1/query/stream`, and the

docs/deployment.md

@@ -43,7 +43,7 @@ For a host installation using the released wheel, follow [Install the released s
 
 ```bash
 docker pull ghcr.io/fireflyframework/flycanon:latest          # latest main
-docker pull ghcr.io/fireflyframework/flycanon:26.10.5          # specific release
+docker pull ghcr.io/fireflyframework/flycanon:26.10.6          # specific release
 docker pull ghcr.io/fireflyframework/flycanon:26.10            # latest 26.10.x
 ```
 
@@ -683,12 +683,12 @@ The recommended upgrade dance:
 
 ```bash
 # 1. Pull the new image
-docker pull ghcr.io/fireflyframework/flycanon:26.10.5
+docker pull ghcr.io/fireflyframework/flycanon:26.10.6
 
 # 2. Run migrations against the live database
 docker run --rm \
   --env-file .env \
-  ghcr.io/fireflyframework/flycanon:26.10.5 migrate
+  ghcr.io/fireflyframework/flycanon:26.10.6 migrate
 
 # 3. Roll the API + worker deployments
 docker compose --env-file .env up -d

docs/released-server-install.md

@@ -6,13 +6,13 @@ Use the released source archive for the operator configuration and migration fil
 
 ## Download and verify
 
-Download `flycanon-26.10.5-py3-none-any.whl`, `flycanon-26.10.5.tar.gz`, `runtime-requirements-26.10.5.txt`, and `SHA256SUMS` from the [26.10.5 release](https://github.com/fireflyframework/flycanon/releases/tag/v26.10.5). Verify the downloaded files against their matching checksum entries before extracting or installing them.
+Download `flycanon-26.10.6-py3-none-any.whl`, `flycanon-26.10.6.tar.gz`, `runtime-requirements-26.10.6.txt`, and `SHA256SUMS` from the [26.10.6 release](https://github.com/fireflyframework/flycanon/releases/tag/v26.10.6). Verify the downloaded files against their matching checksum entries before extracting or installing them.
 
 ```bash
-tar -xzf flycanon-26.10.5.tar.gz
-cd flycanon-26.10.5
+tar -xzf flycanon-26.10.6.tar.gz
+cd flycanon-26.10.6
 uv venv --python 3.13 .venv
-uv pip install --python .venv/bin/python -r ../runtime-requirements-26.10.5.txt
+uv pip install --python .venv/bin/python -r ../runtime-requirements-26.10.6.txt
 ```
 
 The requirements file selects the released server wheel and these canonical framework references:
@@ -26,7 +26,7 @@ The equivalent installation without a requirements file is:
 
 ```bash
 uv pip install --python .venv/bin/python \
-  'flycanon @ https://github.com/fireflyframework/flycanon/releases/download/v26.10.5/flycanon-26.10.5-py3-none-any.whl' \
+  'flycanon @ https://github.com/fireflyframework/flycanon/releases/download/v26.10.6/flycanon-26.10.6-py3-none-any.whl' \
   'pyfly @ git+https://github.com/fireflyframework/fireflyframework-pyfly.git@v26.09.04' \
   'fireflyframework-agentic @ https://github.com/fireflyframework/fireflyframework-agentic/releases/download/v26.10.1/fireflyframework_agentic-26.10.1-py3-none-any.whl'
 ```
@@ -39,11 +39,11 @@ This release retains pyfly `v26.09.04` and the existing PostgreSQL EDA outbox. A
 
 ## Configure the runtime
 
-Keep `pyfly.yaml`, `alembic.ini`, `migrations/`, and `env_template` from the source archive in the working directory. The wheel does not supply the root-level `pyfly.yaml`. The release also provides a byte-identical `pyfly-26.10.5.yaml` copy for operators who already have a migration scaffold; install it as `pyfly.yaml` in the runtime working directory.
+Keep `pyfly.yaml`, `alembic.ini`, `migrations/`, and `env_template` from the source archive in the working directory. The wheel does not supply the root-level `pyfly.yaml`. The release also provides a byte-identical `pyfly-26.10.6.yaml` copy for operators who already have a migration scaffold; install it as `pyfly.yaml` in the runtime working directory.
 
 The configuration selects the PostgreSQL event adapter and separate API/worker consumer groups. Starting without it can activate framework defaults, including an attempted Kafka connection to `localhost:9092` and a separate management listener. Do not rely on those defaults for the released service.
 
-Set the database URL, embedding model and dimensions, provider credentials, platform API keys, object-storage location, and optional Redis settings for your environment. Use [`env_template`](https://github.com/fireflyframework/flycanon/blob/v26.10.5/env_template) and the [deployment guide](https://github.com/fireflyframework/flycanon/blob/v26.10.5/docs/deployment.md) for the exact settings. Store real secrets outside published configuration and supply them through your operator's environment or secret manager.
+Set the database URL, embedding model and dimensions, provider credentials, platform API keys, object-storage location, and optional Redis settings for your environment. Use [`env_template`](https://github.com/fireflyframework/flycanon/blob/v26.10.6/env_template) and the [deployment guide](https://github.com/fireflyframework/flycanon/blob/v26.10.6/docs/deployment.md) for the exact settings. Store real secrets outside published configuration and supply them through your operator's environment or secret manager.
 
 Provision the PostgreSQL database, required extensions and roles using the deployment guide. Apply migrations as a separate operator step before serving a new database:
 

openapi.json

@@ -5103,7 +5103,7 @@
       "url": "https://github.com/fireflyframework/flycanon/blob/main/LICENSE"
     },
     "title": "flycanon",
-    "version": "26.10.5"
+    "version": "26.10.6"
   },
   "openapi": "3.1.0",
   "paths": {

pyproject.toml

@@ -4,7 +4,7 @@
 name = "flycanon"
 # CalVer YY.MM.PP -- bumped per release. Note that PEP 440 normalises
 # ``26.05.04`` -> ``26.5.4`` in the built wheel filename.
-version = "26.10.5"
+version = "26.10.6"
 description = "Operational Knowledge Repository -- versioned, provenance-tracked canonical knowledge with hybrid retrieval and RAG, exposed as a standalone HTTP microservice over fireflyframework-pyfly and -agentic. Part of Firefly Framework, platform-agnostic by design."
 readme = "README.md"
 requires-python = ">=3.13,<3.14"

sdks/java/README.md

@@ -41,16 +41,19 @@ Compatible with **flycanon service version `26.10.x`**.
 
 | SDK | Service |
 |-----|---------|
+| `26.10.6` | `26.10.x` (same request/response schemas; workspace creation requires matching scope) |
 | `26.10.5` | `26.10.x` (same request/response schemas; workspace creation requires matching scope) |
 | `26.5.7` | `26.5.x` |
 
+Nonempty source selectors are applied before BM25 and dense ranking by service `26.10.6` and later. The request schema and SDK search methods are unchanged; older compatible SDKs can use the corrected server.
+
 ## Install
 
 ```xml
 <dependency>
   <groupId>com.firefly</groupId>
   <artifactId>flycanon-sdk</artifactId>
-  <version>26.10.5</version>
+  <version>26.10.6</version>
 </dependency>
 ```
 
@@ -418,7 +421,7 @@ use).
 ## Versioning
 
 The SDK pins its version to the service's CalVer (`YY.MM.PP`), so
-`flycanon-sdk@26.10.5` is the matching client for service version
+`flycanon-sdk@26.10.6` is the matching client for service version
 `26.10.x`. Upgrade the SDK in lockstep with the service. Both
 clients (`CanonClient`, `ReactiveCanonClient`) ship from the same
 artifact -- no separate dependency to manage.

sdks/java/pom.xml

@@ -19,7 +19,7 @@
 
   <groupId>com.firefly</groupId>
   <artifactId>flycanon-sdk</artifactId>
-  <version>26.10.5</version>
+  <version>26.10.6</version>
   <packaging>jar</packaging>
 
   <name>flycanon-sdk</name>

sdks/java/src/main/java/com/firefly/flycanon/sdk/CanonClient.java

@@ -94,7 +94,7 @@ public final class CanonClient {
     static final String HEADER_IDEMPOTENCY_KEY = "Idempotency-Key";
 
     /** SDK version. Mirrors {@code pom.xml}. */
-    public static final String SDK_VERSION = "26.10.5";
+    public static final String SDK_VERSION = "26.10.6";
 
     private final RestClient restClient;
     private final ObjectMapper mapper;

sdks/python/README.md

@@ -33,17 +33,20 @@ Compatible with **flycanon service version `26.10.x`**.
 
 | SDK | Service |
 |-----|---------|
+| `26.10.6` | `26.10.x` (same request/response schemas; workspace creation requires matching scope) |
 | `26.10.5` | `26.10.x` (same request/response schemas; workspace creation requires matching scope) |
 | `26.8.0` | `26.8.x` (embedding sets, `flycanon reindex`, `ChunkStats.by_embedding_model`) |
 | `26.7.1` | `26.7.x` (routes realigned: `?mode=async`, `/query/stream`, `/conversations/{id}/turn`, `/query/suggest`, `?after_id=`; `X-API-Key`; `delete_source`, `purge_workspace`) |
 | `26.5.7` | `26.5.x` |
 
+Nonempty source selectors are applied before BM25 and dense ranking by service `26.10.6` and later. The request schema and SDK search methods are unchanged; older compatible SDKs can use the corrected server.
+
 ## Install
 
 ```bash
-uv add https://github.com/fireflyframework/flycanon/releases/download/v26.10.5/flycanon_sdk-26.10.5-py3-none-any.whl
+uv add https://github.com/fireflyframework/flycanon/releases/download/v26.10.6/flycanon_sdk-26.10.6-py3-none-any.whl
 # or
-pip install https://github.com/fireflyframework/flycanon/releases/download/v26.10.5/flycanon_sdk-26.10.5-py3-none-any.whl
+pip install https://github.com/fireflyframework/flycanon/releases/download/v26.10.6/flycanon_sdk-26.10.6-py3-none-any.whl
 ```
 
 ## Quick start

sdks/python/pyproject.toml

@@ -2,7 +2,7 @@
 # SPDX-License-Identifier: Apache-2.0
 [project]
 name = "flycanon-sdk"
-version = "26.10.5"
+version = "26.10.6"
 description = "Async-first Python client for the flycanon Operational Knowledge Repository service."
 readme = "README.md"
 requires-python = ">=3.11"

sdks/python/src/flycanon_sdk/__init__.py

@@ -120,7 +120,7 @@
     WorkspaceUpdated,
 )
 
-__version__ = "26.10.5"
+__version__ = "26.10.6"
 
 __all__ = [
     "CANON_WORKSPACES_TOPIC",

sdks/python/tests/test_client.py

@@ -592,7 +592,7 @@ async def test_answer_returns_typed_response() -> None:
 
 
 def test_sdk_version_declares_current_release() -> None:
-    assert __version__ == "26.10.5"
+    assert __version__ == "26.10.6"
 
 
 def test_canon_workspaces_topic_constant() -> None:
@@ -674,7 +674,7 @@ async def test_user_agent_advertises_sdk_version() -> None:
             }
         )
         await client.version()
-        assert "flycanon-sdk-python/26.10.5" in route.calls.last.request.headers["User-Agent"]
+        assert "flycanon-sdk-python/26.10.6" in route.calls.last.request.headers["User-Agent"]
 
 
 # ----------------------------------------------------------------------

src/flycanon/core/services/retrieval/postgres_corpus.py

@@ -66,7 +66,7 @@
 
 import asyncio
 import logging
-from collections.abc import AsyncIterator
+from collections.abc import AsyncIterator, Sequence
 from contextlib import asynccontextmanager
 from typing import Any
 
@@ -184,6 +184,7 @@ async def bm25_search(
         top_k: int = 30,
         tenant_id: str = "default",
         workspace_id: str = "default",
+        source_ids: Sequence[str] | None = None,
     ) -> list[ChunkHit]:
         """Return the top-``top_k`` chunks ranked by Postgres BM25.
 
@@ -206,8 +207,9 @@ async def bm25_search(
         q = (query or "").strip()
         if not q:
             return []
+        source_predicate = "" if source_ids is None else "AND c.source_id = ANY(CAST(:source_ids AS text[]))"
         sql = text(
-            """
+            f"""
             SELECT
                 c.id          AS chunk_id,
                 c.source_id   AS doc_id,
@@ -221,6 +223,7 @@ async def bm25_search(
             JOIN canon_sources AS s ON s.id = c.source_id
             WHERE c.tenant_id = :tenant_id
               AND c.workspace_id = :workspace_id
+              {source_predicate}
               AND c.tsv @@ plainto_tsquery(:cfg, :q)
             ORDER BY score DESC, c.created_at DESC
             LIMIT :k
@@ -235,6 +238,7 @@ async def bm25_search(
                     "k": int(top_k),
                     "tenant_id": tenant_id,
                     "workspace_id": workspace_id,
+                    **({"source_ids": list(source_ids)} if source_ids is not None else {}),
                 },
             )
             rows = result.mappings().all()

src/flycanon/core/services/retrieval/retrieval_service.py

@@ -14,15 +14,10 @@
 
 """Hybrid retrieval service.
 
-Wraps :class:`HybridRetriever`, hydrates the resulting
-:class:`ChunkHit` rows with Postgres metadata, and applies the
-caller-supplied filters (domain / jurisdiction / status / explicit
-ids) as a post-retrieval pass.
-
-Filters run AFTER retrieval because :class:`HybridRetriever` does not
-expose pre-filter predicates -- pulling more candidates and trimming
-locally is the strategy here. The ``per_query_k`` knob lets callers
-compensate by widening the retrieval window.
+Binds explicit source membership into both BM25 and dense reads before their
+ranked candidate windows. Hydration still applies every caller filter as a
+defence-in-depth pass; knowledge status, domain and jurisdiction continue to
+use their hydrated metadata.
 """
 
 from __future__ import annotations
@@ -34,6 +29,8 @@
 from dataclasses import dataclass
 from typing import TYPE_CHECKING
 
+from fireflyframework_agentic.vectorstores.types import SearchFilter
+
 from flycanon.core.services.embeddings import (
     EmbeddingRegistry,
     EmbeddingService,
@@ -155,6 +152,8 @@ async def search(
                 "callers must thread the tenant scope explicitly."
             )
 
+        source_ids = tuple(filters.source_ids) if filters is not None and filters.source_ids else None
+
         from flycanon.core.services.retrieval.fusion import HybridRetriever
 
         # Which embedding space this workspace's corpus lives in, and the
@@ -174,12 +173,14 @@ async def search(
             self._context.corpus,
             tenant_id=tenant_id,
             workspace_id=workspace_id,
+            source_ids=source_ids,
         )
         scoped_vector_store = _ScopedVectorStore(
             self._context.vector_store,
             tenant_id=tenant_id,
             workspace_id=workspace_id,
             enabled=binding is not None or self._embedding_sets is None,
+            source_ids=source_ids,
         )
         # The embedder protocol is honoured by EmbeddingService
         # via the ``embed`` method; pass it through directly.
@@ -489,17 +490,21 @@ class _ScopedCorpus:
     through :class:`IndexService` directly, with its own scope threading.
     """
 
-    def __init__(self, inner: object, *, tenant_id: str, workspace_id: str) -> None:
+    def __init__(
+        self, inner: object, *, tenant_id: str, workspace_id: str, source_ids: Sequence[str] | None = None
+    ) -> None:
         self._inner = inner
         self._tenant_id = tenant_id
         self._workspace_id = workspace_id
+        self._source_ids = source_ids
 
     async def bm25_search(self, query: str, *, top_k: int = 30):  # noqa: ANN201
         return await self._inner.bm25_search(  # type: ignore[attr-defined]
             query,
             top_k=top_k,
             tenant_id=self._tenant_id,
             workspace_id=self._workspace_id,
+            **({"source_ids": self._source_ids} if self._source_ids is not None else {}),
         )
 
     async def get_chunks(self, chunk_ids: list[str]):  # noqa: ANN201
@@ -524,10 +529,19 @@ class _ScopedVectorStore:
     foreign-scope chunk is dropped when hydration can't find it.
     """
 
-    def __init__(self, inner: object, *, tenant_id: str, workspace_id: str, enabled: bool = True) -> None:
+    def __init__(
+        self,
+        inner: object,
+        *,
+        tenant_id: str,
+        workspace_id: str,
+        enabled: bool = True,
+        source_ids: Sequence[str] | None = None,
+    ) -> None:
         self._inner = inner
         self._tenant_id = tenant_id
         self._workspace_id = workspace_id
+        self._source_ids = source_ids
         # ``False`` when the workspace has no embedding set: there are no
         # vectors to find, so the dense half returns nothing rather than
         # querying one embedding space with a vector from another. BM25 still
@@ -542,4 +556,9 @@ async def search(self, query_embedding, top_k: int = 5):  # noqa: ANN001, ANN201
             top_k=top_k,
             tenant_id=self._tenant_id,
             workspace_id=self._workspace_id,
+            **(
+                {"filters": [SearchFilter(field="source_id", operator="in", value=list(self._source_ids))]}
+                if self._source_ids is not None
+                else {}
+            ),
         )