Skip to content

Fix HTTP model-download response cleanup and URL diagnostics #799

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
irulappan151204 wants to merge 4 commits into
base: main
Choose a base branch
from
+474 −33
Open

Fix HTTP model-download response cleanup and URL diagnostics #799

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
cab3073
fix: close model download streams and remove URLs from diagnostics
irulappan151204 Oct 10, 2026
ae4bb0c
docs: document verified fork setup and engineering results
irulappan151204 Oct 10, 2026
d08a234
docs: reduce private account detail in the public report
irulappan151204 Oct 10, 2026
cf0965c
Merge branch 'main' into fix/model-download-resources
irulappan151204 Oct 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
170 changes: 170 additions & 0 deletions ENGINEERING_REPORT.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
# FastEmbed engineering report

Date: 10 October 2026 (Asia/Calcutta).

## Delivered change

Selected [qdrant/fastembed](https://github.com/qdrant/fastembed), created the
GitHub fork [irulappan151204/fastembed](https://github.com/irulappan151204/fastembed),
and implemented focused model-download reliability and diagnostic-security fixes.
The work is on `fix/model-download-resources`. The implementation and nine tests
are in commit `cab30737fc8234f903f09898d53e9ef094e9211c`; the README and this
report are committed separately. No changes were pushed to upstream.

## Selection and repository evaluation

All seven repositories exposed by the connected account were screened using
metadata and repository trees, with deeper inspection of the AI candidate and
the Python reporting applications' dependencies.

| Repository | Assessment |
| --- | --- |
| `Chatting-with-Local-RAG` | The clearest personal AI project: MIT-licensed PDF RAG using LangChain, Chroma, and Ollama. Last push December 2024; a single notebook, no automated tests, invalid package names such as `OllamaEmbeddings` in requirements, and deprecated imports. Reviving it would require establishing a maintained application structure and dependency baseline. |
| `DSR_2.0` and a private reporting repository | Flask/SQL reporting applications. Their dependency lists and inspected structure show reporting rather than an established AI pipeline. |
| `rafithub` | TypeScript fitness website. |
| `newPortfolio` and a private portfolio repository | TypeScript portfolio websites. |
| `irulappan151204` | GitHub profile configuration. |

Three external Python projects were compared: FastEmbed, NeuML's txtai, and
AnswerDotAI's rerankers. FastEmbed was pushed on 9 October 2026, has about
3,240 stars in the inspected snapshot, and combines an active maintainer,
compact codebase, real RAG use, and testable infrastructure. txtai was also
active but has a substantially broader orchestration and model surface;
rerankers' last observed push was December 2025. FastEmbed offered the best
balance for a bounded, verifiable contribution. This is an engineering judgment
about these inspected candidates, not a claim to have ranked all AI projects.

Upstream issue review included the
[tuple reranking bug](https://github.com/qdrant/fastembed/issues/796),
[non-finite embedding report](https://github.com/qdrant/fastembed/issues/688),
[CPU/GPU runtime collision](https://github.com/qdrant/fastembed/issues/608),
and [Hub 2.x support](https://github.com/qdrant/fastembed/issues/742).
The tuple and NaN bugs already have open pull requests
[797](https://github.com/qdrant/fastembed/pull/797) and
[691](https://github.com/qdrant/fastembed/pull/691), so their fixes were not
duplicated. The delivered bugs were found by inspecting the current downloader.

## Architecture, quality, dependencies, and documentation

The baseline is FastEmbed 0.9.0, commit
`d076f083519746d132888079e6e790f2193c9c08`.

Public facades select registered dense, sparse, image, late-interaction, and
cross-encoder implementations. Shared model management resolves cached files,
downloads weights/configuration, verifies metadata, and stages archives.
Tokenizers and image processors construct ONNX inputs; ONNX Runtime performs
inference. A worker pool supports ordered multiprocessing. Common numeric
helpers and MUVERA handle postprocessing.

The codebase uses typed interfaces, dataclass model descriptions, pytest,
Ruff, mypy, and CI across Python 3.10–3.14. Existing cache validation, archive
path checks, bounded HTTP timeouts, and staging logic are valuable safeguards.
The main test suite depends heavily on downloaded models and some hardware.
The downloader lacked exception-safe HTTP response ownership and exposed
custom source URLs in three diagnostics. The README's usage examples were
useful, but this fork adds a reproducible contributor workflow and configuration
guidance. No architecture rewrite or inference-algorithm changes were needed.

Runtime and test dependencies were installed from the existing `poetry.lock`.
Observed versions include NumPy 2.4.6, Requests 2.34.2, ONNX Runtime 1.30.0,
Hugging Face Hub 1.33.0, Tokenizers 0.23.2, Pillow 12.3.0, pytest 9.1.1, and
mypy 2.4.0. The upstream pre-commit Ruff pin, 0.3.4, was used for final lint
and formatting checks. `pyproject.toml` and `poetry.lock` remain unchanged.

## Reproduced issues and fixes

1. **Unclosed HTTP streams on failure.** Real local HTTP responses remained open
after 403/404 responses, malformed content-length values, and destination-open
errors. The response now has a context manager covering status validation,
header parsing, progress reporting, and file writing. Existing exception types,
timeouts, chunk sizes, return values, and successful file bytes are preserved.
2. **Sensitive source URLs in diagnostics.** Missing/zero-length warnings,
fallback-download logs, and missing-model-directory errors exposed a synthetic
signed-URL marker. Those diagnostics now omit the source URL; errors identify
the model and the warning uses the existing Loguru logger.
3. **Development-tool advisories.** A temporary Poetry 2.2.1 environment brought
in Dulwich 0.24.10. pip-audit reported 18 advisory entries: 14 for Dulwich and
four for Poetry, including overlapping advisory aliases. These were tooling
findings, not newly identified FastEmbed runtime vulnerabilities. Upgrading
the isolated tooling to Poetry 2.5.1 brought Dulwich 1.2.17 and resolved the
audit findings. The README pins the verified Poetry version. Poetry's
[published wheel-path advisory](https://github.com/python-poetry/poetry/security/advisories/GHSA-2599-h6xx-hpxp)
independently confirms the risk in the older tooling.

Nine HTTP regression/compatibility cases exercise real local transport rather
than mocked responses. They cover the failure paths, successful bytes and
closure, absent/zero length, fallback logs, and incomplete archives. Captured
responses and logger sinks are cleaned up by test fixtures. The local-download
test overrides external-model offline mode for isolation.

## Verification evidence

Windows, CPython 3.11.15, CPU inference. No API keys or paid inference services
were used.

| Verification | Observed result |
| --- | --- |
| Initial seven download tests, before production changes | 6 failed, 1 passed; four open-stream assertions and two URL-disclosure assertions reproduced the bugs. |
| Two added diagnostics tests, before their fixes | Both reproduced URL disclosure after fixing a logger-capture issue in the fixture. |
| Focused tests: download, model management, common utilities, image transforms, worker pool | 32 passed; the final run after the tooling upgrade took 8.37 seconds. |
| Existing `tests/test_text_cross_encoder.py`, `CI=1`, online model download | 6 passed in 52.72 seconds with the real `Xenova/ms-marco-MiniLM-L-6-v2` model: scores, batching, lazy loading, parallel inference, token counting, and session configuration. |
| Full suite, `HF_HUB_OFFLINE=1`, `CI=1`, dedicated cache | 57 passed, 23 skipped, 74 failed, 39 setup errors, 4 warnings. All 113 remaining failure/error entries report an uncached model. Full online-suite success is not claimed. |
| Ruff 0.3.4, explicit repository configuration | Check and format check passed for both changed Python files. |
| mypy with upstream CI flags | Success across 66 source files. |
| `uv pip check` | All 76 installed packages compatible. |
| pip-audit after tooling upgrade | 76 packages audited, zero known advisory entries, no skipped packages. This is a database snapshot, not proof of complete security. |
| Git checks | Diff whitespace check passed; license, notice, dependency declarations, and lockfile unchanged. |

The full offline run also exposed one environment-sensitive assertion in a new
test. Its fixture was corrected, and the full run was repeated; the final counts
above include the correction. The four warnings concern the upstream
ColModernVBERT model update. The accompanying offline log names every failed
and errored test; initial/final dependency-audit JSON files preserve that audit's
progression.

## Reproduction and local environment

Use the installation, configuration, and commands in the updated README.
Poetry 2.5.1 successfully installed the unchanged lockfile. This host initially
omitted `PROCESSOR_ARCHITECTURE`, causing Poetry's marker parser to fail;
setting the process variable to its actual value, `AMD64`, resolved that issue.

Git could not create files under Documents directly. The cloned source is in
the chat workspace's `work/fastembed`; its `.git` pointer refers to metadata in
`%TEMP%/codex-fastembed-20261010/.git`. The isolated virtual environment is
`%TEMP%/codex-fastembed-venv`. Caches were disabled or placed in temporary
directories. A fresh clone in a writable development directory is recommended
for continued work; the delivered Git bundle is a portable recovery copy.

## Licensing and security boundaries

The original Apache-2.0 `LICENSE`, Qdrant attribution, and third-party `NOTICE`
were retained without changes. A modified-file notice was added to the changed
source, and the README identifies the upstream revision and modifications.
Model weights have separate licenses, including restricted licenses mentioned
in `NOTICE`; the library's license does not replace them.

GitHub's connector lacks a fork operation and the browser was signed out, so
the fork was created through GitHub's API using the existing Git Credential
Manager credential in memory. It was not printed, embedded in a URL, or saved
to a file. Git remotes use ordinary HTTPS URLs. Production applications and
databases were not accessed or modified.

Kluster startup reported `connectionError`; automatic-review and
dependency-check tools were unavailable. No Kluster review result is claimed.
Testing, static checks, and the dependency-advisory audit are recorded separately.

## Remaining limitations and next steps

- Run the full online suite with all required licensed model assets and GPU
hardware, and the supported Python/OS matrix. The six real inference tests
cover one representative reranker, not every model family.
- The delivered diagnostics fixes do not sanitize all third-party HTTP
exceptions. Avoid indiscriminately logging raw exceptions or configuration;
a wider diagnostic audit would be a separate change.
- Monitor upstream PRs 691/797 and the CPU/GPU dependency and Hub-version
issues rather than applying broad compatibility changes without their tests.
- Keep the development tooling pin and advisory checks current. No throughput
improvement or memory-performance benchmark is claimed.
- Review this fork's branch before proposing an upstream contribution. No
upstream pull request or production deployment was made.
82 changes: 82 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,88 @@ pip install fastembed
pip install fastembed-gpu
```

## Development of this fork

This fork is based on [Qdrant FastEmbed](https://github.com/qdrant/fastembed) at
`d076f083519746d132888079e6e790f2193c9c08` (FastEmbed 0.9.0). It preserves
the upstream architecture, public API, dependency declarations, `poetry.lock`,
`LICENSE`, and `NOTICE`.

### Install and test the source

Python 3.10 or newer is required; this change was tested on Python 3.11.15 on
Windows using Poetry 2.5.1 and the existing dependency lockfile.

```bash
git clone https://github.com/irulappan151204/fastembed.git
cd fastembed
git switch fix/model-download-resources
python -m pip install "poetry==2.5.1"
poetry install --only main,test,types
poetry run pytest tests/test_gcs_download.py tests/test_model_management.py tests/test_common.py tests/test_image_transform.py tests/test_parallel_processor.py
poetry run mypy fastembed --disallow-incomplete-defs --disallow-untyped-defs --disable-error-code=import-untyped
```

For linting, the upstream pre-commit hooks pin Ruff 0.3.4. With `uv` installed:

```bash
uv tool run --from "ruff==0.3.4" ruff check --config pyproject.toml fastembed/common/model_management.py tests/test_gcs_download.py
uv tool run --from "ruff==0.3.4" ruff format --check --config pyproject.toml fastembed/common/model_management.py tests/test_gcs_download.py
```

The tests above use local HTTP fixtures and require no model weights, GPU, API
keys, paid services, or external model endpoints. The full suite is
`poetry run pytest`; it includes model downloads and hardware-dependent tests.
To run the representative reranker tests using CPU inference and the upstream
CI model-selection policy (which also cleans up downloaded test models):

```bash
CI=1 poetry run pytest tests/test_text_cross_encoder.py
```

### Environment configuration

FastEmbed accepts constructor arguments such as `cache_dir`, `threads`,
`providers`, and `local_files_only`. The following environment variables are
also useful:

| Variable | Purpose |
| --- | --- |
| `FASTEMBED_CACHE_PATH` | Choose a writable directory for downloaded models. |
| `HF_HUB_OFFLINE=1` | Require cached models; uncached inference tests will fail. |
| `HF_ENDPOINT` | Select a trusted Hugging Face mirror if required. |
| `HF_TOKEN` | Optional authentication for restricted models; set privately and never commit it. |

For example, in PowerShell:

```powershell
$env:FASTEMBED_CACHE_PATH = Join-Path $env:TEMP "fastembed-models"
poetry run pytest tests/test_text_cross_encoder.py
```

In restricted Windows environments, a missing `PROCESSOR_ARCHITECTURE` can
cause Poetry to report `Could not parse version constraint`. Set that variable
to the actual architecture for the current process (`AMD64` on the tested
64-bit Windows host). If Documents disallows tool writes, put the checkout and
virtual environment in a writable development directory, or redirect tool
caches to a writable directory. `pytest -p no:cacheprovider` and Ruff's
`--no-cache` avoid writing their caches into the checkout.

### Fixes in this fork

- Streamed model-download responses now close on success and failure, including
HTTP errors, malformed length headers, and destination-open errors. Existing
exception types, timeouts, chunk sizes, and return values are preserved.
- The missing-length warning, fallback-download log, and missing-model-directory
error omit custom source URLs, which may contain signed access tokens. The
warning uses the existing Loguru logger and still explains disabled progress.
- Nine regression and compatibility tests use a real local HTTP server to
verify resource cleanup, unchanged file bytes, and safe diagnostics.

These changes do not sanitize every third-party exception; callers should avoid
logging raw HTTP exceptions or sensitive configuration. Model weights have
their own licenses; see `NOTICE` and each model's license before using them.

## 📖 Quickstart

```python
Expand Down
70 changes: 37 additions & 33 deletions fastembed/common/model_management.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# Modified in this fork: close download streams and omit source URLs from diagnostics.
import os
import time
import gzip
Expand Down Expand Up @@ -126,7 +127,9 @@ def _get_model_description(cls, model_name: str) -> T:
@classmethod
def download_file_from_gcs(cls, url: str, output_path: str, show_progress: bool = True) -> str:
"""
Downloads a file from Google Cloud Storage.
Downloads a file from Google Cloud Storage, closing its HTTP stream on every exit.

Download warnings omit the URL because custom sources may contain credentials.

Args:
url (str): The URL to download the file from.
Expand All @@ -137,37 +140,38 @@ def download_file_from_gcs(cls, url: str, output_path: str, show_progress: bool
str: The path to the downloaded file.
"""

response = requests.get(url, stream=True, timeout=(10, 120))
with requests.get(url, stream=True, timeout=(10, 120)) as response:
# Handle HTTP errors
if response.status_code == 403:
raise PermissionError(
"Authentication Error: You do not have permission to access this resource. "
"Please check your credentials."
)
# Otherwise an error page gets written out as though it were the archive.
response.raise_for_status()

# Handle HTTP errors
if response.status_code == 403:
raise PermissionError(
"Authentication Error: You do not have permission to access this resource. "
"Please check your credentials."
)
# Otherwise an error page gets written out as though it were the archive.
response.raise_for_status()

# Get the total size of the file
total_size_in_bytes = int(response.headers.get("content-length", 0))

# Warn if the total size is zero
if total_size_in_bytes == 0:
print(f"Warning: Content-length header is missing or zero in the response from {url}.")

show_progress = bool(total_size_in_bytes and show_progress)

with tqdm(
total=total_size_in_bytes,
unit="iB",
unit_scale=True,
disable=not show_progress,
) as progress_bar:
with open(output_path, "wb") as file:
for chunk in response.iter_content(chunk_size=_DOWNLOAD_CHUNK_SIZE):
if chunk: # Filter out keep-alive new chunks
progress_bar.update(len(chunk))
file.write(chunk)
# Get the total size of the file
total_size_in_bytes = int(response.headers.get("content-length", 0))

# Do not log signed URLs or credentials embedded in custom source URLs.
if total_size_in_bytes == 0:
logger.warning(
"Content-length header is missing or zero; download progress is disabled."
)

show_progress = bool(total_size_in_bytes and show_progress)

with tqdm(
total=total_size_in_bytes,
unit="iB",
unit_scale=True,
disable=not show_progress,
) as progress_bar:
with open(output_path, "wb") as file:
for chunk in response.iter_content(chunk_size=_DOWNLOAD_CHUNK_SIZE):
if chunk: # Filter out keep-alive new chunks
progress_bar.update(len(chunk))
file.write(chunk)
return output_path

@classmethod
Expand Down Expand Up @@ -583,7 +587,7 @@ def retrieve_model_gcs(
model_tmp_dir = staging_dir / fast_model_name
if not model_tmp_dir.is_dir() or model_tmp_dir.is_symlink():
raise ValueError(
f"The archive from {source_url} has no {fast_model_name} directory"
f"The archive for {model_name} has no {fast_model_name} directory"
)

# Replace a stale empty model_dir, which Windows will not rename onto. rmdir leaves
Expand Down Expand Up @@ -726,7 +730,7 @@ def download_model(cls, model: T, cache_dir: str, retries: int = 3, **kwargs: An
)
except Exception:
if not local_files_only:
logger.error(f"Could not download model from url: {url_source}")
logger.error(f"Could not download model {model.model} from its URL source")
elif model.sources.deprecated_tar_struct:
# Nothing is downloaded from the bucket anymore, but an old copy may still be cached.
legacy_dir = Path(cache_dir) / f"fast-{model.model.split('/')[-1]}"
Expand Down
Loading