diff --git a/sync/README.md b/sync/README.md index 8161a1c..760dc17 100644 --- a/sync/README.md +++ b/sync/README.md @@ -12,7 +12,9 @@ The stack (`docker-compose.universal-3-5-pro.yml`) runs two containers — `sync (GPU) and `license-and-usage-proxy` — with no nginx load balancer and no separate ASR backend. Authentication and rate limiting are expected to be handled at your own infrastructure layer (reverse proxy / API gateway); the -service accepts all requests. +service does not validate credentials, but every request must still carry a +**non-empty `Authorization` header** (any value works). A missing or empty +header returns `401`, so make sure your proxy doesn't strip it. | File | API | Models served | GPU requirement | |------|-----|--------------|-----------------| @@ -59,19 +61,24 @@ curl -fsS http://localhost:8080/readyz ## Transcribe `POST /transcribe` takes `multipart/form-data` with a required `audio` part and -an optional `config` JSON part. Audio constraints: 80 ms – 120 s, ≤ 40 MB, -16-bit, mono or stereo, sample rate one of -`{8000, 16000, 22050, 24000, 32000, 44100, 48000}`. +an optional `config` JSON part. Accepted audio formats: **16-bit PCM WAV** +(`audio/wav`) or **raw S16LE PCM** (`audio/pcm`, with `sample_rate` and +`channels` in the config part) — compressed formats like MP3 are rejected +with `415`. Audio constraints: 80 ms – 120 s, ≤ 40 MB, 16-bit, mono or +stereo, sample rate one of `{8000, 16000, 22050, 24000, 32000, 44100, 48000}`. ```bash -curl -F 'audio=@sample.wav;type=audio/wav' \ +curl -F 'audio=@example/example_audio_file.wav;type=audio/wav' \ -F 'config={"language_code":"en"};type=application/json' \ - -H 'Authorization: self-hosted' \ + -H 'Authorization: any value works' \ http://localhost:8080/transcribe ``` -The full request/response contract (config fields, error envelope, response -shape) is documented in the service's `API.md`. +The optional `config` part also accepts `language_code`, `prompt`, +`word_boost`, and `conversation_context`. Unknown fields are silently ignored, +so double-check spelling if an option seems to have no effect. For transcription +options and further help, see the [AssemblyAI documentation](https://www.assemblyai.com/docs) +or reach out to your AssemblyAI contact. ## Running the sync example @@ -81,7 +88,8 @@ A Python example is provided in `example/`: cd example python -m venv sync_venv && source sync_venv/bin/activate pip install -r requirements.txt -python transcribe_file.py path/to/audio.wav +python transcribe_file.py # uses the bundled example_audio_file.wav +python transcribe_file.py path/to/audio.wav # or your own 16-bit PCM WAV ``` ## Production deployment recommendations @@ -94,6 +102,14 @@ for the license-and-usage-proxy. - **Hardware Requirements**: NVIDIA L40S, RTX PRO 4500, or RTX PRO 6000. The model weights use ~11 GB of VRAM; the remaining VRAM becomes vLLM KV cache and sets max concurrency (e.g. ~74 concurrent max-length requests on 96 GB — more VRAM, higher concurrency). Allow ~30 GB of disk for the ~23 GB Docker image plus working space. - **Deployment Strategy**: Sync requests are short-lived HTTP calls, so rolling deployments work well. Drain in-flight requests before stopping a container. - **Scaling**: The load signal that matters is concurrent in-flight `/transcribe` requests (equivalently, the total in-flight audio duration) — this is what fills the GPU KV cache. Scale out before the container saturates; once vLLM's queue backs up, latency climbs sharply. A container's capacity is bounded by KV-cache headroom (and thus GPU VRAM), so load-test your specific GPU to find the concurrency at which latency degrades, and set that as your scale-out threshold. -- **Authentication & rate limiting**: Handle these at your own reverse proxy / API gateway — the service accepts all requests. +- **Authentication & rate limiting**: Handle these at your own reverse proxy / API gateway — the service does not validate credentials (though every request must carry a non-empty `Authorization` header). - **Health Checks**: Use `GET /readyz` (200 once warm) as the target-group health check; `GET /healthz` is always 200. - **Monitoring**: Monitor logs during deployment and watch for warning-level messages. + +## Troubleshooting + +- **`deep_gemm` `AssertionError` traceback during warmup**: harmless. The + inference engine probes for an optional GEMM kernel at startup and falls + back when the probe fails; the traceback is noisy but does not affect + readiness or transcription quality. The container is healthy once + `GET /readyz` returns `200`. diff --git a/sync/example/example_audio_file.wav b/sync/example/example_audio_file.wav new file mode 100644 index 0000000..453f59d Binary files /dev/null and b/sync/example/example_audio_file.wav differ diff --git a/sync/example/transcribe_file.py b/sync/example/transcribe_file.py index 7cb3a6b..2893236 100644 --- a/sync/example/transcribe_file.py +++ b/sync/example/transcribe_file.py @@ -3,27 +3,30 @@ POSTs a WAV file to the local sync-api container and prints the transcript. Usage: - python transcribe_file.py path/to/audio.wav + python transcribe_file.py [path/to/audio.wav] -The self-hosted stack uses the no-op SelfHostedAuthorizer, so any non-empty -Authorization value is accepted and there is no ALB, so no X-AAI-Model header -is required. Against the cloud API both would carry real values. +Defaults to the bundled example_audio_file.wav when no path is given. + +The self-hosted service does not validate credentials, but every request +must carry a non-empty Authorization header — any value works. """ import json +import pathlib import sys import requests SYNC_ENDPOINT = "http://localhost:8080/transcribe" +DEFAULT_AUDIO = pathlib.Path(__file__).parent / "example_audio_file.wav" def main() -> None: - if len(sys.argv) != 2: - print("usage: python transcribe_file.py ", file=sys.stderr) + if len(sys.argv) > 2: + print("usage: python transcribe_file.py [audio.wav]", file=sys.stderr) sys.exit(2) - audio_path = sys.argv[1] + audio_path = sys.argv[1] if len(sys.argv) == 2 else str(DEFAULT_AUDIO) with open(audio_path, "rb") as f: audio = f.read() @@ -37,7 +40,7 @@ def main() -> None: "application/json", ), }, - headers={"Authorization": "self-hosted"}, + headers={"Authorization": "any value works"}, timeout=120, ) response.raise_for_status()