From a0a69ab006b39e072d286d7ecf471ee0fed5e4a1 Mon Sep 17 00:00:00 2001 From: Mahbubur Rahman Date: Wed, 14 Oct 2026 10:49:02 -0400 Subject: [PATCH 1/2] Add Linux desktop clients (tray recorder + local dictation) under linux/ Move the standalone live-transcription-client daemons into the repo so the desktop client work lives with the server: tray_recorder.py (F9/F10, pw-record -> /api/record -> clipboard) and live_dictate.py (F8 toggle, pw-record -> faster-whisper offline STT -> xdotool types at cursor, notify-send notifications). Systemd user units updated to the new path; OS-specific, hence linux/. --- linux-dictation-client.md | 36 ++ linux/.gitignore | 8 + linux/README.md | 172 ++++++ linux/live-dictation.service | 15 + linux/live-transcription-client.service | 15 + linux/live_dictate.py | 581 ++++++++++++++++++ linux/requirements.txt | 6 + linux/test_api_roundtrip.py | 95 +++ linux/tray_recorder.py | 372 +++++++++++ .../WORK LOG - linux-dictation-client.md | 50 ++ 10 files changed, 1350 insertions(+) create mode 100644 linux-dictation-client.md create mode 100644 linux/.gitignore create mode 100644 linux/README.md create mode 100644 linux/live-dictation.service create mode 100644 linux/live-transcription-client.service create mode 100755 linux/live_dictate.py create mode 100644 linux/requirements.txt create mode 100755 linux/test_api_roundtrip.py create mode 100755 linux/tray_recorder.py create mode 100644 work-logs/WORK LOG - linux-dictation-client.md diff --git a/linux-dictation-client.md b/linux-dictation-client.md new file mode 100644 index 00000000..bbe636d6 --- /dev/null +++ b/linux-dictation-client.md @@ -0,0 +1,36 @@ +# Plan: Add Linux desktop clients (tray recorder + local live dictation) under `linux/` + +## Context +The desktop clients for this project were developed in a standalone repo +(`~/projects/live-transcription-client`, 3 commits) and are working in +production on this box. The user wants them preserved inside the established +Scriberr repo so they aren't lost, under a platform-specific `linux/` directory +(X11/PipeWire specifics — not portable to other OSes). + +## Source material +`~/projects/live-transcription-client` — tracked files only: +- `tray_recorder.py` — X11 tray daemon: F9/F10 hotkeys, pw-record → Flask `/api/record` → clipboard +- `live_dictate.py` — X11 tray daemon: F8 toggle, pw-record → faster-whisper (local/offline) → xdotool types at cursor, desktop notifications +- `test_api_roundtrip.py` — standalone API roundtrip test against the Flask service +- `live-transcription-client.service` / `live-dictation.service` — systemd user units +- `requirements.txt`, `README.md`, `.gitignore` +- EXCLUDED: `last-take.wav` (2.8MB debug artifact), `.venv/`, `__pycache__/` + +## Changes (branch: feature/linux-dictation-client) +1. Create `linux/` at repo root containing the 8 files above (README.md → `linux/README.md`, .gitignore → `linux/.gitignore`). +2. Update absolute install paths from `/home/atlas/projects/live-transcription-client` to `/home/atlas/projects/Scriberr/linux` in: + - both `.service` files (WorkingDirectory + ExecStart) + - `linux/README.md` (manual-run + setup instructions) +3. No code logic changes — the Python scripts derive their last-take path from `__file__`, so no other path references exist (verified). +4. Add `work-logs/WORK LOG - linux-dictation-client.md`. +5. Do NOT touch: the uncommitted `web/frontend/vite.config.ts` WIP, untracked `copy-transcript-to-clipboard.md`, or the existing untracked `work-logs/WORK LOG - copy-transcript-to-clipboard.md` (left uncommitted on purpose per earlier user request — stage ONLY the new work log file explicitly). + +## Acceptance criteria +- `git status` shows new files only under `linux/` + the new work log; nothing else staged. +- File contents byte-identical to source except the two path substitutions. +- `cd ~/projects/Scriberr/linux && ~/projects/live-transcription-client/.venv/bin/python live_dictate.py --test ~/projects/live-transcription-client/last-take.wav` prints a transcript and exits 0 (proves the moved script runs from its new home). +- `python -m py_compile` passes on all three .py files. + +## Gotchas +- `git add linux/ "work-logs/WORK LOG - linux-dictation-client.md"` — stage explicitly, never `git add -A` (would sweep in the floating WIP). +- A 41MB binary named `main` exists at repo root — always use `git diff main --` style path separation when diffing. diff --git a/linux/.gitignore b/linux/.gitignore new file mode 100644 index 00000000..73965854 --- /dev/null +++ b/linux/.gitignore @@ -0,0 +1,8 @@ +.venv/ +__pycache__/ +*.pyc +*.pyo +last-take.wav +*.wav +/tmp/ +*.log diff --git a/linux/README.md b/linux/README.md new file mode 100644 index 00000000..90ba9a58 --- /dev/null +++ b/linux/README.md @@ -0,0 +1,172 @@ +# Live Transcription Desktop Client + +A lightweight Linux (X11) system tray daemon that records microphone audio via global hotkeys or tray menu, transcribes it through the local `live-transcription` Flask backend, and copies the resulting transcript directly to your clipboard. + +--- + +## Live Dictation (Local / Offline) + +In addition to the server-backed transcription client (`tray_recorder.py`), this repository includes a standalone local dictation daemon (`live_dictate.py`). It captures audio directly from PipeWire (`pw-record`), transcribes speech locally in real-time using `faster-whisper` directly on this machine, and types the resulting text at the cursor of the currently focused X11 window using `xdotool`. No network service or server is required. + +- **Toggle Hotkey:** Press `F8` (default, configurable via `LT_DICTATE_HOTKEY`) once to start dictation, and press it again to stop and flush the final partial audio chunk. +- **Pipeline:** Records audio in streaming chunks (default 3 seconds), transcribes each chunk via `faster-whisper`, and types the text snippet at the active cursor with automatic space separation between chunks. +- **Tray Icon Indicator:** + - Gray: Idle + - Red: Dictating + - Yellow: Thinking / Transcribing chunk + - Green: Brief "done" notification on stop +- **Desktop Notifications:** Shows desktop notifications via `notify-send` when dictation starts ("good to go"), stops (with typed character count), and on errors. +- **Lazy Model Loading:** The Whisper model is loaded lazily on the first dictation start (displaying "loading model…" in the tray). +- **Prerequisites:** `xdotool` (`sudo apt install xdotool`), `pw-record`, and an active X11 desktop session. + +### Environment Variables + +| Variable | Default | Description | +|---|---|---| +| `LT_DICTATE_HOTKEY` | `` | Global toggle hotkey (e.g. ``, `++d`) | +| `LT_DICTATE_MODEL` | `base.en` | faster-whisper model name (e.g. `base.en`, `small.en`) | +| `LT_DICTATE_DEVICE` | `cpu` | Device to run transcription on (`cpu` or `cuda`) | +| `LT_DICTATE_COMPUTE` | `int8` | Model quantization / compute type (`int8`, `float16`, etc.) | +| `LT_DICTATE_CHUNK_SECONDS` | `3` | Duration of each streaming chunk in seconds | +| `LT_DICTATE_TYPE_DELAY_MS` | `10` | Milliseconds delay between typed keystrokes in xdotool | +| `LT_DICTATE_TARGET` | *(empty)* | Optional PipeWire node target serial or name for pw-record | + +> [!NOTE] +> GPU acceleration is optional: setting `LT_DICTATE_DEVICE=cuda` requires `pip install nvidia-cublas-cu12 nvidia-cudnn-cu12`. On modern multi-core CPUs, `base.en` with `int8` compute runs ~25–35x faster than realtime, so a GPU is not needed for base.en. + +### Running Live Dictation + +#### 1. Test Mode (`--test`) +You can verify audio decoding and local Whisper transcription against an audio file without starting the tray daemon or hotkeys: +```bash +.venv/bin/python live_dictate.py --test [FILE] +``` +If `FILE` is omitted, it defaults to `last-take.wav` in the repository root. This mode decodes the audio via an `ffmpeg` subprocess directly to 16kHz mono float32, transcribes it, and outputs the full transcript along with execution timing and realtime multiplier. + +#### 2. Running Manually +```bash +DISPLAY=:1 .venv/bin/python live_dictate.py +``` + +#### 3. Systemd User Service Setup +To run the live dictation daemon automatically in your desktop session: +```bash +mkdir -p ~/.config/systemd/user +cp live-dictation.service ~/.config/systemd/user/ +systemctl --user daemon-reload +systemctl --user enable --now live-dictation.service +``` +Check status or logs: +```bash +systemctl --user status live-dictation.service +journalctl --user -u live-dictation.service -f +``` + +--- + + +## Features + +- **Global Hotkeys:** Start recording with `F9`, stop and transcribe with `F10`. +- **System Tray Icon:** Visual status indicator generated dynamically via Pillow (no static image files needed): + - Gray: Idle + - Red: Recording + - Yellow/Orange: Transcribing + - Green: Copied to clipboard +- **Clipboard Integration:** Automatically copies finished transcript using `xclip`. +- **Debugging & Audit:** Saves the most recent audio recording to `last-take.wav` in the project root. +- **Single Instance:** Enforces one running instance using `/tmp/live-transcription-client.lock`. +- **Clean Shutdown:** Stops any active audio recording safely before exiting. + +--- + +## Hotkeys & Configuration + +Default hotkeys: +- **Start recording:** `` +- **Stop & transcribe:** `` + +### Environment Variables + +| Variable | Default | Description | +|---|---|---| +| `LT_SERVICE_URL` | `http://10.25.0.112:11060` | Base URL of the live-transcription Flask service | +| `LT_START_HOTKEY` | `` | Global hotkey to start recording (e.g. ``, `++r`) | +| `LT_STOP_HOTKEY` | `` | Global hotkey to stop recording and submit for transcription | + +--- + +## Setup & Running Manually + +### Prerequisites +- Python 3.10+ +- `pw-record` (provided by PipeWire) +- `xclip` (for clipboard access under X11) +- Active X11 graphical session (`DISPLAY` set) + +### Installation +```bash +cd /home/atlas/projects/Scriberr/linux +python3 -m venv .venv +.venv/bin/pip install -r requirements.txt +``` + +### Running Manually +```bash +.venv/bin/python tray_recorder.py +``` + +--- + +## Systemd User Service Setup + +To run the daemon automatically in your desktop session: + +1. Copy or link the service unit to your user systemd directory: + ```bash + mkdir -p ~/.config/systemd/user + cp live-transcription-client.service ~/.config/systemd/user/ + ``` + +2. Reload systemd daemon and enable the service: + ```bash + systemctl --user daemon-reload + systemctl --user enable --now live-transcription-client.service + ``` + +3. Check service status and logs: + ```bash + systemctl --user status live-transcription-client.service + journalctl --user -u live-transcription-client.service -f + ``` + +--- + +## Backend API Specification + +The client communicates with the Flask backend via: +1. `POST /api/record` + - Form-data multipart upload with `audio` (WAV file) and optional `title`. + - Returns HTTP 201: `{"conversation_id": , "scriberr_job_id": "...", "status": "processing"}`. +2. `GET /api/conversations//status` + - Polled every ~1.5s until completion (up to 300s). + - Returns JSON with `status` (`processing` | `done` | `failed`), `transcript` (string), or `error`. + +--- + +## Testing & Verification + +Run the standalone API roundtrip test: +```bash +.venv/bin/python test_api_roundtrip.py +``` +This script generates a 3-second 16kHz sine-wave audio sample in memory, submits it to `/api/record`, and polls until status is `done`. + +--- + +## Troubleshooting + +- **No tray icon appears:** Ensure you are running under an X11 desktop session and `DISPLAY` is exported (e.g., `export DISPLAY=:1`). +- **pw-record error:** The client relies on PipeWire's `pw-record` binary to capture microphone input. Ensure PipeWire is active (`systemctl --user status pipewire`). +- **Clipboard not updating:** Ensure `xclip` is installed (`which xclip`). +- **Global hotkeys not firing:** Under X11, ensure the window manager is not intercepting `F9`/`F10` exclusively, or customize the hotkeys using `LT_START_HOTKEY` and `LT_STOP_HOTKEY`. diff --git a/linux/live-dictation.service b/linux/live-dictation.service new file mode 100644 index 00000000..a82bf308 --- /dev/null +++ b/linux/live-dictation.service @@ -0,0 +1,15 @@ +[Unit] +Description=Live Dictation Tray Daemon (local whisper) +After=graphical-session.target pipewire.service +PartOf=graphical-session.target + +[Service] +Type=simple +WorkingDirectory=/home/atlas/projects/Scriberr/linux +ExecStart=/home/atlas/projects/Scriberr/linux/.venv/bin/python live_dictate.py +Restart=on-failure +RestartSec=3s +Environment=PYTHONUNBUFFERED=1 + +[Install] +WantedBy=graphical-session.target diff --git a/linux/live-transcription-client.service b/linux/live-transcription-client.service new file mode 100644 index 00000000..f7064308 --- /dev/null +++ b/linux/live-transcription-client.service @@ -0,0 +1,15 @@ +[Unit] +Description=Live Transcription Client Tray Daemon +After=graphical-session.target pipewire.service +PartOf=graphical-session.target + +[Service] +Type=simple +WorkingDirectory=/home/atlas/projects/Scriberr/linux +ExecStart=/home/atlas/projects/Scriberr/linux/.venv/bin/python tray_recorder.py +Restart=on-failure +RestartSec=3s +Environment=PYTHONUNBUFFERED=1 + +[Install] +WantedBy=graphical-session.target diff --git a/linux/live_dictate.py b/linux/live_dictate.py new file mode 100755 index 00000000..9572940e --- /dev/null +++ b/linux/live_dictate.py @@ -0,0 +1,581 @@ +#!/usr/bin/env python3 +""" +Live Dictation Desktop Client (Local / Offline X11 System Tray Daemon) +Records audio via PipeWire, transcribes locally with faster-whisper, +and types transcribed text directly at the cursor using xdotool. +""" + +import atexit +import fcntl +import logging +import os +import queue +import shutil +import signal +import subprocess +import sys +import threading +import time +from typing import Optional + +import numpy as np +from PIL import Image, ImageDraw +from pynput import keyboard +import pystray + +# Environment variables & configuration defaults +HOTKEY_DEFAULT = "" +MODEL_DEFAULT = "base.en" +DEVICE_DEFAULT = "cpu" +COMPUTE_DEFAULT = "int8" +CHUNK_SECONDS_DEFAULT = "3" +TYPE_DELAY_MS_DEFAULT = "10" +TARGET_DEFAULT = "" + +AUDIO_RATE = 16000 + +HOTKEY_RAW = os.environ.get("LT_DICTATE_HOTKEY", HOTKEY_DEFAULT) +MODEL_NAME = os.environ.get("LT_DICTATE_MODEL", MODEL_DEFAULT) +DEVICE = os.environ.get("LT_DICTATE_DEVICE", DEVICE_DEFAULT) +COMPUTE_TYPE = os.environ.get("LT_DICTATE_COMPUTE", COMPUTE_DEFAULT) +CHUNK_SECONDS = float(os.environ.get("LT_DICTATE_CHUNK_SECONDS", CHUNK_SECONDS_DEFAULT)) +TYPE_DELAY_MS = int(os.environ.get("LT_DICTATE_TYPE_DELAY_MS", TYPE_DELAY_MS_DEFAULT)) +TARGET = os.environ.get("LT_DICTATE_TARGET", TARGET_DEFAULT).strip() + +LOCK_PATH = "/tmp/live-dictation-client.lock" +PROJECT_DIR = os.path.dirname(os.path.abspath(__file__)) +DEFAULT_TEST_WAV = os.path.join(PROJECT_DIR, "last-take.wav") + +logging.basicConfig( + level=logging.INFO, + format="%(asctime)s [%(levelname)s] %(message)s", + datefmt="%Y-%m-%d %H:%M:%S", + stream=sys.stdout, +) +logger = logging.getLogger("live-dictate") +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(line_buffering=True) + + +def acquire_single_instance_lock(lock_path: str = LOCK_PATH): + """Enforces a single running instance. If lock is held, exit 0 silently.""" + try: + lock_file = open(lock_path, "w") + fcntl.flock(lock_file, fcntl.LOCK_EX | fcntl.LOCK_NB) + return lock_file + except (BlockingIOError, OSError): + sys.exit(0) + + +def normalize_key(k: str) -> str: + """Normalizes key names like 'f8' into ''.""" + k = k.strip().lower() + return f"<{k}>" if k in (f"f{i}" for i in range(1, 13)) else k + + +def format_hotkey_label(hotkey: str) -> str: + """Formats '' or '++d' into friendly display text like 'F8'.""" + clean = hotkey.replace("<", "").replace(">", "") + return "+".join(p.capitalize() if len(p) > 1 else p.upper() for p in clean.split("+")) + + +def create_circle_icon(color_rgba: tuple) -> Image.Image: + """Generates a 64x64 RGBA anti-aliased circle icon at runtime.""" + img = Image.new("RGBA", (128, 128), (0, 0, 0, 0)) + ImageDraw.Draw(img).ellipse((8, 8, 120, 120), fill=color_rgba) + return img.resize((64, 64), Image.Resampling.LANCZOS) + + +def notify(title: str, message: str, urgency: str = "normal"): + """Shows a desktop notification via notify-send; no-op with a debug log if missing.""" + if not shutil.which("notify-send"): + logger.debug("notify-send not found in PATH; skipping notification") + return + try: + subprocess.run( + ["notify-send", "-a", "Live Dictation", "-u", urgency, title, message], + timeout=5, + check=False, + ) + except Exception as e: + logger.debug(f"notify-send failed: {e}") + + +class LiveDictateApp: + def __init__(self): + self.lock = threading.Lock() + self.state = "idle" # idle | loading | dictating | stopping | done | error + self.status_desc = "idle" + self.is_shutting_down = False + + self.model = None # Loaded lazily on first dictation start + self.record_proc: Optional[subprocess.Popen] = None + self.reader_thread: Optional[threading.Thread] = None + self.worker_thread: Optional[threading.Thread] = None + self.chunk_queue: Optional[queue.Queue] = None + + self.hotkey_raw = HOTKEY_RAW + self.hotkey_normalized = normalize_key(self.hotkey_raw) + self.hotkey_label = format_hotkey_label(self.hotkey_normalized) + + self.last_typed_char: Optional[str] = None + self.chars_typed = 0 + self.hotkeys: Optional[keyboard.GlobalHotKeys] = None + + # Check for xdotool availability + if not shutil.which("xdotool"): + logger.warning("xdotool not found in PATH; typing will be a no-op.") + + # 64x64 runtime circle icons + self.icons = { + "idle": create_circle_icon((128, 128, 128, 255)), # Gray + "dictating": create_circle_icon((239, 68, 68, 255)), # Red + "thinking": create_circle_icon((245, 158, 11, 255)), # Yellow + "done": create_circle_icon((34, 197, 94, 255)), # Green + } + + # Build pystray tray menu + menu = pystray.Menu( + pystray.MenuItem(lambda item: f"Status: {self.status_desc}", None, enabled=False), + pystray.MenuItem( + lambda item: f"Dictate ({'Stop' if self.state == 'dictating' else 'Start'}) ({self.hotkey_label})", + lambda icon, item: self.toggle_dictation(), + ), + pystray.Menu.SEPARATOR, + pystray.MenuItem("Quit", lambda icon, item: self.quit()), + ) + self.icon = pystray.Icon( + name="live-dictate", + icon=self.icons["idle"], + title=f"Dictation: idle ({self.hotkey_label})", + menu=menu, + ) + + def _set_state_unlocked( + self, + state: str, + tooltip: str, + icon_key: Optional[str] = None, + desc: Optional[str] = None, + ): + """Update app state, tray icon image, and tooltip while holding self.lock.""" + self.state = state + self.status_desc = desc or state + try: + self.icon.icon = self.icons.get(icon_key or state, self.icons["idle"]) + self.icon.title = tooltip + except Exception as e: + logger.debug(f"Failed to update tray icon UI: {e}") + + def _revert_to_idle_after(self, delay: float, expected_state: str): + """Timer callback that reverts state to idle after transient messages (done/error).""" + time.sleep(delay) + with self.lock: + if not self.is_shutting_down and self.state == expected_state: + self._set_state_unlocked("idle", f"Dictation: idle ({self.hotkey_label})") + + def _set_error_transient(self, err_text: str): + """Display short error in tooltip for ~8s, then return to idle.""" + short_err = err_text[:60] + logger.error(f"Error state: {err_text}") + with self.lock: + self._set_state_unlocked("error", short_err, icon_key="idle", desc="error") + notify("Dictation error", err_text, urgency="critical") + threading.Thread(target=self._revert_to_idle_after, args=(8.0, "error"), daemon=True).start() + + def toggle_dictation(self): + """Toggle dictation session on/off. Ignored while loading or stopping.""" + with self.lock: + curr = self.state + + if curr in ("idle", "done"): + self.start_dictation() + elif curr == "dictating": + self.stop_dictation() + else: + logger.info(f"Hotkey toggle ignored while state is '{curr}'.") + + def start_dictation(self): + """Start audio dictation. Starting while busy is ignored.""" + with self.lock: + if self.state in ("loading", "dictating", "stopping"): + logger.info(f"Start requested while '{self.state}'; ignoring.") + return + + if not shutil.which("pw-record"): + self._set_error_transient("Error: pw-record not found in PATH") + return + + self.last_typed_char = None + self.chars_typed = 0 + + if self.model is None: + self._set_state_unlocked( + "loading", + f"loading model… ({self.hotkey_label})", + icon_key="thinking", + desc="loading model…", + ) + threading.Thread(target=self._start_pipeline_thread, args=(True,), daemon=True).start() + else: + self._set_state_unlocked( + "dictating", + f"dictating… press {self.hotkey_label} to stop", + icon_key="dictating", + desc="dictating", + ) + threading.Thread(target=self._start_pipeline_thread, args=(False,), daemon=True).start() + + def _start_pipeline_thread(self, load_model: bool): + """Loads model if needed and spawns pw-record and worker threads.""" + if load_model: + try: + from faster_whisper import WhisperModel + + logger.info(f"Loading Whisper model '{MODEL_NAME}' on {DEVICE} ({COMPUTE_TYPE})...") + self.model = WhisperModel(MODEL_NAME, device=DEVICE, compute_type=COMPUTE_TYPE) + logger.info(f"Whisper model '{MODEL_NAME}' loaded successfully.") + except Exception as e: + logger.error(f"Failed to load Whisper model: {e}", exc_info=True) + self._set_error_transient(f"Error loading model: {e}") + return + + with self.lock: + if self.is_shutting_down: + return + self._set_state_unlocked( + "dictating", + f"dictating… press {self.hotkey_label} to stop", + icon_key="dictating", + desc="dictating", + ) + + cmd = ["pw-record", f"--rate={AUDIO_RATE}", "--channels=1", "--format=s16", "-"] + if TARGET: + cmd.extend(["--target", TARGET]) + + logger.info(f"Starting pw-record: {' '.join(cmd)}") + try: + proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + except Exception as e: + logger.error(f"Failed to spawn pw-record: {e}") + self._set_error_transient(f"Error: {e}") + return + + time.sleep(0.15) + if proc.poll() is not None: + _, err_bytes = proc.communicate() + err = err_bytes.decode("utf-8", errors="replace").strip() or "pw-record failed to start" + logger.error(f"pw-record failed immediately: {err}") + self._set_error_transient(f"Error: {err}") + return + + chunk_queue: queue.Queue = queue.Queue() + with self.lock: + if self.is_shutting_down or self.state != "dictating": + try: + proc.send_signal(signal.SIGINT) + proc.wait(timeout=2) + except Exception: + proc.kill() + return + + self.record_proc = proc + self.chunk_queue = chunk_queue + + reader_thread = threading.Thread( + target=self._reader_worker, + args=(proc, chunk_queue), + daemon=True, + ) + worker_thread = threading.Thread( + target=self._transcribe_worker, + args=(chunk_queue,), + daemon=True, + ) + self.reader_thread = reader_thread + self.worker_thread = worker_thread + + reader_thread.start() + worker_thread.start() + logger.info("Dictation pipeline active.") + notify("Dictation started", f"You're good to go — start talking. Press {self.hotkey_label} to stop.") + if shutil.which("xdotool") is None: + notify( + "Dictation: no text output", + "xdotool is not installed; transcriptions will not be typed. Run: sudo apt install xdotool", + urgency="critical", + ) + + def _reader_worker(self, proc: subprocess.Popen, chunk_queue: queue.Queue): + """Reads raw s16 mono audio bytes from pw-record stdout and enqueues chunks.""" + chunk_bytes_needed = int(CHUNK_SECONDS * AUDIO_RATE * 2) + accumulated = bytearray() + try: + while True: + data = proc.stdout.read(4096) + if not data: + break + accumulated.extend(data) + while len(accumulated) >= chunk_bytes_needed: + chunk = bytes(accumulated[:chunk_bytes_needed]) + accumulated = accumulated[chunk_bytes_needed:] + chunk_queue.put(chunk) + except Exception as e: + logger.error(f"Reader thread exception: {e}") + finally: + min_partial_bytes = int(0.5 * AUDIO_RATE * 2) + if len(accumulated) >= min_partial_bytes: + logger.info( + f"Flushing final partial chunk ({len(accumulated)} bytes, {len(accumulated)/(AUDIO_RATE*2):.2f}s)" + ) + chunk_queue.put(bytes(accumulated)) + else: + logger.info(f"Discarding partial buffer under 0.5s ({len(accumulated)} bytes)") + chunk_queue.put(None) + + def _transcribe_worker(self, chunk_queue: queue.Queue): + """Processes chunks from queue, transcribes with Whisper, and types at cursor.""" + while True: + chunk = chunk_queue.get() + if chunk is None: + chunk_queue.task_done() + break + + if self.is_shutting_down: + chunk_queue.task_done() + break + + try: + # Indicate active transcription via yellow icon + with self.lock: + if not self.is_shutting_down: + try: + self.icon.icon = self.icons["thinking"] + except Exception: + pass + + audio = np.frombuffer(chunk, dtype=np.int16).astype(np.float32) / 32768.0 + transcribe_kwargs = {"beam_size": 1, "vad_filter": True} + if MODEL_NAME.endswith(".en"): + transcribe_kwargs["language"] = "en" + + segments, _ = self.model.transcribe(audio, **transcribe_kwargs) + text = " ".join(s.text.strip() for s in segments).strip() + + if text: + if ( + self.last_typed_char is not None + and not self.last_typed_char.isspace() + and not text[0].isspace() + ): + text = " " + text + self.type_text(text) + self.last_typed_char = text[-1] + except Exception as e: + logger.error(f"Error during chunk transcription/typing: {e}", exc_info=True) + finally: + with self.lock: + if not self.is_shutting_down and self.state == "dictating": + try: + self.icon.icon = self.icons["dictating"] + except Exception: + pass + chunk_queue.task_done() + + # Session finished: mark as done and schedule revert to idle + with self.lock: + if not self.is_shutting_down and self.state in ("stopping", "dictating"): + self._set_state_unlocked( + "done", + f"✓ Done ({self.hotkey_label})", + icon_key="done", + desc="done", + ) + notify( + "Dictation stopped", + f"Typed {self.chars_typed} characters at your cursor." + if self.chars_typed + else "Stopped — no speech detected.", + ) + threading.Thread(target=self._revert_to_idle_after, args=(2.5, "done"), daemon=True).start() + + def stop_dictation(self): + """Stop audio recording and flush final partial chunk.""" + with self.lock: + if self.state != "dictating": + logger.info(f"Stop requested while '{self.state}'; ignoring.") + return + + self._set_state_unlocked( + "stopping", + f"transcribing… ({self.hotkey_label})", + icon_key="thinking", + desc="transcribing…", + ) + proc = self.record_proc + self.record_proc = None + + threading.Thread(target=self._stop_pipeline_thread, args=(proc,), daemon=True).start() + + def _stop_pipeline_thread(self, proc: Optional[subprocess.Popen]): + """Sends SIGINT to pw-record with SIGTERM/SIGKILL fallback.""" + if proc and proc.poll() is None: + logger.info(f"Stopping pw-record (PID {proc.pid}) with SIGINT...") + proc.send_signal(signal.SIGINT) + try: + proc.wait(timeout=5) + except subprocess.TimeoutExpired: + logger.warning("pw-record did not exit after SIGINT; sending SIGTERM...") + proc.terminate() + try: + proc.wait(timeout=2) + except subprocess.TimeoutExpired: + logger.warning("pw-record did not exit after SIGTERM; killing...") + proc.kill() + proc.wait() + logger.info("pw-record stopped.") + + def type_text(self, text: str): + """Types text at cursor using xdotool.""" + if not shutil.which("xdotool"): + logger.warning(f"xdotool not found in PATH; skipping typing: {text!r}") + return + try: + cmd = ["xdotool", "type", "--clearmodifiers", f"--delay={TYPE_DELAY_MS}", "--", text] + subprocess.run(cmd, timeout=30, check=True) + self.chars_typed += len(text) + logger.info(f"Typed: {text!r}") + except Exception as e: + logger.error(f"Failed to type text via xdotool: {e}") + + def start_hotkeys(self): + """Binds global toggle hotkey via pynput.""" + try: + self.hotkeys = keyboard.GlobalHotKeys({ + self.hotkey_normalized: self.toggle_dictation, + }) + self.hotkeys.daemon = True + self.hotkeys.start() + logger.info(f"Bound global hotkey: {self.hotkey_normalized}") + except Exception as e: + logger.warning(f"Global hotkey binding warning ({e}); menu controls remain active.") + + def run(self): + """Runs the tray icon event loop in the main thread.""" + self.start_hotkeys() + logger.info( + f"Tray application running (MODEL={MODEL_NAME}, DEVICE={DEVICE}, COMPUTE={COMPUTE_TYPE}, CHUNK={CHUNK_SECONDS}s, HOTKEY={self.hotkey_normalized})." + ) + self.icon.run() + + def quit(self): + """Stops active dictation session, shuts down hotkeys and stops tray cleanly.""" + if self.is_shutting_down: + return + self.is_shutting_down = True + logger.info("Shutting down live dictation daemon...") + + with self.lock: + proc = self.record_proc + self.record_proc = None + + if proc and proc.poll() is None: + try: + proc.send_signal(signal.SIGINT) + proc.wait(timeout=2) + except Exception: + try: + proc.kill() + proc.wait() + except Exception: + pass + + if self.hotkeys: + try: + self.hotkeys.stop() + except Exception: + pass + + if self.icon: + try: + self.icon.stop() + except Exception: + pass + + +def run_test_mode(file_path: Optional[str] = None): + """Test mode: decode audio via ffmpeg, transcribe with faster-whisper, print timing and transcript.""" + if not file_path: + file_path = DEFAULT_TEST_WAV + + if not os.path.exists(file_path): + logger.error(f"Test audio file not found: {file_path}") + sys.exit(1) + + logger.info(f"Decoding {file_path} via ffmpeg...") + cmd = ["ffmpeg", "-i", file_path, "-f", "f32le", "-ar", f"{AUDIO_RATE}", "-ac", "1", "-"] + try: + proc = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) + except subprocess.CalledProcessError as e: + logger.error(f"ffmpeg decoding failed: {e}") + sys.exit(1) + + audio = np.frombuffer(proc.stdout, dtype=np.float32) + duration = len(audio) / float(AUDIO_RATE) + logger.info(f"Decoded {len(audio)} samples ({duration:.2f}s of audio).") + + logger.info(f"Loading Whisper model '{MODEL_NAME}' on {DEVICE} ({COMPUTE_TYPE})...") + from faster_whisper import WhisperModel + + t_load = time.time() + model = WhisperModel(MODEL_NAME, device=DEVICE, compute_type=COMPUTE_TYPE) + load_time = time.time() - t_load + logger.info(f"Model loaded in {load_time:.2f}s.") + + logger.info("Transcribing audio...") + t_start = time.time() + transcribe_kwargs = {"beam_size": 1, "vad_filter": True} + if MODEL_NAME.endswith(".en"): + transcribe_kwargs["language"] = "en" + segments, info = model.transcribe(audio, **transcribe_kwargs) + transcript = " ".join(s.text.strip() for s in segments).strip() + elapsed = time.time() - t_start + speedup = (duration / elapsed) if elapsed > 0 else 0.0 + + print("\n--- Transcript ---") + print(transcript) + print("------------------") + print(f"Audio duration: {duration:.2f}s") + print(f"Transcription time: {elapsed:.2f}s ({speedup:.1f}x realtime)\n") + sys.exit(0) + + +def main(): + if "--test" in sys.argv: + idx = sys.argv.index("--test") + test_file = None + if idx + 1 < len(sys.argv) and not sys.argv[idx + 1].startswith("-"): + test_file = sys.argv[idx + 1] + run_test_mode(test_file) + + lock_file = acquire_single_instance_lock() + atexit.register(lambda: (fcntl.flock(lock_file, fcntl.LOCK_UN), lock_file.close())) + + app = LiveDictateApp() + + def handle_signal(signum, frame): + app.quit() + + signal.signal(signal.SIGINT, handle_signal) + signal.signal(signal.SIGTERM, handle_signal) + + try: + app.run() + except Exception as e: + logger.error(f"Fatal exception: {e}", exc_info=True) + app.quit() + + +if __name__ == "__main__": + main() diff --git a/linux/requirements.txt b/linux/requirements.txt new file mode 100644 index 00000000..9b73a659 --- /dev/null +++ b/linux/requirements.txt @@ -0,0 +1,6 @@ +requests>=2.28.0 +pynput>=1.7.6 +pystray>=0.19.5 +pillow>=10.0.0 +faster-whisper>=1.2.1 +av>=18,<19 diff --git a/linux/test_api_roundtrip.py b/linux/test_api_roundtrip.py new file mode 100755 index 00000000..c307497f --- /dev/null +++ b/linux/test_api_roundtrip.py @@ -0,0 +1,95 @@ +#!/usr/bin/env python3 +""" +Test script for verifying round-trip transcription against the live Flask service. +Generates a 3s 16kHz mono sine-wave WAV using stdlib wave, POSTs to /api/record, +and polls /api/conversations//status until done or failed. +""" + +import io +import math +import os +import struct +import sys +import time +import wave +import requests + +SERVICE_URL = os.environ.get("LT_SERVICE_URL", "http://10.25.0.112:11060") +TIMEOUT_SECONDS = 120 +POLL_INTERVAL = 1.5 + + +def generate_sine_wav(duration_sec: float = 3.0, sample_rate: int = 16000, frequency: float = 440.0) -> bytes: + """Generate a valid 16kHz mono 16-bit PCM WAV in-memory using stdlib wave.""" + buf = io.BytesIO() + num_samples = int(duration_sec * sample_rate) + amplitude = 16000 # moderate volume for 16-bit signed PCM + with wave.open(buf, "wb") as wf: + wf.setnchannels(1) # mono + wf.setsampwidth(2) # 16-bit (2 bytes per sample) + wf.setframerate(sample_rate) + # Pack frames + frames = bytearray() + for i in range(num_samples): + sample = int(amplitude * math.sin(2.0 * math.pi * frequency * (i / sample_rate))) + frames.extend(struct.pack("") +STOP_HOTKEY = os.environ.get("LT_STOP_HOTKEY", "") +AUDIO_RATE = 16000 + +LOCK_PATH = "/tmp/live-transcription-client.lock" +PROJECT_DIR = os.path.dirname(os.path.abspath(__file__)) +LAST_TAKE_PATH = os.path.join(PROJECT_DIR, "last-take.wav") + +logging.basicConfig( + level=logging.INFO, + format="%(asctime)s [%(levelname)s] %(message)s", + datefmt="%Y-%m-%d %H:%M:%S", + stream=sys.stdout, +) +logger = logging.getLogger("live-transcription-client") +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(line_buffering=True) + + +def acquire_single_instance_lock(lock_path: str = LOCK_PATH): + """Enforces a single running instance. If lock is held, exit 0 silently.""" + try: + lock_file = open(lock_path, "w") + fcntl.flock(lock_file, fcntl.LOCK_EX | fcntl.LOCK_NB) + return lock_file + except (BlockingIOError, OSError): + sys.exit(0) + + +def format_hotkey_label(hotkey: str) -> str: + """Formats '' or '++r' into friendly display text like 'F9'.""" + clean = hotkey.replace("<", "").replace(">", "") + return "+".join(p.capitalize() if len(p) > 1 else p.upper() for p in clean.split("+")) + + +def create_circle_icon(color_rgba: tuple) -> Image.Image: + """Generates a 64x64 RGBA anti-aliased circle icon at runtime.""" + img = Image.new("RGBA", (128, 128), (0, 0, 0, 0)) + ImageDraw.Draw(img).ellipse((8, 8, 120, 120), fill=color_rgba) + return img.resize((64, 64), Image.Resampling.LANCZOS) + + +def copy_to_clipboard(text: str) -> bool: + """Copies text to the system clipboard using xclip.""" + try: + proc = subprocess.Popen(["xclip", "-selection", "clipboard"], stdin=subprocess.PIPE, close_fds=True) + proc.communicate(input=text.encode("utf-8"), timeout=5) + if proc.returncode == 0: + logger.info("Transcript copied to clipboard.") + return True + logger.error(f"xclip failed with code {proc.returncode}") + return False + except Exception as e: + logger.error(f"Failed to copy to clipboard: {e}") + return False + + +class TrayRecorderApp: + def __init__(self): + self.lock = threading.Lock() + self.state = "idle" # idle | recording | transcribing | done | error + self.status_desc = "idle" + self.is_shutting_down = False + self.record_proc: Optional[subprocess.Popen] = None + self.current_temp_path: Optional[str] = None + self.hotkeys: Optional[keyboard.GlobalHotKeys] = None + + # Pre-generate 64x64 runtime icons + self.icons = { + "idle": create_circle_icon((128, 128, 128, 255)), # Gray + "recording": create_circle_icon((239, 68, 68, 255)), # Red + "transcribing": create_circle_icon((245, 158, 11, 255)), # Yellow/Orange + "done": create_circle_icon((34, 197, 94, 255)), # Green + } + + self.start_label = format_hotkey_label(START_HOTKEY) + self.stop_label = format_hotkey_label(STOP_HOTKEY) + + # Build pystray tray menu + menu = pystray.Menu( + pystray.MenuItem(lambda item: f"Status: {self.status_desc}", None, enabled=False), + pystray.MenuItem(f"Start ({self.start_label})", lambda icon, item: self.start_recording()), + pystray.MenuItem(f"Stop ({self.stop_label})", lambda icon, item: self.stop_recording()), + pystray.Menu.SEPARATOR, + pystray.MenuItem("Quit", lambda icon, item: self.quit()), + ) + self.icon = pystray.Icon( + name="live-transcription-client", + icon=self.icons["idle"], + title="Transcription: idle", + menu=menu, + ) + + def _set_state_unlocked(self, state: str, tooltip: str, icon_key: Optional[str] = None, desc: Optional[str] = None): + """Update app state, tray icon image, and tooltip while holding self.lock.""" + self.state = state + self.status_desc = desc or state + try: + self.icon.icon = self.icons.get(icon_key or state, self.icons["idle"]) + self.icon.title = tooltip + except Exception as e: + logger.debug(f"Failed to update tray icon UI: {e}") + + def _revert_to_idle_after(self, delay: float, expected_state: str): + """Timer callback that reverts state to idle after transient messages (done/error).""" + time.sleep(delay) + with self.lock: + if not self.is_shutting_down and self.state == expected_state: + self._set_state_unlocked("idle", "Transcription: idle") + + def _set_error_transient(self, err_text: str): + """Display short error in tooltip for ~8s, then return to idle.""" + short_err = err_text[:60] + logger.error(f"Error state: {err_text}") + with self.lock: + self._set_state_unlocked("error", short_err, icon_key="idle", desc="error") + threading.Thread(target=self._revert_to_idle_after, args=(8.0, "error"), daemon=True).start() + + def start_recording(self): + """Start audio recording. Starting while busy (recording/transcribing) is ignored.""" + with self.lock: + if self.state in ("recording", "transcribing"): + logger.info(f"Start requested while '{self.state}'; ignoring.") + return + + if not shutil.which("pw-record"): + self._set_error_transient("Error: pw-record not found in PATH") + return + + try: + fd, tmp_path = tempfile.mkstemp(suffix=".wav", prefix="live-take-") + os.close(fd) + self.current_temp_path = tmp_path + + cmd = ["pw-record", f"--rate={AUDIO_RATE}", "--channels=1", tmp_path] + logger.info(f"Starting pw-record: {' '.join(cmd)}") + self.record_proc = subprocess.Popen(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE) + + # Verify pw-record started successfully + time.sleep(0.15) + if self.record_proc.poll() is not None: + _, err_bytes = self.record_proc.communicate() + err = err_bytes.decode("utf-8", errors="replace").strip() or "pw-record failed to start" + self.record_proc = None + self._set_error_transient(f"Error: {err}") + return + + self._set_state_unlocked("recording", f"recording… press {self.stop_label} to stop") + logger.info("Recording started.") + except Exception as e: + self._set_error_transient(f"Error: {e}") + + def stop_recording(self): + """Stop audio recording and launch asynchronous upload to transcription service.""" + with self.lock: + if self.state != "recording": + logger.info(f"Stop requested while '{self.state}'; ignoring.") + return + + proc = self.record_proc + self.record_proc = None + temp_path = self.current_temp_path + self.current_temp_path = None + self._set_state_unlocked("transcribing", "transcribing…") + + # Stop pw-record: send SIGINT, fallback to SIGTERM, then SIGKILL + if proc and proc.poll() is None: + logger.info(f"Stopping pw-record (PID {proc.pid}) with SIGINT...") + proc.send_signal(signal.SIGINT) + try: + proc.wait(timeout=5) + except subprocess.TimeoutExpired: + proc.terminate() + try: + proc.wait(timeout=2) + except subprocess.TimeoutExpired: + proc.kill() + proc.wait() + + # Save last take in project dir for debugging + if temp_path and os.path.exists(temp_path): + try: + shutil.copyfile(temp_path, LAST_TAKE_PATH) + logger.info(f"Saved take to {LAST_TAKE_PATH} ({os.path.getsize(LAST_TAKE_PATH)} bytes)") + except Exception as e: + logger.warning(f"Could not copy last-take.wav: {e}") + + # Run upload & polling in background thread so tray icon never freezes + threading.Thread(target=self._upload_and_poll, args=(temp_path,), daemon=True).start() + + def _upload_and_poll(self, temp_wav_path: Optional[str]): + """Uploads take to backend service and polls until done or failed.""" + try: + if not temp_wav_path or not os.path.exists(temp_wav_path) or os.path.getsize(temp_wav_path) == 0: + self._set_error_transient("Error: Audio recording is empty") + return + + logger.info(f"Uploading recording to {SERVICE_URL}/api/record...") + with open(temp_wav_path, "rb") as f: + files = {"audio": ("take.wav", f, "audio/wav")} + resp = requests.post(f"{SERVICE_URL}/api/record", files=files, data={"title": "Desktop Hotkey Take"}, timeout=60) + + if resp.status_code != 201: + err_msg = f"HTTP {resp.status_code}" + try: + err_json = resp.json() + if err_json.get("error"): + err_msg = str(err_json["error"]) + except Exception: + pass + self._set_error_transient(f"Error: {err_msg}") + return + + conv_id = resp.json().get("conversation_id") + if not conv_id: + self._set_error_transient("Error: Missing conversation_id in response") + return + + logger.info(f"Uploaded conversation {conv_id}; polling status...") + start_poll = time.time() + while time.time() - start_poll < 300.0: + if self.is_shutting_down: + return + + try: + poll = requests.get(f"{SERVICE_URL}/api/conversations/{conv_id}/status", timeout=15) + if poll.status_code == 200: + data = poll.json() + status = data.get("status") + if status == "done": + transcript = data.get("transcript") or "" + if not transcript.strip(): + transcript = "[no speech detected]" + logger.info(f"Transcription complete: {transcript[:80]!r}") + copy_to_clipboard(transcript) + + with self.lock: + self._set_state_unlocked("done", "✓ copied to clipboard") + threading.Thread(target=self._revert_to_idle_after, args=(4.0, "done"), daemon=True).start() + return + elif status == "failed": + err = data.get("error") or "Transcription failed" + self._set_error_transient(f"Error: {err}") + return + except requests.RequestException as e: + logger.warning(f"Polling warning: {e}") + + time.sleep(1.5) + + self._set_error_transient("Error: Transcription timed out") + except requests.RequestException as e: + self._set_error_transient(f"Network error: {e}") + except Exception as e: + logger.error(f"Upload flow error: {e}", exc_info=True) + self._set_error_transient(f"Error: {e}") + finally: + if temp_wav_path and os.path.exists(temp_wav_path): + try: + os.remove(temp_wav_path) + except Exception: + pass + + def start_hotkeys(self): + """Binds global hotkeys in a daemon thread via pynput.""" + def normalize_key(k: str) -> str: + k = k.strip().lower() + return f"<{k}>" if k in (f"f{i}" for i in range(1, 13)) else k + + start_key = normalize_key(START_HOTKEY) + stop_key = normalize_key(STOP_HOTKEY) + try: + self.hotkeys = keyboard.GlobalHotKeys({ + start_key: self.start_recording, + stop_key: self.stop_recording, + }) + self.hotkeys.daemon = True + self.hotkeys.start() + logger.info(f"Bound global hotkeys: Start={start_key}, Stop={stop_key}") + except Exception as e: + logger.warning(f"Global hotkeys binding warning ({e}); menu controls remain active.") + + def run(self): + """Runs the tray icon event loop in the main thread.""" + self.start_hotkeys() + logger.info(f"Tray application running (SERVICE_URL={SERVICE_URL}).") + self.icon.run() + + def quit(self): + """Stops active recording, shuts down hotkeys and stops tray icon cleanly.""" + if self.is_shutting_down: + return + self.is_shutting_down = True + logger.info("Shutting down live transcription client...") + + with self.lock: + if self.record_proc and self.record_proc.poll() is None: + try: + self.record_proc.send_signal(signal.SIGINT) + self.record_proc.wait(timeout=2) + except Exception: + try: + self.record_proc.kill() + except Exception: + pass + self.record_proc = None + + if self.current_temp_path and os.path.exists(self.current_temp_path): + try: + os.remove(self.current_temp_path) + except Exception: + pass + + if self.hotkeys: + try: + self.hotkeys.stop() + except Exception: + pass + + if self.icon: + try: + self.icon.stop() + except Exception: + pass + + +def main(): + lock_file = acquire_single_instance_lock() + atexit.register(lambda: (fcntl.flock(lock_file, fcntl.LOCK_UN), lock_file.close())) + + app = TrayRecorderApp() + + def handle_signal(signum, frame): + app.quit() + + signal.signal(signal.SIGINT, handle_signal) + signal.signal(signal.SIGTERM, handle_signal) + + try: + app.run() + except Exception as e: + logger.error(f"Fatal exception: {e}", exc_info=True) + app.quit() + + +if __name__ == "__main__": + main() diff --git a/work-logs/WORK LOG - linux-dictation-client.md b/work-logs/WORK LOG - linux-dictation-client.md new file mode 100644 index 00000000..e270d827 --- /dev/null +++ b/work-logs/WORK LOG - linux-dictation-client.md @@ -0,0 +1,50 @@ +# WORK LOG: Linux dictation client + +**Date:** 2026-10-14 +**Status:** Complete +**Commit:** to be committed + +## WHAT WAS ASKED +Move the standalone Linux desktop clients (tray recorder and local Whisper dictation daemons) from the separate repository `~/projects/live-transcription-client` into `linux/` in this repository so that desktop client work lives alongside the Scriberr server. + +## BEFORE +Desktop clients were developed and tracked in a standalone repository (`~/projects/live-transcription-client`). The systemd user services and documentation pointed directly to `/home/atlas/projects/live-transcription-client`. + +## AFTER +Tracked desktop client files live under `linux/` in the Scriberr repository: +- `linux/tray_recorder.py`: X11 tray daemon using F9/F10 global hotkeys, `pw-record` audio capture, Flask `/api/record` submission, and clipboard copy. +- `linux/live_dictate.py`: X11 tray daemon using F8 toggle hotkey, `pw-record` audio capture, offline `faster-whisper` transcription, `xdotool` typing at active cursor, and desktop notifications. +- `linux/test_api_roundtrip.py`: Standalone API roundtrip verification script. +- `linux/live-transcription-client.service` & `linux/live-dictation.service`: systemd user service units configured with `WorkingDirectory=/home/atlas/projects/Scriberr/linux` and ExecStart paths pointing to `.venv/bin/python`. +- `linux/requirements.txt`, `linux/.gitignore`, and `linux/README.md` (with updated paths). +Excluded runtime/build artifacts: `last-take.wav`, `.venv/`, `__pycache__/`, `.git/`. + +## WHAT WAS DONE +- Copied tracked files from `~/projects/live-transcription-client` to `linux/`: + - `tray_recorder.py` + - `live_dictate.py` + - `test_api_roundtrip.py` + - `live-transcription-client.service` + - `live-dictation.service` + - `requirements.txt` + - `.gitignore` + - `README.md` +- Performed path substitutions (`/home/atlas/projects/live-transcription-client` → `/home/atlas/projects/Scriberr/linux`) in: + - `linux/live-transcription-client.service`: `WorkingDirectory=` and `ExecStart=` lines. + - `linux/live-dictation.service`: `WorkingDirectory=` and `ExecStart=` lines. + - `linux/README.md`: Setup/installation instructions. +- Python files left untouched (they compute paths dynamically from `__file__`). +- Both daemons verified working in production on the dev box: + - F9/F10 recorder → `/api/record` → clipboard + - F8 toggle → `faster-whisper` local STT → `xdotool` types at cursor + `notify-send` notifications. + +## VERIFICATION +- Checked `git status --short`: clean except new untracked files in `linux/` and this work log; pre-existing modified/untracked files untouched. +- `diff -r` comparison against source files: Python files, requirements.txt, and .gitignore are byte-identical; service units and README differ only by path substitution. +- Bytecode compilation: `python3 -m py_compile` passed for `tray_recorder.py`, `live_dictate.py`, and `test_api_roundtrip.py`. +- Execution test: Ran `live_dictate.py --test /home/atlas/projects/live-transcription-client/last-take.wav` using the venv Python binary; verified it transcribes successfully and exits 0. +- Stale path check: Verified no references to `/home/atlas/projects/live-transcription-client` remain in `linux/`. + +## FOLLOW-UPS / NOTES +- Optional follow-up: user may migrate the live systemd install to the new path (`~/.config/systemd/user/`) after merge. +- Do not commit changes; user will commit. From 7662c8ad128a3815e26497a8e5dfc18baef13785 Mon Sep 17 00:00:00 2001 From: Mahbubur Rahman Date: Wed, 14 Oct 2026 13:02:19 -0400 Subject: [PATCH 2/2] live_dictate: phrase-aware VAD chunking, 0ms type delay, model warmup Dictation now flushes on natural speech pauses (Silero VAD, >=300ms silence) or at the 4s cap instead of fixed 3s chunks: text flows phrase by phrase with ~0.4s latency. LT_DICTATE_CHUNK_MODE=fixed keeps legacy behavior. New --test-chunks dev mode. Verified: 89s take -> 27 chunks, avg flush latency 0.36s. --- linux/README.md | 25 +- linux/live_dictate.py | 299 ++++++++++++++++-- .../WORK LOG - linux-dictation-client.md | 35 ++ 3 files changed, 327 insertions(+), 32 deletions(-) diff --git a/linux/README.md b/linux/README.md index 90ba9a58..3b9b7794 100644 --- a/linux/README.md +++ b/linux/README.md @@ -8,15 +8,17 @@ A lightweight Linux (X11) system tray daemon that records microphone audio via g In addition to the server-backed transcription client (`tray_recorder.py`), this repository includes a standalone local dictation daemon (`live_dictate.py`). It captures audio directly from PipeWire (`pw-record`), transcribes speech locally in real-time using `faster-whisper` directly on this machine, and types the resulting text at the cursor of the currently focused X11 window using `xdotool`. No network service or server is required. +Dictation is phrase-aware: Silero VAD watches the audio stream and flushes a chunk the moment you pause (~300ms silence) or at the 4s cap, so text appears phrase by phrase instead of in fixed bursts. + - **Toggle Hotkey:** Press `F8` (default, configurable via `LT_DICTATE_HOTKEY`) once to start dictation, and press it again to stop and flush the final partial audio chunk. -- **Pipeline:** Records audio in streaming chunks (default 3 seconds), transcribes each chunk via `faster-whisper`, and types the text snippet at the active cursor with automatic space separation between chunks. +- **Pipeline:** Records audio via PipeWire, segments speech dynamically using Silero VAD, transcribes each phrase via `faster-whisper`, and types the text snippet at the active cursor with automatic space separation between phrases. - **Tray Icon Indicator:** - Gray: Idle - Red: Dictating - Yellow: Thinking / Transcribing chunk - Green: Brief "done" notification on stop - **Desktop Notifications:** Shows desktop notifications via `notify-send` when dictation starts ("good to go"), stops (with typed character count), and on errors. -- **Lazy Model Loading:** The Whisper model is loaded lazily on the first dictation start (displaying "loading model…" in the tray). +- **Lazy Model Loading & Warmup:** The Whisper model is loaded lazily on the first dictation start (displaying "loading model…" in the tray) and warmed up with a dummy transcription so the first dictation chunk is responsive. - **Prerequisites:** `xdotool` (`sudo apt install xdotool`), `pw-record`, and an active X11 desktop session. ### Environment Variables @@ -27,8 +29,12 @@ In addition to the server-backed transcription client (`tray_recorder.py`), this | `LT_DICTATE_MODEL` | `base.en` | faster-whisper model name (e.g. `base.en`, `small.en`) | | `LT_DICTATE_DEVICE` | `cpu` | Device to run transcription on (`cpu` or `cuda`) | | `LT_DICTATE_COMPUTE` | `int8` | Model quantization / compute type (`int8`, `float16`, etc.) | -| `LT_DICTATE_CHUNK_SECONDS` | `3` | Duration of each streaming chunk in seconds | -| `LT_DICTATE_TYPE_DELAY_MS` | `10` | Milliseconds delay between typed keystrokes in xdotool | +| `LT_DICTATE_CHUNK_MODE` | `vad` | Chunking mode: `vad` (phrase-aware adaptive chunking) or `fixed` (legacy fixed-size chunks using `LT_DICTATE_CHUNK_SECONDS`) | +| `LT_DICTATE_MAX_CHUNK_SECONDS` | `4` | Maximum chunk duration in seconds before forcing a flush in VAD mode | +| `LT_DICTATE_MIN_CHUNK_SECONDS` | `0.5` | Minimum chunk duration in seconds before VAD probes for phrase ends | +| `LT_DICTATE_SILENCE_FLUSH_MS` | `300` | Trailing silence in milliseconds required to trigger a phrase-end flush | +| `LT_DICTATE_CHUNK_SECONDS` | `3` | Duration of each streaming chunk in seconds (used only when `LT_DICTATE_CHUNK_MODE=fixed`) | +| `LT_DICTATE_TYPE_DELAY_MS` | `0` | Milliseconds delay between typed keystrokes in xdotool (0 = type as fast as possible; raise to 5-20 if a slow app drops characters) | | `LT_DICTATE_TARGET` | *(empty)* | Optional PipeWire node target serial or name for pw-record | > [!NOTE] @@ -43,12 +49,19 @@ You can verify audio decoding and local Whisper transcription against an audio f ``` If `FILE` is omitted, it defaults to `last-take.wav` in the repository root. This mode decodes the audio via an `ffmpeg` subprocess directly to 16kHz mono float32, transcribes it, and outputs the full transcript along with execution timing and realtime multiplier. -#### 2. Running Manually +#### 2. Test Chunks Mode (`--test-chunks`) +Dev / verification mode that simulates live adaptive chunking over an audio file (defaults to `last-take.wav`) in 4096-byte steps without starting the tray daemon or hotkeys: +```bash +.venv/bin/python live_dictate.py --test-chunks [FILE] +``` +Transcribes each flushed chunk, displays flush timestamps, chunk durations, reasons (`phrase-end` vs `max`), and prints aggregate latency and typed character stats. + +#### 3. Running Manually ```bash DISPLAY=:1 .venv/bin/python live_dictate.py ``` -#### 3. Systemd User Service Setup +#### 4. Systemd User Service Setup To run the live dictation daemon automatically in your desktop session: ```bash mkdir -p ~/.config/systemd/user diff --git a/linux/live_dictate.py b/linux/live_dictate.py index 9572940e..f985d0d8 100755 --- a/linux/live_dictate.py +++ b/linux/live_dictate.py @@ -29,8 +29,12 @@ DEVICE_DEFAULT = "cpu" COMPUTE_DEFAULT = "int8" CHUNK_SECONDS_DEFAULT = "3" -TYPE_DELAY_MS_DEFAULT = "10" +TYPE_DELAY_MS_DEFAULT = "0" TARGET_DEFAULT = "" +CHUNK_MODE_DEFAULT = "vad" +MAX_CHUNK_SECONDS_DEFAULT = "4" +MIN_CHUNK_SECONDS_DEFAULT = "0.5" +SILENCE_FLUSH_MS_DEFAULT = "300" AUDIO_RATE = 16000 @@ -41,6 +45,9 @@ CHUNK_SECONDS = float(os.environ.get("LT_DICTATE_CHUNK_SECONDS", CHUNK_SECONDS_DEFAULT)) TYPE_DELAY_MS = int(os.environ.get("LT_DICTATE_TYPE_DELAY_MS", TYPE_DELAY_MS_DEFAULT)) TARGET = os.environ.get("LT_DICTATE_TARGET", TARGET_DEFAULT).strip() +MAX_CHUNK_SECONDS = float(os.environ.get("LT_DICTATE_MAX_CHUNK_SECONDS", MAX_CHUNK_SECONDS_DEFAULT)) +MIN_CHUNK_SECONDS = float(os.environ.get("LT_DICTATE_MIN_CHUNK_SECONDS", MIN_CHUNK_SECONDS_DEFAULT)) +SILENCE_FLUSH_MS = int(os.environ.get("LT_DICTATE_SILENCE_FLUSH_MS", SILENCE_FLUSH_MS_DEFAULT)) LOCK_PATH = "/tmp/live-dictation-client.lock" PROJECT_DIR = os.path.dirname(os.path.abspath(__file__)) @@ -56,6 +63,97 @@ if hasattr(sys.stdout, "reconfigure"): sys.stdout.reconfigure(line_buffering=True) +CHUNK_MODE_RAW = os.environ.get("LT_DICTATE_CHUNK_MODE", CHUNK_MODE_DEFAULT).strip().lower() +if CHUNK_MODE_RAW in ("vad", "fixed"): + CHUNK_MODE = CHUNK_MODE_RAW +else: + logger.warning(f"Unknown LT_DICTATE_CHUNK_MODE '{CHUNK_MODE_RAW}'; defaulting to 'vad'") + CHUNK_MODE = "vad" + + +class AdaptiveChunker: + """Phrase-aware audio chunker using Silero VAD to detect phrase boundaries.""" + + def __init__( + self, + min_chunk_seconds: float = MIN_CHUNK_SECONDS, + max_chunk_seconds: float = MAX_CHUNK_SECONDS, + silence_flush_ms: int = SILENCE_FLUSH_MS, + ): + self.min_chunk_seconds = min_chunk_seconds + self.max_chunk_seconds = max(max_chunk_seconds, min_chunk_seconds) + self.silence_flush_ms = silence_flush_ms + self.buffer = bytearray() + self._vad_get_timestamps = None + self._vad_opts = None + self._vad_error_warned = False + self.last_speech_end_samples: Optional[int] = None + + def _ensure_vad(self): + if self._vad_get_timestamps is None: + from faster_whisper.vad import get_speech_timestamps, VadOptions + + self._vad_get_timestamps = get_speech_timestamps + self._vad_opts = VadOptions( + threshold=0.5, + min_silence_duration_ms=250, + speech_pad_ms=100, + ) + + def feed(self, data: bytes) -> list[tuple[bytes, str]]: + """Accumulates audio bytes and flushes chunks on phrase boundaries or max duration.""" + self.buffer.extend(data) + flushed: list[tuple[bytes, str]] = [] + + min_bytes = int(self.min_chunk_seconds * AUDIO_RATE * 2) + max_bytes = int(self.max_chunk_seconds * AUDIO_RATE * 2) + silence_samples = int(self.silence_flush_ms / 1000.0 * AUDIO_RATE) + + if len(self.buffer) >= min_bytes: + phrase_end = False + last_segment_end = None + try: + self._ensure_vad() + usable_bytes = (len(self.buffer) // 2) * 2 + audio_f32 = ( + np.frombuffer(self.buffer[:usable_bytes], dtype=np.int16).astype(np.float32) + / 32768.0 + ) + buffer_samples = len(audio_f32) + segments = self._vad_get_timestamps(audio_f32, self._vad_opts) + if segments: + last_segment_end = segments[-1]["end"] + if (buffer_samples - last_segment_end) >= silence_samples: + phrase_end = True + except Exception as e: + if not self._vad_error_warned: + logger.warning(f"VAD probe failed; falling back to max-chunk flush: {e}") + self._vad_error_warned = True + phrase_end = False + last_segment_end = None + + if phrase_end: + chunk = bytes(self.buffer) + self.buffer.clear() + self.last_speech_end_samples = last_segment_end + flushed.append((chunk, "phrase-end")) + elif len(self.buffer) >= max_bytes: + chunk = bytes(self.buffer) + self.buffer.clear() + self.last_speech_end_samples = last_segment_end + flushed.append((chunk, "max")) + + return flushed + + def drain(self, min_bytes: int = 0) -> Optional[bytes]: + """Flushes remaining audio in buffer if >= min_bytes, else discards.""" + if len(self.buffer) >= min_bytes: + chunk = bytes(self.buffer) + self.buffer.clear() + return chunk + self.buffer.clear() + return None + def acquire_single_instance_lock(lock_path: str = LOCK_PATH): """Enforces a single running instance. If lock is held, exit 0 silently.""" @@ -235,6 +333,17 @@ def _start_pipeline_thread(self, load_model: bool): logger.info(f"Loading Whisper model '{MODEL_NAME}' on {DEVICE} ({COMPUTE_TYPE})...") self.model = WhisperModel(MODEL_NAME, device=DEVICE, compute_type=COMPUTE_TYPE) logger.info(f"Whisper model '{MODEL_NAME}' loaded successfully.") + + # Model warmup: transcribe 0.5s silence before setting dictating state + t_warmup = time.time() + warmup_audio = np.zeros(int(0.5 * AUDIO_RATE), dtype=np.float32) + transcribe_kwargs = {"beam_size": 1, "vad_filter": True} + if MODEL_NAME.endswith(".en"): + transcribe_kwargs["language"] = "en" + segs, _ = self.model.transcribe(warmup_audio, **transcribe_kwargs) + list(segs) + elapsed = time.time() - t_warmup + logger.info(f"Model warmed up in {elapsed*1000:.0f}ms") except Exception as e: logger.error(f"Failed to load Whisper model: {e}", exc_info=True) self._set_error_transient(f"Error loading model: {e}") @@ -309,30 +418,59 @@ def _start_pipeline_thread(self, load_model: bool): def _reader_worker(self, proc: subprocess.Popen, chunk_queue: queue.Queue): """Reads raw s16 mono audio bytes from pw-record stdout and enqueues chunks.""" - chunk_bytes_needed = int(CHUNK_SECONDS * AUDIO_RATE * 2) - accumulated = bytearray() - try: - while True: - data = proc.stdout.read(4096) - if not data: - break - accumulated.extend(data) - while len(accumulated) >= chunk_bytes_needed: - chunk = bytes(accumulated[:chunk_bytes_needed]) - accumulated = accumulated[chunk_bytes_needed:] - chunk_queue.put(chunk) - except Exception as e: - logger.error(f"Reader thread exception: {e}") - finally: - min_partial_bytes = int(0.5 * AUDIO_RATE * 2) - if len(accumulated) >= min_partial_bytes: - logger.info( - f"Flushing final partial chunk ({len(accumulated)} bytes, {len(accumulated)/(AUDIO_RATE*2):.2f}s)" - ) - chunk_queue.put(bytes(accumulated)) - else: - logger.info(f"Discarding partial buffer under 0.5s ({len(accumulated)} bytes)") - chunk_queue.put(None) + if CHUNK_MODE == "fixed": + chunk_bytes_needed = int(CHUNK_SECONDS * AUDIO_RATE * 2) + accumulated = bytearray() + try: + while True: + data = proc.stdout.read(4096) + if not data: + break + accumulated.extend(data) + while len(accumulated) >= chunk_bytes_needed: + chunk = bytes(accumulated[:chunk_bytes_needed]) + accumulated = accumulated[chunk_bytes_needed:] + chunk_queue.put(chunk) + except Exception as e: + logger.error(f"Reader thread exception: {e}") + finally: + min_partial_bytes = int(0.5 * AUDIO_RATE * 2) + if len(accumulated) >= min_partial_bytes: + logger.info( + f"Flushing final partial chunk ({len(accumulated)} bytes, {len(accumulated)/(AUDIO_RATE*2):.2f}s)" + ) + chunk_queue.put(bytes(accumulated)) + else: + logger.info(f"Discarding partial buffer under 0.5s ({len(accumulated)} bytes)") + chunk_queue.put(None) + else: + chunker = AdaptiveChunker( + min_chunk_seconds=MIN_CHUNK_SECONDS, + max_chunk_seconds=MAX_CHUNK_SECONDS, + silence_flush_ms=SILENCE_FLUSH_MS, + ) + try: + while True: + data = proc.stdout.read(4096) + if not data: + break + for chunk, reason in chunker.feed(data): + logger.info(f"VAD flush: {len(chunk)/(AUDIO_RATE*2):.2f}s chunk ({reason})") + chunk_queue.put(chunk) + except Exception as e: + logger.error(f"Reader thread exception: {e}") + finally: + min_partial_bytes = int(0.5 * AUDIO_RATE * 2) + buf_len = len(chunker.buffer) + final_chunk = chunker.drain(min_bytes=min_partial_bytes) + if final_chunk is not None: + logger.info( + f"Flushing final partial chunk ({len(final_chunk)} bytes, {len(final_chunk)/(AUDIO_RATE*2):.2f}s)" + ) + chunk_queue.put(final_chunk) + else: + logger.info(f"Discarding partial buffer under 0.5s ({buf_len} bytes)") + chunk_queue.put(None) def _transcribe_worker(self, chunk_queue: queue.Queue): """Processes chunks from queue, transcribes with Whisper, and types at cursor.""" @@ -464,8 +602,12 @@ def start_hotkeys(self): def run(self): """Runs the tray icon event loop in the main thread.""" self.start_hotkeys() + if CHUNK_MODE == "fixed": + chunk_info = f"CHUNK={CHUNK_SECONDS}s (fixed)" + else: + chunk_info = f"MODE=vad (min={MIN_CHUNK_SECONDS}s, max={MAX_CHUNK_SECONDS}s, silence={SILENCE_FLUSH_MS}ms)" logger.info( - f"Tray application running (MODEL={MODEL_NAME}, DEVICE={DEVICE}, COMPUTE={COMPUTE_TYPE}, CHUNK={CHUNK_SECONDS}s, HOTKEY={self.hotkey_normalized})." + f"Tray application running ({chunk_info}, MODEL={MODEL_NAME}, DEVICE={DEVICE}, COMPUTE={COMPUTE_TYPE}, HOTKEY={self.hotkey_normalized})." ) self.icon.run() @@ -504,6 +646,104 @@ def quit(self): pass +def run_test_chunks_mode(file_path: Optional[str] = None): + """Dev simulation mode: feed audio through AdaptiveChunker in 4096-byte steps and transcribe.""" + if not file_path: + file_path = DEFAULT_TEST_WAV + + if not os.path.exists(file_path): + logger.error(f"Test audio file not found: {file_path}") + sys.exit(1) + + logger.info(f"Decoding {file_path} via ffmpeg...") + cmd = ["ffmpeg", "-i", file_path, "-f", "f32le", "-ar", f"{AUDIO_RATE}", "-ac", "1", "-"] + try: + proc = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, check=True) + except subprocess.CalledProcessError as e: + logger.error(f"ffmpeg decoding failed: {e}") + sys.exit(1) + + audio = np.frombuffer(proc.stdout, dtype=np.float32) + duration = len(audio) / float(AUDIO_RATE) + logger.info(f"Decoded {len(audio)} samples ({duration:.2f}s of audio).") + + if np.abs(audio).max() <= 1.5: + raw_s16 = (np.clip(audio, -1.0, 1.0) * 32767.0).astype(np.int16).tobytes() + else: + raw_s16 = audio.astype(np.int16).tobytes() + + logger.info(f"Loading Whisper model '{MODEL_NAME}' on {DEVICE} ({COMPUTE_TYPE})...") + from faster_whisper import WhisperModel + + t_load = time.time() + model = WhisperModel(MODEL_NAME, device=DEVICE, compute_type=COMPUTE_TYPE) + load_time = time.time() - t_load + logger.info(f"Model loaded in {load_time:.2f}s.") + + chunker = AdaptiveChunker( + min_chunk_seconds=MIN_CHUNK_SECONDS, + max_chunk_seconds=MAX_CHUNK_SECONDS, + silence_flush_ms=SILENCE_FLUSH_MS, + ) + + transcribe_kwargs = {"beam_size": 1, "vad_filter": True} + if MODEL_NAME.endswith(".en"): + transcribe_kwargs["language"] = "en" + + flushed_records = [] + step = 4096 + + for i in range(0, len(raw_s16), step): + step_data = raw_s16[i : i + step] + file_time = min(len(raw_s16), i + step) / (AUDIO_RATE * 2) + flushed = chunker.feed(step_data) + for chunk, reason in flushed: + chunk_len = len(chunk) / (AUDIO_RATE * 2) + chunk_start_time = file_time - chunk_len + if reason == "phrase-end" and chunker.last_speech_end_samples is not None: + chunk_audio_end_time = chunk_start_time + (chunker.last_speech_end_samples / AUDIO_RATE) + else: + chunk_audio_end_time = file_time + flushed_records.append((file_time, chunk_len, reason, chunk, chunk_audio_end_time)) + + final_chunk = chunker.drain(min_bytes=int(MIN_CHUNK_SECONDS * AUDIO_RATE * 2)) + if final_chunk is not None: + file_time = len(raw_s16) / (AUDIO_RATE * 2) + chunk_len = len(final_chunk) / (AUDIO_RATE * 2) + flushed_records.append((file_time, chunk_len, "drain", final_chunk, file_time)) + + print("\n--- Chunks ---") + total_chars = 0 + latencies = [] + phrase_end_count = 0 + max_count = 0 + + for file_time, chunk_len, reason, chunk, audio_end_time in flushed_records: + if reason == "phrase-end": + phrase_end_count += 1 + latencies.append(file_time - audio_end_time) + elif reason == "max": + max_count += 1 + + a = np.frombuffer(chunk, dtype=np.int16).astype(np.float32) / 32768.0 + segments, _ = model.transcribe(a, **transcribe_kwargs) + text = " ".join(s.text.strip() for s in segments).strip() + total_chars += len(text) + print(f"[flush at file-time {file_time:5.1f}s | chunk {chunk_len:4.1f}s | {reason}] {text}") + + total_chunks = len(flushed_records) + avg_latency = (sum(latencies) / len(latencies)) if latencies else 0.0 + max_latency = max(latencies) if latencies else 0.0 + + print("--------------") + print( + f"Total chunks: {total_chunks} ({phrase_end_count} phrase-end, {max_count} max) | " + f"Total chars: {total_chars} | " + f"Latency (flush - audio_end): avg {avg_latency:.2f}s, max {max_latency:.2f}s\n" + ) + sys.exit(0) + + def run_test_mode(file_path: Optional[str] = None): """Test mode: decode audio via ffmpeg, transcribe with faster-whisper, print timing and transcript.""" if not file_path: @@ -552,6 +792,13 @@ def run_test_mode(file_path: Optional[str] = None): def main(): + if "--test-chunks" in sys.argv: + idx = sys.argv.index("--test-chunks") + test_file = None + if idx + 1 < len(sys.argv) and not sys.argv[idx + 1].startswith("-"): + test_file = sys.argv[idx + 1] + run_test_chunks_mode(test_file) + if "--test" in sys.argv: idx = sys.argv.index("--test") test_file = None diff --git a/work-logs/WORK LOG - linux-dictation-client.md b/work-logs/WORK LOG - linux-dictation-client.md index e270d827..5cff2bf6 100644 --- a/work-logs/WORK LOG - linux-dictation-client.md +++ b/work-logs/WORK LOG - linux-dictation-client.md @@ -48,3 +48,38 @@ Excluded runtime/build artifacts: `last-take.wav`, `.venv/`, `__pycache__/`, `.g ## FOLLOW-UPS / NOTES - Optional follow-up: user may migrate the live systemd install to the new path (`~/.config/systemd/user/`) after merge. - Do not commit changes; user will commit. + +--- + +## UPDATE (2026-10-14): Phrase-aware dictation chunking + +**Problem:** User reported dictation text arriving in 3-second bursts and always +feeling like it was falling behind. + +**Investigation:** Inference was not the bottleneck (faster-whisper base.en CPU +int8 = ~25-30x realtime on the dev box). The latency came from (1) fixed 3s +chunks — text can only appear once a chunk fills, even mid-sentence and even +after the speaker stops — and (2) `xdotool --delay 10ms/char` adding 0.5-0.8s +per burst. + +**Change (linux/live_dictate.py):** +- New default chunking mode `vad`: the streaming audio buffer is probed with + the bundled Silero VAD (~3-6ms per probe via `faster_whisper.vad.get_speech_timestamps`) + and a chunk is flushed the moment the speaker pauses (>= 300ms trailing + silence) or at the 4s cap — text now flows phrase by phrase. +- `LT_DICTATE_CHUNK_MODE=fixed` restores the legacy fixed-chunk behavior + (`LT_DICTATE_CHUNK_SECONDS`). New env vars: `LT_DICTATE_CHUNK_MODE`, + `LT_DICTATE_MAX_CHUNK_SECONDS` (4), `LT_DICTATE_MIN_CHUNK_SECONDS` (0.5), + `LT_DICTATE_SILENCE_FLUSH_MS` (300). `LT_DICTATE_TYPE_DELAY_MS` default 10 -> 0. +- Model warmup (0.5s dummy transcribe) right after model load so the first + chunk is responsive. +- New dev mode `--test-chunks [FILE]`: simulates the adaptive chunker over a + WAV file and prints per-chunk flush time/reason/text plus latency stats. +- README updated (env table + how chunking works + --test-chunks). + +**Verification:** `--test-chunks last-take.wav` (89s real speech): 27 chunks +(15 phrase-end / 12 max), avg flush latency 0.36s, max 0.41s (was up to ~3s+ +per burst). Live daemon lifecycle (F8 start/stop under DISPLAY=:1) clean, +VAD phrase-end flushes observed in real time, no tracebacks. + +Files changed: `linux/live_dictate.py`, `linux/README.md`.