Repository navigation
Commit 28362d2
docs(ai): add garak LLM security scanning on Workbench (#848)
* docs(ai): add garak LLM security scanning on Workbench
Prebuilt Workbench image with garak 0.17.0 and its offline detector
models, datasets and NLTK corpora, so an isolated cluster can scan a
published inference service without reaching PyPI or Hugging Face.
Covers obtaining the image, importing the WorkspaceKind the console
needs to offer it, preparing the scan configuration, running a scan and
reading the reports, plus adapting the chat template for non-Qwen models
and using an LLM judge detector.
Verified against Qwen3.5-0.8B on vLLM: 21 probes in 775s, reports
written to the Workspace volume, detectors loading from the image with
the container offline.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(ai): address review on garak scanning article
- drop the verified-versions line from Environment
- any registry the cluster can pull from, not specifically the platform
Private Registry
- remove the Image contents section
- condense image retrieval to the address plus the push commands, and
keep section anchors only where another section links to them
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(ai): drop the offline tar note from the Image section
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(ai): drop the WorkspaceKind label and podMetadata notes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(ai): make the chat endpoint the recommended scan path
The shipped sample posts to /v1/completions because the chat endpoint of
the verification service did not respond, but chat is the endpoint most
services expose and it lets the server apply the model's own chat
template. Present both plugins sections, chat first, and keep the
template-adaptation section for the completions fallback only.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(ai): scan the chat endpoint only, add application-layer scanning
The chat endpoint is what services expose for instruct models and what
applications actually call, and the server applies the model's own chat
template. Scanning through /v1/completions needed a hand-built template
per model, and the rest generator drops all but the last turn, which
silently degrades the multi-turn probes; drop that path and the template
adaptation section with it.
Fix the thinking-mode setting: chat_template_kwargs has to be nested
under extra_body, otherwise the OpenAI client rejects it as an unexpected
keyword argument.
Add a section on what to scan: the model endpoint measures the model,
while the risk that matters lives in the application in front of it, so
cover passing the production system prompt and pointing the rest
generator at the application's own API.
Results table re-measured through the chat endpoint (21 probes, 701s),
plus a note on the report corruption that can break HTML generation on
long parallel runs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(ai): state the image address instead of registry CLI commands
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(ai): any reachable OpenAI-compatible endpoint, not just cluster DNS
The target does not have to be the in-cluster predictor Service: a
gateway, an ingress or any other endpoint serving /v1/chat/completions
works, so describe it as a base URL and mention the API key case.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(ai): write reports next to the config, drop the results section
- reports go to ~/garak/report via an absolute reporting.report_dir; a
relative path lands under ~/.local/share/garak, which is awkward to
open from the file browser
- remove the verification-results section and the report-corruption note
- refresh the smoke test output for the chat generator
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(ai): condense the image upgrade section
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(ai): run the scan in the foreground
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>1 parent d7d451c commit 28362d2
1 file changed
Lines changed: 455 additions & 0 deletions
0 commit comments