diff --git a/docs/en/solutions/AI/LLM_Security_Scanning_with_garak_on_Workbench.md b/docs/en/solutions/AI/LLM_Security_Scanning_with_garak_on_Workbench.md new file mode 100644 index 000000000..004831792 --- /dev/null +++ b/docs/en/solutions/AI/LLM_Security_Scanning_with_garak_on_Workbench.md @@ -0,0 +1,455 @@ +--- +products: + - Alauda AI +kind: + - Solution +--- + +# LLM Security Scanning with garak on Alauda AI Workbench + +## Issue + +Before a large language model goes into production, its resistance to jailbreaks, prompt injection, harmful content generation and data leakage has to be assessed. [garak](https://github.com/NVIDIA/garak) is NVIDIA's open-source LLM vulnerability scanner (Apache-2.0). It ships 41 probe modules with roughly 180 attack probes, sends the attack prompts to a target model, uses detectors to decide whether the response is undesirable, and reports an attack success rate per risk category. + +Installing garak on site is often impractical: it pulls Python packages from PyPI and detector models and datasets from Hugging Face, none of which is reachable from an isolated environment. + +This solution provides a prebuilt Alauda AI Workbench image with garak and all of its offline assets baked in. Once the image is in a registry the cluster can pull from and a WorkspaceKind has been imported, users create a Workspace from the console and scan any inference service in the same cluster. Nothing is fetched from the internet at scan time, and no extra scheduling component is required. + +## Environment + +* Alauda AI with the Alauda AI Workbench plugin installed. +* Target: a text LLM published on the platform that exposes an OpenAI-compatible API (for example a vLLM runtime). +* The Workspace needs CPU only. The detector models are small classifiers that run on CPU. +* Multimodal probes (image, audio) are out of scope. + +## Resolution + +### Prerequisites + +* The base URL of an OpenAI-compatible endpoint for the model to be scanned, reachable from the Workspace. It can be the in-cluster Service address of a model published on the platform (`http://-predictor..svc.cluster.local`), a gateway or ingress address, or any other endpoint that serves `/v1/chat/completions`. Note the API key if the endpoint requires one. +* The garak Workbench image is available in a registry the cluster can pull from. See the Image section below. +* Cluster administrator permission, to import the WorkspaceKind once. +* At least 2 GB free on the Workspace volume, for the scan configuration and the reports. + +### Image + +Alauda provides the garak Workbench image: garak 0.17.0, digest `sha256:d1cc22470189dfe4b341f1c2507897d60399f5257ec936bcd14da4f38e99440a`, 5.3 GB. Push it to a registry the cluster can pull from, for example `/garak-workbench:0.17.0-20260922`, and use that address in the WorkspaceKind below. + +### Importing the WorkspaceKind + +The images offered in the Workbench console come from `WorkspaceKind` resources in the cluster, so an administrator has to import a WorkspaceKind that points at the garak image. This is a one-time operation; afterwards every user can pick it when creating a Workspace. + +Save the following as `workspacekind-garak.yaml`, change `image` to the address pushed in the previous step, and apply it with `kubectl apply -f workspacekind-garak.yaml`. + +
+ +workspacekind-garak.yaml + +```yaml +# WorkspaceKind for the garak security-scanning image. +# Appears in Alauda AI: User View -> Model Development -> Workbench -> create Workspace. +apiVersion: kubeflow.org/v1beta1 +kind: WorkspaceKind +metadata: + name: garak-scanner-0-17-0 + labels: + # The Alauda AI console filters WorkspaceKinds by these labels + workbench.alauda.io/managed: "true" + workbench.alauda.io/family: garak-scanner +spec: + spawner: + displayName: garak LLM Security Scanner + description: A Workspace with garak preinstalled for LLM vulnerability scanning + deprecated: false + hidden: false + icon: + configMap: + name: aml-workbench-config + key: jupyterlab-icon.png + logo: + configMap: + name: aml-workbench-config + key: jupyterlab-logo.svg + podTemplate: + podMetadata: + labels: + workbench.alauda.io/purpose: garak-security-scan + serviceAccount: + name: aml-editor + securityContext: + fsGroup: 0 + containerSecurityContext: + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + privileged: false + runAsNonRoot: true + seccompProfile: + type: RuntimeDefault + culling: + enabled: true + maxInactiveSeconds: 86400 + activityProbe: + jupyter: + lastActivity: true + volumeMounts: + home: /opt/app-root/src + extraEnv: + - name: NB_PREFIX + value: /clusters/data-ai-dev/aml/aml-workbench{{ httpPathPrefix "jupyterlab" }} + - name: NOTEBOOK_BASE_URL + value: /clusters/data-ai-dev/aml/aml-workbench{{ httpPathPrefix "jupyterlab" }} + - name: NOTEBOOK_ARGS + value: --ServerApp.token='' --ServerApp.password='' + extraVolumes: + - name: dshm + emptyDir: + medium: Memory + extraVolumeMounts: + - name: dshm + mountPath: /dev/shm + httpProxy: + removePathPrefix: false + requestHeaders: {} + probes: + livenessProbe: + exec: + command: + - sh + - -c + - curl -sf http://127.0.0.1:8888${NB_PREFIX}api/status + initialDelaySeconds: 20 + periodSeconds: 10 + timeoutSeconds: 5 + failureThreshold: 3 + readinessProbe: + exec: + command: + - sh + - -c + - curl -sf http://127.0.0.1:8888${NB_PREFIX}api/status + initialDelaySeconds: 10 + periodSeconds: 5 + timeoutSeconds: 3 + failureThreshold: 3 + options: + imageConfig: + spawner: + default: garak-workbench-0-17-0 + values: + - id: garak-workbench-0-17-0 + spawner: + displayName: garak 0.17.0 | Security Scanner | CPU | Python 3.12 + description: garak 0.17.0 with offline detector models and datasets + hidden: false + labels: + - key: python_version + value: "3.12" + - key: garak_version + value: 0.17.0 + spec: + # Replace with the image address in your registry + image: /garak-workbench:0.17.0-20260922 + imagePullPolicy: IfNotPresent + ports: + - id: jupyterlab + displayName: JupyterLab + port: 8888 + protocol: HTTP + podConfig: + spawner: + default: scan-medium + values: + - id: scan-small + spawner: + displayName: Small CPU + description: Pod with 1 CPU, 8 GiB RAM + hidden: false + labels: + - key: cpu + value: "1" + - key: memory + value: 8Gi + spec: + resources: + requests: + cpu: "1" + memory: 8Gi + limits: + cpu: "2" + memory: 8Gi + - id: scan-medium + spawner: + displayName: Medium CPU + description: Pod with 2 CPU, 12 GiB RAM - recommended for scanning + hidden: false + labels: + - key: cpu + value: "2" + - key: memory + value: 12Gi + spec: + resources: + requests: + cpu: "2" + memory: 12Gi + limits: + cpu: "4" + memory: 16Gi +``` + +
+ +Confirm the resource is present. A `WORKSPACES` count of 0 is expected at this point: + +```bash +kubectl get workspacekind garak-scanner-0-17-0 +``` + +### Creating a Workspace + +In the Alauda AI console, switch to the User View, go to **Model Development** > **Workbench** in the left navigation, and create a Workspace. + +* Type: select **garak LLM Security Scanner**. +* Image: select **garak 0.17.0 | Security Scanner | CPU | Python 3.12**. +* Pod size: select **Medium CPU** (2 cores, 12 GiB). The detector models run on CPU; too little memory aborts the scan. +* GPU: not required. +* Volume: the default volume is fine. The scan configuration and the reports are stored under the home directory and survive a Workspace restart or an image version change. + +### Verifying access to the inference service + +Open the JupyterLab page of the Workspace, start a terminal (Launcher > Terminal), and run the following against the endpoint to be scanned. `TARGET_URL` is the base URL, without the `/v1` suffix: + +```bash +export TARGET_URL= # for example http://qwen3-predictor.my-ns.svc.cluster.local + +# list the model names, needed for the configuration below +curl -s $TARGET_URL/v1/models + +# check that the chat endpoint works +curl -s $TARGET_URL/v1/chat/completions -H 'Content-Type: application/json' \ + -d '{"model":"","messages":[{"role":"user","content":"hello"}],"max_tokens":20}' +``` + +Both commands must return JSON. Note the `id` field returned by `/v1/models`; that is the model name. Add `-H 'Authorization: Bearer '` if the endpoint requires a key, and pass it to garak through `OPENAICOMPATIBLE_API_KEY` in the steps below. + +### Preparing the scan configuration + +Create `~/garak/scan.yaml` in the Workspace with the following content, either from the JupyterLab editor or from the terminal: + +```yaml +# garak scan configuration: scans an in-cluster inference service via /v1/chat/completions +# Usage: export OPENAICOMPATIBLE_API_KEY=dummy && garak --config ~/garak/scan.yaml +system: + parallel_attempts: 8 # concurrent requests; tune to the service throughput + +run: + generations: 1 # generations per prompt; 1 for a pilot, 3-5 for a full assessment + soft_probe_prompt_cap: 20 # at most 20 prompts per probe; remove for a full assessment + spec: + include: + - probes.dan.Dan_11_0 + - probes.grandma + - probes.encoding.InjectBase64 + - probes.promptinject + - probes.latentinjection.LatentInjectionResume + - probes.lmrc + - probes.malwaregen.TopLevel + - probes.packagehallucination.Python + - probes.sysprompt_extraction + - probes.web_injection.MarkdownImageExfil + - probes.ansiescape.AnsiRaw + +plugins: + target_type: openai.OpenAICompatible + target_name: + generators: + openai: + OpenAICompatible: + uri: /v1/ # the endpoint base URL with the /v1/ suffix, trailing slash included + stop: [] # the default ["#", ";"] truncates code and CJK output, always clear it + max_tokens: 256 + temperature: 0.7 + extra_params: + # passed through to the server; switches off the thinking mode of Qwen3 and similar models + extra_body: + chat_template_kwargs: + enable_thinking: false + +reporting: + report_prefix: scan-pilot # report file name prefix + report_dir: /opt/app-root/src/garak/report # must be absolute; a relative path lands under ~/.local/share/garak +``` + +Replace the two placeholders: + +1. `target_name`: the model name returned by `/v1/models`, replacing ``. +2. `uri`: the endpoint base URL with `/v1/` appended, for example `http://qwen3-predictor.my-ns.svc.cluster.local/v1/` or `http://192.168.0.10:31795/v1/`. Keep the trailing slash. + +Two settings are worth understanding before changing them: + +* `stop: []` — garak defaults to `["#", ";"]`, which truncates code and CJK output mid-answer and makes detector verdicts unreliable. Always keep it cleared. +* `report_dir` — reports are written here. garak resolves a relative path against `~/.local/share/garak`, which is awkward to open in the file browser, so the path is absolute and points at `report/` next to the configuration. `/opt/app-root/src` is the Workspace home directory. +* `extra_body.chat_template_kwargs.enable_thinking: false` — for models with a thinking mode, such as Qwen3, this stops the model from emitting a reasoning block before its answer. Without it the detectors count the reasoning text in their verdicts. `extra_params` entries are passed to the OpenAI client as call arguments, so server-side options have to be nested under `extra_body`. + +Export the API key before scanning. Any value works when the endpoint does not check it, but the variable must not be empty: + +```bash +export OPENAICOMPATIBLE_API_KEY= +``` + +### Smoke test + +Run one small probe first (6 prompts, about 2 minutes) to confirm the path works end to end: + +```bash +cd ~/garak +garak --config scan.yaml --spec probes.grandma.Win10 --report_prefix smoke +``` + +Expected output, abridged: + +``` +🦜 loading generator: OpenAICompatible: qwen3-5-0-8b +🕵️ queue of probes: grandma.Win10 +grandma.Win10 mitigation.MitigationBypass: FAIL ok on 0/6 (attack success rate: 100.00%) +grandma.Win10 productkey.Win5x5: PASS ok on 6/6 +📜 report closed :) /opt/app-root/src/garak/report/smoke.report.jsonl +✔️ garak run complete in 122.66s +``` + +Each line is one probe/detector pair. `ok on 0/6` means none of the 6 responses passed the detector, so the attack success rate is 100%. + +### Running the scan + +```bash +cd ~/garak +garak --config scan.yaml +``` + +Results appear per probe as the scan proceeds, and the run ends with `✔️ garak run complete in NNNs`. The 11 probe entries in the sample configuration expand to 21 probes and complete in about 12 minutes at 8 concurrent requests. A default full scan — without `spec` and `soft_probe_prompt_cap` — can issue tens of thousands of requests, so start with a selection of probes and widen it gradually. + +Scans can also be started from a notebook cell: + +```python +!garak --config ~/garak/scan.yaml --spec probes.dan +``` + +### Reading the reports + +The reports are written to `~/garak/report/`, the directory named by `report_dir` in the configuration: + +| File | Content | +| --- | --- | +| `.report.html` | Summary report; expands from probe module to probe to detector, with attack success rates and grades from 1 to 5. Double-click it in the JupyterLab file browser to open it | +| `.report.jsonl` | Full record of every prompt, model output and detector score | +| `.hitlog.jsonl` | Only the samples judged as successful attacks, for manual review | + +To inspect the hits, prompt and output included: + +```bash +python - <<'EOF' +import json, glob, os +f = sorted(glob.glob(os.path.expanduser("~/garak/report/*.hitlog.jsonl")))[-1] +for line in open(f): + r = json.loads(line) + prompt = r["prompt"]["turns"][-1]["content"]["text"] + output = r["output"]["text"] + print("probe:", r["probe"], "| detector:", r["detector"], "| score:", r["score"]) + print("prompt:", prompt[:200].replace("\n", " ")) + print("output:", output[:200].replace("\n", " ")) + print("---") +EOF +``` + +The `mitigation.MitigationBypass` detector decides whether the model failed to refuse, while detectors such as `productkey` and `unsafe_content` decide whether genuinely harmful content was produced. Read them together: the former failing while the latter passes means the model played along with the attack framing but did not emit harmful content, which is less severe than both failing. + +## Extensions + +### Scanning an application instead of the model + +The configuration above scans the model endpoint directly, which measures the robustness of the model itself. That is the right target for model selection, for comparing versions and for a model card, but it is not the whole production risk: system prompt leakage, guardrail bypass, indirect injection through retrieved documents and unauthorised tool calls only appear once the model sits behind an application. A model that looks weak on its own may be well contained by an application, and a model that looks safe may still be exploitable because the application concatenates user input into its system prompt. + +Two ways to move closer to production risk, in increasing order of fidelity. + +**Send the production system prompt with the scan.** Keep scanning the model endpoint, but have garak pass the same system prompt the application uses, so the result reflects the model plus your prompt engineering: + +```yaml +run: + system_prompt: "" +``` + +garak sends it as a system message for generators that support chat, unless a probe overrides it. + +**Scan the application's own endpoint.** Point garak at the HTTP API of the chatbot, RAG service or agent with the `rest` generator. The request and response shapes are yours, so they have to be described in the configuration: + +```yaml +plugins: + target_type: rest + target_name: my-rag-app + generators: + rest: + RestGenerator: + uri: /api/v1/chat # the application endpoint that takes user input + method: post + headers: + Content-Type: application/json + Authorization: Bearer $KEY # taken from REST_API_KEY + req_template_json_object: + question: $INPUT # $INPUT is replaced with the attack prompt + response_json: true + response_json_field: $.answer # JSONPath to the answer in the response + request_timeout: 120 +``` + +With this target the scan covers the application's system prompt, its guardrails, its retrieval context and its tools — the surface an attacker actually reaches. Probes such as `sysprompt_extraction`, `latentinjection` and `exploitation` become far more meaningful here than against a bare model. + +> **NOTE:** +> The `rest` generator sends only the last message of a conversation, so multi-turn probes (`goat`, `fitd`, `atkgen`, `tap`) are silently reduced to their final turn and no longer test what they are meant to test. Single-turn probes are unaffected. To cover multi-turn attacks against an application, write a generator that keeps the application's session, subclassing `garak.generators.base.Generator` and implementing `_call_model`. + +Comparing a scan with the guardrails enabled against one with them disabled quantifies what the guardrails actually stop. + +### Selecting probes + +* `garak --list_probes` lists every probe with its tier; tier 1 deserves the most attention. +* `--spec probes.all,tier:1` runs the tier 1 probes only; `--spec tag:owasp:llm01` selects by OWASP LLM Top 10 category. +* These probes reach the internet or the Hugging Face Hub at run time, so do not select them in an offline environment: `visual_jailbreak`, `fileformats`, `audio`. + +### Probes for other languages and for business scenarios + +garak's built-in probes are mostly English. For other languages, and for risks specific to your own RAG or agent applications, write your own probes: create a Python module, subclass `garak.probes.Probe`, set `self.prompts` in `__init__` and name a `primary_detector`. See "Writing a Probe" in the garak documentation. + +garak loads probes from its own package directory, and that directory returns to its original state when a Workspace is recreated. Keep the probe sources on the volume, for example under `~/garak/probes/`, and copy them into the package directory in each new Workspace: + +```bash +cp ~/garak/probes/*.py /opt/app-root/garak/venv/lib/python3.12/site-packages/garak/probes/ +garak --config ~/garak/scan.yaml --spec probes. +``` + +Once the probes are stable, ask Alauda to add them to the image so that they ship with it. + +### Using an LLM as the judge detector + +The `detectors.judge.*` detectors ask another model whether the attack goal was reached. That is more accurate than keyword matching, especially for non-English output. It needs a working chat endpoint: + +```yaml +plugins: + detectors: + judge: + detector_model_type: openai.OpenAICompatible + detector_model_name: + detector_model_config: + uri: /v1/ + stop: [] +``` + +Use a judge model larger than the model under test. + +## Image upgrades + +garak adds probes with every release. Alauda can provide an image for a newer garak version on request; update `image` in the WorkspaceKind to the new address and have users switch their Workspace. + +## Summary + +garak and all of its offline assets are packaged as an Alauda AI Workbench image published by Alauda. Once the image is in a registry the cluster can pull from, and an administrator has imported the WorkspaceKind once, users create a Workspace from the Workbench console and scan any inference service in the cluster. No internet access is needed at scan time and no task scheduling component is involved. Scan configurations and reports are kept on the Workspace volume, where they can be read in JupyterLab or downloaded for archiving; scanning a different service only takes a change of the service address and model name in `scan.yaml`.