streamguard

A real-time streaming guardrail engine for LLM outputs written with Rust.

Instead of waiting for a full LLM response before running safety checks, streamguard evaluates output token-by-token as it streams catching policy violations early and terminating the stream before harmful content reaches the user.

Architecture

User prompt
    |
    v
Anthropic SSE Stream
    |
    v
PolicyEngine  (runs policies concurrently per chunk)
    |-- JailbreakDetector
    |-- PiiDetector
    |-- CustomPolicy (pluggable via Policy trait)
    |
    v
Decision: Pass / Flag / Block
    |
    v
Consumer (clean stream or early termination)

Usage

Start the server:

export ANTHROPIC_API_KEY=sk-ant-...
cargo run

Send a prompt:

curl -X POST http://localhost:3000/check \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Make up a fake user profile with email and phone number"}'

Response:

{
  "blocked": false,
  "block_reason": null,
  "violations": [
    {
      "policy": "pii_detector",
      "message": "Email address detected",
      "severity": "high",
      "matched_text": "james.mitchell@gmail.com",
      "chunk_index": 1
    }
  ],
  "passed_text": "..."
}

Custom Policies

Any custom rule can be plugged in by implementing the Policy trait:

use streamguard::policy::Policy;
use streamguard::types::{Decision, StreamChunk, PolicyViolation, Severity};
use async_trait::async_trait;

struct ProfanityFilter;

#[async_trait]
impl Policy for ProfanityFilter {
    fn name(&self) -> &str { "profanity_filter" }

    async fn evaluate(&self, chunk: &StreamChunk, window: &str) -> Decision {
        if window.to_lowercase().contains("badword") {
            return Decision::Block(PolicyViolation {
                policy_name: self.name().to_string(),
                message: "Profanity detected".to_string(),
                severity: Severity::Medium,
                matched_text: Some("badword".to_string()),
                chunk_index: chunk.index,
            });
        }
        Decision::Pass
    }
}

let engine = GuardrailEngine::new()
    .add_policy(JailbreakDetector)
    .add_policy(PiiDetector::default())
    .add_policy(ProfanityFilter);