Guardian Agent

Voice-first personal safety companion disguised as a normal phone call.

HackYeah.2026.Guardian.Agent.mp4

What is Guardian Agent?

Guardian Agent is a voice-first personal safety application. To an outside observer, it looks and sounds like you are having a normal phone conversation. In reality, the AI agent is actively monitoring your speech for specific distress phrases. When triggered, it initiates a silent, hands-free emergency activation, capturing your GPS location and sending an immediate emergency alert to a live dispatcher dashboard.

The Problem

In a genuine emergency or threatening situation:

• You may not be able to safely take out and unlock your phone.
• Reaching for a panic button or dialing emergency services can escalate the danger if noticed by an attacker.
• You need a discrete, passive way to call for help without breaking your cover.

Disguising your plea for help as a casual phone call is a natural human defense mechanism.

Our Solution

1. Voice input: You speak naturally to an AI voice assistant (disguised as a contact, e.g., "Mom").
2. Distress phrase detection: The app continuously listens for pre-configured secret phrases (e.g., "Did you feed the cat?").
3. Speech normalization: Polish speech is handled accurately.
4. Rolling transcript & Severity: The conversation is transcribed and analyzed.
5. GPS: High-accuracy location is captured.
6. Alert dispatch: A silent HTTP POST is sent to the backend.
7. Dispatcher dashboard: The incident appears in real-time on a web-based command center.
8. Voice feedback: The AI naturally acknowledges the situation without breaking character.

Key Features

• Fake phone call interface with real-time AI voice conversation (ElevenLabs + Gemini).
• Silent distress phrase detection running in the background.
• Live GPS tracking attached to alerts.
• Real-time Dispatcher Dashboard (/dispatcher) showing incidents on a map.
• Mock SMS notifier for demo purposes.

Why it is different

• Voice-first safety interaction: No fumbling with buttons; your voice is the trigger.
• Hands-free activation: Works even if your phone is in your pocket (with a headset).
• Polish speech normalization: Optimized for natural Polish emergency phrases.
• Configurable distress phrases: Customize what triggers the alarm.
• Automatic location: GPS coordinates are sent instantly.
• Dispatcher workflow: Professional web interface for monitoring incidents.
• Immediate voice feedback: The AI keeps you calm and confirms help is on the way (discreetly).
• Cooldown: Protection against repeated accidental triggers.

Instructions for the Jury

1. Clone the repository.
2. Configure the local .env file using .env.example.
3. Run .\start-pitch.ps1 (Windows) or make dev (Mac/Linux).
4. Scan the generated QR code with a mobile phone (or open http://localhost:5173).
5. Open the dispatcher URL on your laptop (/dispatcher).
6. Test the emergency voice flow (e.g., by saying the distress phrase into the phone).
7. Observe the incident appearing on the dispatcher dashboard.

How to Run the Application

1. Prerequisites

• Python 3.11+
• Node.js 20+
• make (optional, for convenience shortcuts)
• Docker & Docker Compose (optional)

2. Environment Configuration

Guardian Agent requires a single .env file located in the root directory. You can use the provided .env.example as a template:

cp .env.example .env

Required variables:

• ELEVENLABS_API_KEY
• ELEVENLABS_AGENT_ID
• PORT (default: 4000)
• BACKEND_URL (default: http://localhost:8000)
• CORS_ORIGIN (default: http://localhost:5173)
• CORS_ORIGINS (default: http://localhost:4000)
• APP_ENV (default: development)

Note: The .env file is local and must never be committed to the repository. ElevenLabs credentials must be configured locally for the voice features to function properly. Without them, the app works but falls back to an offline browser speech synthesizer.

3. Starting the Services

Option A: Quick Start (Mac / Linux with Make)

make setup   # Sets up .env, Python venv, and npm dependencies (first time only)
make dev     # Starts backend, BFF, and frontend concurrently in one terminal

Option B: Windows Automated Pitch Environment

Ideal for jury presentation and live mobile testing:

.\start-pitch.ps1

This script starts all services, sets up an ephemeral Cloudflare tunnel for HTTPS mobile microphone access, and outputs a QR code to scan with a phone.

To stop all services:

.\stop-pitch.ps1

Option C: Docker Compose (Cross-Platform)

make up
# or: docker compose up --build

Option D: Manual Startup (Separate Terminals)

# Terminal 1 - Backend (FastAPI)
cd backend && source .venv/bin/activate && uvicorn app.main:app --reload --port 8000

# Terminal 2 - BFF (Express / TypeScript)
cd bff && npm run dev

# Terminal 3 - Frontend (React / Vite)
cd frontend && npm run dev

4. Application URLs

Once running, you can access the components at:

• Phone UI (Caller interface): http://localhost:5173
• Dispatcher Dashboard: http://localhost:5173/dispatcher
• Mock SMS Phone: http://localhost:5173/sms (or port 3001 via make sms)
• Backend API Documentation: http://localhost:8000/docs
• BFF Health Check: http://localhost:4000/health

Architecture

User
 ↓
Frontend / Voice Assistant
 ↓
Safety Pipeline
 ↓
BFF API
 ↓
Backend
 ↓
Dispatcher Dashboard

Note: ElevenLabs is utilized in the Frontend/BFF layer for the Conversational AI.

Project Structure

• backend/ - Python API (FastAPI)
• bff/ - Backend-for-Frontend (Node.js, Express, TypeScript)
• frontend/ - React application (Vite, TypeScript)
• docs/ - Documentation & architecture

Development Stack

• Backend: Python 3.11+ (FastAPI)
• BFF: Node.js (Express, TypeScript)
• Frontend: React (Vite, TypeScript)
• Containerization: Docker & Docker Compose

Current Implementation Status

Implemented

• AI Voice Agent integration (ElevenLabs).
• Background keyword detection (Polish/English).
• Alert dispatching to backend.
• Real-time Dispatcher Dashboard with Incident Map.
• Automated pitch environment (Cloudflare tunnel + QR).

Pitch Goal

• Advanced reasoning for determining emergency severity dynamically.
• Integration with real emergency services (e112).
• Native iOS/Android apps (currently PWA/Web).