POC: Local Chat Panel with ollama model.

[rushtong]

Addresses

https://broadworkbench.atlassian.net/browse/DT-3266

Summary

Proof of concept implementation of a local Node/Ollama AI chat set up.

Architecture

Additionally, session handling will need to span the different pods in the DUOS app k8s namespace. All pods will need to be able to communicate with a Redis StatefulSet as the implementation solution:

Data Flow

Browser (React chat UI)
  │  POST /api/chat  { message }
  │  Authorization: Bearer <oidcToken>
  ▼
Fastify server  (same Docker container, port 8080)
  ├─ stores oidcToken → sessionStore[sessionId]  (TTL = token expiry)
  ├─ calls LLM backend
  │    local:      OLLAMA_URL/api/chat  (OpenAI-compatible)
  │    production: Vertex AI Gemini API  (via @google-cloud/vertexai SDK)
  │  streams SSE tokens back to browser while agentic loop runs
  │
  │  (tool calls from LLM loop back into same server)
  ├─ POST /mcp/:sessionId  (JSON-RPC 2.0)
  │    looks up oidcToken from sessionStore
  │    calls Consent API with Bearer <oidcToken>
  ▼
Consent API → data → MCP handler → agentic loop → SSE → browser

Running Locally

docker build . -t duos
docker compose --profile local-ai up -d
docker exec duos-ollama ollama pull llama3.2:3b

Tools implemented are in consentClient.js:

• list_datasets
• get_dataset
• list_dar_collections

Chat Panel In Action

---

Have you read Terra's Contributing Guide lately? If not, do that first.

• Label PR with a Jira ticket number and include a link to the ticket
• Label PR with a security risk modifier [no, low, medium, high]
• PR describes scope of changes
• Get a minimum of one thumbs worth of review, preferably two if enough team members are available
• Get PO sign-off for all non-trivial UI or workflow changes
• Verify all tests go green
• Test this change deployed correctly and works on dev environment after deployment

[sonarqubecloud]

Quality Gate passed

Issues
12 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud