DocSense — Document Intelligence Platform

Upload PDFs, DOCX, and CSV files. Ask questions in plain English. Get cited answers instantly.

---

What This Does

Feature	Description
Multi-format Upload	PDF, DOCX, CSV — drag & drop
Semantic Q&A	Natural language questions answered from document content
Source Citations	Every answer references exact page, clause, or row
Auto Summary	TL;DR generated on upload
Multi-doc Cross-ref	Query across multiple docs simultaneously
Smart Alerts	Deadline & expiry extraction from contracts
CSV Anomaly Detection	Statistical outlier and missing value detection
Audit Trail	Full log of every query for compliance

---

Architecture

┌─────────────────────────────────┐     ┌──────────────────────────────────┐
│   Frontend (Next.js 14)         │────▶│   Backend (FastAPI + Python)     │
│   Vercel deploy                 │     │   Local / Render / Railway       │
│                                 │     │                                  │
│  /            Dashboard         │     │  POST /documents/upload          │
│  /upload      File manager      │     │  GET  /documents/                │
│  /query       Q&A interface     │     │  POST /query/                    │
│  /alerts      Smart alerts      │     │  GET  /analytics/audit           │
│  /anomalies   CSV analysis      │     │  POST /analytics/anomalies/{id}  │
│  /audit       Audit trail       │     │  GET  /analytics/alerts          │
└─────────────────────────────────┘     └──────────────────────────────────┘
                                                       │
                                        ┌──────────────┴───────────────┐
                                        │                              │
                                   FAISS Index                   OpenAI API
                                   (local disk)              (Embeddings + Chat)

---

Prerequisites

• Node.js 18+
• Python 3.10+
• OpenAI API Key — get one at https://platform.openai.com/api-keys

---

Setup — Step by Step

1. Clone / Download the project

git clone https://github.com/your-username/docsense.git
cd docsense

2. Backend Setup

cd backend

# Option A: Use the start script (recommended)
bash start.sh

# Option B: Manual
python3 -m venv venv
source venv/bin/activate          # Windows: venv\Scripts\activate
pip install -r requirements.txt

cp .env.example .env
# → Open .env and set OPENAI_API_KEY=sk-your-key-here

mkdir -p uploads vector_store
uvicorn main:app --host 0.0.0.0 --port 8000 --reload

Backend runs at: http://localhost:8000 Interactive API docs: http://localhost:8000/docs

---

3. Frontend Setup

Open a new terminal tab:

cd frontend
npm install

cp .env.local.example .env.local
# → NEXT_PUBLIC_API_URL=http://localhost:8000  (already set)

npm run dev

Frontend runs at: http://localhost:3000

---

Deploy Frontend to Vercel

cd frontend
npm i -g vercel
vercel

When prompted:

• Set NEXT_PUBLIC_API_URL to your backend URL (e.g. https://your-api.onrender.com)

Note on CORS: When deploying frontend to Vercel, update ALLOWED_ORIGINS in backend .env:

ALLOWED_ORIGINS=https://your-app.vercel.app,http://localhost:3000

---

Deploy Backend to Render (Free Tier)

1. Push the backend/ folder to GitHub
2. Go to https://render.com → New Web Service
3. Connect your repo, set:
   • Build Command: pip install -r requirements.txt
   • Start Command: uvicorn main:app --host 0.0.0.0 --port $PORT
4. Add environment variables:
   • OPENAI_API_KEY = your key
   • ALLOWED_ORIGINS = your Vercel URL

---

Project Structure

docsense/
├── backend/
│   ├── main.py                   # FastAPI app entry point
│   ├── config.py                 # Settings from .env
│   ├── requirements.txt
│   ├── start.sh                  # Quick start script
│   ├── .env.example
│   ├── models/
│   │   └── schemas.py            # Pydantic models
│   ├── routers/
│   │   ├── documents.py          # Upload, list, delete
│   │   ├── query.py              # Q&A endpoint
│   │   └── analytics.py         # Audit, alerts, anomalies
│   └── services/
│       ├── parser.py             # PDF / DOCX / CSV parsing
│       ├── vector_store.py       # FAISS index build & search
│       ├── llm.py                # OpenAI Q&A, summary, anomaly
│       └── store.py              # In-memory document registry
│
└── frontend/
    ├── package.json
    ├── next.config.js
    ├── tailwind.config.js
    ├── tsconfig.json
    ├── vercel.json
    ├── .env.local.example
    └── src/
        ├── app/
        │   ├── layout.tsx         # Root layout + sidebar
        │   ├── page.tsx           # Dashboard
        │   ├── upload/page.tsx    # File upload + summaries
        │   ├── query/page.tsx     # Q&A interface
        │   ├── alerts/page.tsx    # Smart alerts
        │   ├── anomalies/page.tsx # CSV anomaly detection
        │   └── audit/page.tsx     # Audit trail
        ├── components/
        │   └── layout/
        │       ├── Sidebar.tsx
        │       └── QueryProvider.tsx
        ├── lib/
        │   └── api.ts             # All API calls
        └── types/
            └── index.ts           # TypeScript interfaces

---

Environment Variables Reference

Backend .env

Variable	Required	Default	Description
OPENAI_API_KEY	✅ Yes	—	Your OpenAI API key
OPENAI_MODEL	No	gpt-4o-mini	Model to use for Q&A
ALLOWED_ORIGINS	No	http://localhost:3000	Comma-separated CORS origins
UPLOAD_DIR	No	./uploads	Where to store uploaded files
VECTOR_DIR	No	./vector_store	Where to store FAISS indexes
MAX_UPLOAD_MB	No	20	Max file size in MB
CHUNK_SIZE	No	800	Text chunk size for embedding
CHUNK_OVERLAP	No	100	Overlap between chunks
TOP_K_RESULTS	No	5	Number of chunks to retrieve per query

Frontend .env.local

Variable	Required	Description
NEXT_PUBLIC_API_URL	✅ Yes	URL of the FastAPI backend

---

How It Works — RAG Pipeline

User uploads file
       │
       ▼
  Parse document
  (PDF→text, DOCX→paragraphs, CSV→row groups)
       │
       ▼
  Split into chunks (800 tokens, 100 overlap)
       │
       ▼
  Embed with OpenAI text-embedding-3-small
       │
       ▼
  Store in FAISS index (one per document)
       │
  ─────┴──── Background also runs:
             • Auto-summary (LLM)
             • Alert extraction (dates, deadlines)
       │
       ▼
  User asks a question
       │
       ▼
  Embed question → Search FAISS across selected docs
       │
       ▼
  Retrieve top-K chunks (ranked by cosine similarity)
       │
       ▼
  LLM generates answer with [Source N] citations
       │
       ▼
  Return answer + citation cards to frontend
  Log query to audit trail

---

Demo Use Cases to Show Clients

Legal / Contracts

1. Upload a contract PDF
2. Ask: "What is the penalty clause?"
3. Ask: "When does this agreement expire?"
4. Check Alerts tab for extracted deadlines

Finance / CSV

1. Upload an expense CSV
2. Ask: "What was the largest expense in May?"
3. Go to Anomalies → Run analysis → See flagged rows

Multi-document

1. Upload two contracts
2. In Ask Docs, select both
3. Ask: "What are the differences in payment terms between the two agreements?"

---

Roadmap

• PostgreSQL / Supabase persistence (replace in-memory store)
• Pinecone vector DB (replace local FAISS for production scale)
• Clerk authentication + team workspaces
• Stripe billing integration
• API access tier
• White-label / custom domain support
• Slack + Google Drive integrations
• Bangla language support (South Asian market expansion — can be activated as a region-specific feature)

---

Tech Stack

Layer	Technology
Frontend	Next.js 14, Tailwind CSS, React Query
Backend	FastAPI, Python 3.10+
AI / LLM	LangChain, OpenAI GPT-4o-mini
Embeddings	OpenAI text-embedding-3-small
Vector Search	FAISS (local)
Document Parsing	pypdf, python-docx, pandas
Deploy (FE)	Vercel
Deploy (BE)	Render / Railway

---

License

MIT — free to use, modify, and build on.

---

DocSense — Turning documents into decisions.