DocuParse

Intelligent document parser for financial filings - Extracts text, tables, and validates data from SEC 10-K/10-Q documents using multiple AI models and cross-verification.

What This Does

DocuParse is a complete pipeline that:

• 📄 Downloads SEC filings (10-K, 10-Q) automatically
• 🔍 Extracts text using OCR + native PDF parsing
• 📊 Finds tables using multiple detection methods
• 🏗️ Detects layout with dual AI models (Detectron2 + LayoutLMv3)
• 🤖 Processes with Docling (IBM's document AI)
• ✅ Validates data against official XBRL filings
• 📈 Benchmarks performance and estimates costs

🎥 Demo Video

Watch the complete pipeline in action: DocuParse Pipeline Demo

📚 Interactive Tutorial

Follow the complete implementation guide: DocuParse CodeLabs

Quick Start

# 1. Setup
git clone <repository-url>
cd DocuParse
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# 2. Run the complete pipeline
dvc repro

# 3. Check results
ls data/exports/

📊 Results Dashboard

An interactive Streamlit dashboard visualizes the pipeline's recorded outputs — evaluation metrics, per-stage benchmarks (runtime & memory), build-vs-buy cost analysis, distribution drift, and the analysis reports. It reads the committed JSON/markdown outputs and does not re-run the extraction pipeline, so it is lightweight and independent of the heavy extraction stack (Docling, layout models, OCR).

Run locally

# Dashboard-only dependencies (no torch/docling needed)
pip install -r dashboard/requirements.txt

# Launch
streamlit run dashboard/app.py

Then open http://localhost:8501.

Deploy to Streamlit Community Cloud (free)

The dashboard is set up to deploy as-is — no extra work needed to keep the heavy pipeline dependencies out of the hosted build:

1. Push this repo to GitHub (already done).
2. Go to share.streamlit.io → Create app → Deploy a public app from GitHub.
3. Set:
   • Repository: Effyrt/Docuparse
   • Branch: main
   • Main file path: dashboard/app.py
   • Python version (Advanced settings): 3.13
4. Click Deploy.

Why this works cleanly: Community Cloud searches the entrypoint's directory before the repo root and uses the first dependency file it finds, so it installs dashboard/requirements.txt (streamlit + plotly + pandas only) instead of the heavy root requirements.txt. The Streamlit config lives at .streamlit/config.toml, which is where Community Cloud reads it from when the entrypoint is in a subdirectory.

A live "upload a PDF and parse it" demo is intentionally out of scope: the extraction stack is heavy and takes minutes per document, which is a poor fit for an always-on hosted demo. The dashboard focuses on the results the pipeline produces.

Project Structure

DocuParse/
├── 📁 src/                    # Source code
│   ├── extractors/           # Text, table, layout extraction
│   ├── downloaders/          # SEC filing downloads
│   ├── validation/           # XBRL cross-verification
│   └── benchmarking/         # Performance measurement
│
├── 📁 data/                   # Data files (DVC managed)
│   ├── raw/                  # Downloaded PDFs
│   ├── parsed/               # Extracted data
│   └── exports/              # Final outputs
│
├── 📁 reports/               # Analysis & findings
├── 📁 evaluation/            # Quality metrics & testing
├── 📁 notebooks/             # Development & demos
│
├── ⚙️ dvc.yaml               # Pipeline configuration
├── ⚙️ params.yaml            # Parameters
└── 📄 requirements.txt       # Dependencies

Key Features

🔧 Extraction Pipeline

• Text Extraction: Native PDF parsing + OCR fallback
• Table Detection: Camelot + pdfplumber hybrid approach
• Layout Analysis: Detectron2 + LayoutLMv3 dual models
• Docling Integration: IBM's enterprise document AI

✅ Quality Assurance

• XBRL Validation: Cross-check against authoritative financial data
• Accuracy Metrics: Word Error Rate, table precision/recall
• Regression Testing: Automated quality threshold validation
• Distribution Monitoring: Detect data drift and anomalies

📊 Performance Analysis

• Real Benchmarks: Actual runtime and memory measurements
• Cost Estimation: Cloud API pricing vs open-source infrastructure
• Bottleneck Analysis: Identify slowest pipeline stages
• Scaling Recommendations: Hardware and concurrency guidance

🏢 Build vs Buy Analysis

Comprehensive comparison against major cloud document AI services:

• Google Document AI: $1.50-$50 per 1,000 pages
• AWS Textract: $1.50-$50 per 1,000 pages
• Azure Form Recognizer: $10-$50 per 1,000 pages
• Our Solution: $1.05 per 1,000 pages (30-98% cost savings)

Additional Benefits: Complete data privacy, no API rate limits, custom financial document optimization

Pipeline Stages

1. Download → SEC filings from EDGAR
2. Text → Extract text with OCR fallback
3. Tables → Detect and extract financial tables
4. Layout → Identify document structure
5. Docling → AI-powered document understanding
6. Export → Consolidate results in multiple formats

Results

The pipeline processes 676 pages in ~3 minutes with:

• ✅ 99.56% native text extraction (minimal OCR needed)
• ✅ 187 tables extracted from financial statements
• ✅ 2,954 XBRL concepts validated against authoritative data
• ✅ Multiple formats: JSON, Markdown, CSV outputs

Key Reports

• reports/benchmarks.md - Performance analysis and cost estimates
• reports/text_analysis.md - Text extraction deep dive
• reports/docling_vs_custom_comparison.md - AI vs traditional methods
• reports/xbrl_cross_verification_report.md - Financial data validation
• benchmarks/results/cost_analysis_20250926_135135.json - Cloud API cost comparison

Configuration

Edit params.yaml to customize:

• Companies and filing years to process
• OCR and extraction thresholds
• Layout detection parameters
• Export formats and options

Dependencies

• Python 3.9+ with DVC for pipeline management
• Computer Vision: Detectron2, LayoutLMv3 for layout detection
• Document AI: Docling, pdfplumber, Camelot for extraction
• Validation: SEC EDGAR API, XBRL parsing libraries

👥 Team Contributions

This project was developed by a collaborative team with the following contributions:

• Hemanth Rayudu - 45%
• Peiying Chen - 45%
• Om Sailesh Raut - 10%

---

Built for: Financial document analysis, regulatory compliance, and automated data extraction from SEC filings.