Compares two versions of a CSV file and classifies every row as added, deleted, modified, or unchanged. For each modified row it produces a cell-level audit log recording the exact before and after value for every changed field.
When a dataset is refreshed — a contract register, a vendor list, an HR export, a regulatory submission — stakeholders need to know exactly what changed. Eyeballing two spreadsheets is error-prone, and a row count alone tells you nothing about which records were touched or which specific fields changed.
Contract Officers and CORs routinely receive revised deliverables and must document what the contractor changed. Data Stewards preparing DCAA audit packages need a traceable record of every amendment. Compliance teams submitting to federal data systems need a change log they can attach to the submission record.
This tool answers those questions automatically:
- Which records were added to or removed from the latest export?
- Which existing records had values changed, and in which fields specifically?
- How many individual cells changed across the entire dataset?
- Which columns change most frequently — potentially pointing to data quality or process issues?
| Feature | Description |
|---|---|
| Five-way row classification | Added, deleted, modified, unchanged counts with percentages; total cell-change count in the header |
| Cell-level audit log | Every changed field with before and after values, filterable by column name; downloadable as audit_log.csv |
| Schema mismatch detection | Columns present only in v1 or only in v2 are flagged in the header |
| Most-changed columns chart | Horizontal bar chart showing which fields are most volatile across the dataset |
| Modification heatmap | Binary matrix of modified rows × shared columns; suppressed when more than 60 rows are modified |
| Per-category downloads | Separate CSVs for added rows, deleted rows, modified rows, and the full audit log |
| Key validation | Stops with a clear error if the key column has duplicates or nulls rather than silently producing wrong output |
| Metadata columns | _changed_cols and _change_count added to modified rows so the most-changed records are immediately visible |
| Skill | Implementation |
|---|---|
| Data structures | DiffResult dataclass with 8 typed fields; changes DataFrame with key/column/before/after schema |
| Set-based algorithm | Outer join via set operations: added = after_keys - before_keys; no external diff library required |
| Type normalization | _normalise() converts to str, strips whitespace, unifies nan/None/<NA> to empty string |
| API design | Public API: 7 exports (1 class + 6 functions); validate_key() returns (bool, str) tuple |
| Caching strategy | @st.cache_data at two levels: file load and diff computation; diff keyed by JSON serialization |
| Plotly visualization | Donut chart, horizontal bar chart, heatmap (px.imshow) with consistent color scheme |
| Test coverage | 45 tests across 4 classes in 2 files; edge cases: empty DataFrames, whitespace normalization, schema drift |
| Compliance framing | Cell-level audit log; schema change warnings; per-category export for downstream records management |
csv-diff-analyzer/
├── app/main.py ← Streamlit UI: 5 tabs, sidebar, header metrics
│
├── src/
│ ├── differ.py ← DiffResult dataclass; compute_diff(); validate_key(); _normalise()
│ └── summary.py ← diff_summary(); column_change_counts(); change_log(); modification_heatmap_data()
│
├── data/
│ ├── contracts_v1.csv ← "before" snapshot (100 rows, 11 cols)
│ └── contracts_v2.csv ← "after" snapshot (104 rows, 11 cols; adds/deletes/edits)
│
├── tests/
│ ├── test_differ.py ← 22 tests: TestComputeDiff(18) + TestValidateKey(4)
│ └── test_summary.py ← 23 tests: TestDiffSummary(8) + TestColumnChangeCounts(5) + TestChangeLog(4) + TestModificationHeatmapData(6)
│
└── docs/
├── ARCHITECTURE.md ← Module reference, algorithm, design decisions
├── TESTING.md ← Test inventory, per-class breakdown, CI
└── DATA_DICTIONARY.md ← Input/output schemas, public API reference
| Outcome | Condition |
|---|---|
| Added | Key present in "after" only |
| Deleted | Key present in "before" only |
| Modified | Key in both; at least one shared column differs (after normalization) |
| Unchanged | Key in both; all shared columns identical |
| Schema only | Column present in only one file; flagged but not row-classified |
| Field | Type | Description |
|---|---|---|
key_col |
str |
Name of the key column used for matching |
added |
DataFrame |
Rows new in "after" |
deleted |
DataFrame |
Rows removed in "after" |
modified |
DataFrame |
Rows with at least one changed cell (includes _changed_cols, _change_count) |
unchanged |
DataFrame |
Rows with identical values in all shared columns |
changes |
DataFrame |
Cell-level change log: (key, column, before, after) |
shared_cols |
list |
Columns present in both files |
only_before |
set |
Columns present only in "before" |
only_after |
set |
Columns present only in "after" |
csv-diff-analyzer/
├── .github/
│ └── workflows/
│ └── tests.yml # CI — runs pytest on push and pull_request
├── .streamlit/
│ └── config.toml # Light theme configuration
├── app/
│ └── main.py # Streamlit entry point
├── data/
│ ├── contracts_v1.csv # "Before" contract snapshot
│ └── contracts_v2.csv # "After" contract snapshot
├── docs/
│ ├── ARCHITECTURE.md # Design decisions and module reference
│ ├── DATA_DICTIONARY.md # Input/output schemas and API reference
│ ├── TESTING.md # Test inventory and coverage details
│ └── ENGINEERING_DECISIONS.md # Six annotated design decisions with alternatives and rationale
├── scripts/
│ └── generate_sample_data.py # Sample data generator
├── screenshots/
│ ├── 01_overview.png # Dashboard overview — header metrics and donut chart
│ ├── 02_core_feature.png # Modified rows tab — changed cols and change count
│ └── 03_results.png # Audit log tab — cell-level before/after change log
├── src/
│ ├── __init__.py # Public API exports
│ ├── differ.py # DiffResult, compute_diff, validate_key
│ └── summary.py # Aggregation and reporting helpers
├── tests/
│ ├── test_differ.py # compute_diff and validate_key tests
│ └── test_summary.py # Summary function tests
├── CHANGELOG.md
├── LICENSE
└── requirements.txt
Python 3.10+ is required.
git clone https://github.com/RichieGarafola/csv-diff-analyzer.git
cd csv-diff-analyzer
python -m venv .venv
# Windows
.venv\Scripts\activate
# macOS / Linux
source .venv/bin/activate
pip install -r requirements.txtstreamlit run app/main.pyThe app opens at http://localhost:8501 by default.
Steps:
- Open the sidebar and toggle Use sample data (enabled by default, loads the contract snapshots)
- To use your own files, disable sample data and upload a "before" (v1) and "after" (v2) CSV
- Select the Key column — the unique identifier that links rows across both files
- The header row shows Added / Deleted / Modified / Unchanged counts and total Cells Changed
- Use the five tabs to explore results and download per-category CSVs
- The Audit Log tab shows every changed cell and can be filtered by column name
Tabs:
| Tab | Contents |
|---|---|
| Overview | Donut chart (row status), most-changed columns bar chart, modification heatmap (≤ 60 modified rows) |
| Added | Table of rows new in the "after" file; CSV download |
| Deleted | Table of rows removed in the "after" file; CSV download |
| Modified | Rows with changed values including _changed_cols and _change_count; CSV download |
| Audit Log | Cell-level change log filterable by column name; CSV download |
Programmatic usage:
import pandas as pd
from src import DiffResult, compute_diff, validate_key, diff_summary, change_log
df_v1 = pd.read_csv("data/contracts_v1.csv", dtype=str)
df_v2 = pd.read_csv("data/contracts_v2.csv", dtype=str)
ok, err = validate_key(df_v1, "contract_id")
if not ok:
raise ValueError(err)
result: DiffResult = compute_diff(df_v1, df_v2, "contract_id")
summary = diff_summary(result)
print(f"Added: {summary['rows_added']}, Deleted: {summary['rows_deleted']}, "
f"Modified: {summary['rows_modified']}, Cells changed: {summary['cells_changed']}")
audit = change_log(result)
audit.to_csv("audit_log.csv", index=False)data/contracts_v1.csv and data/contracts_v2.csv are two snapshots of a government contract register. The pair includes deliberate adds, deletes, and field-level edits to exercise all five diff categories.
| Column | Type | Description |
|---|---|---|
contract_id |
string | Unique key (CT-XXXX) |
vendor_name |
string | Contractor name |
award_amount |
integer | Total award value |
obligated_amount |
integer | Amount obligated to date |
contract_type |
string | FFP, IDIQ, Cost-Plus, BPA, T&M |
status |
string | Active, Pending, Closed, Suspended |
pm_name |
string | Program manager name |
state |
string | Place of performance (state abbreviation) |
start_date |
date | Contract start date (YYYY-MM-DD) |
end_date |
date | Contract end date (YYYY-MM-DD) |
description |
string | Short text description |
Regenerate with: python scripts/generate_sample_data.py
pytest tests/ -v45 tests across 4 classes in 2 files; all passing under 2 seconds.
| File | Class | Tests | Scope |
|---|---|---|---|
test_differ.py |
TestComputeDiff |
18 | Added/deleted/modified/unchanged classification; cell-level changes; whitespace normalization; schema tracking; empty DataFrames; metadata columns; row count invariant |
test_differ.py |
TestValidateKey |
4 | Valid key; missing column; duplicate keys; null keys |
test_summary.py |
TestDiffSummary |
8 | Required keys; row counts; totals; percentages; zero-diff case |
test_summary.py |
TestColumnChangeCounts |
5 | DataFrame shape; column presence; sort order; zero-diff case |
test_summary.py |
TestChangeLog |
4 | DataFrame shape; column presence; correct before/after values; zero-diff case |
test_summary.py |
TestModificationHeatmapData |
6 | DataFrame shape; key column present; only modified rows; binary values; changed cell = 1; zero-diff case |
- Unique key required — the diff is key-based. Files without a natural unique identifier cannot be compared without adding one.
- Duplicate key handling — if either file contains duplicate keys, the app halts rather than attempting positional matching.
- Column schema changes — columns present in only one file are flagged but not diffed. Adding or removing a column does not create row-level modifications for affected rows.
- All values treated as strings — both files are read with
dtype=str. Numeric formatting differences (1000vs1,000) are reported as changes. - Heatmap suppressed at scale — not rendered when more than 60 rows are modified.
- Memory — both files are loaded into memory. Very large files may exhaust available RAM.
- Two-file limit — only two snapshots are compared per session. Tracking changes across three or more versions requires separate sessions.
- Column ignore list to exclude always-changing fields like
last_updated - Fuzzy row matching for files without a natural unique key
- Side-by-side cell comparison view for a single selected modified row
- Sequential diff across more than two snapshots
- Configurable null equivalence rules (treat
0and blank as equivalent) - Summary report export as Markdown or HTML
| Overview | Core Feature | Results |
|---|---|---|
 |
 |
 |
| Header metrics and donut chart | Modified rows with change count | Audit log — cell-level before/after |
| Document | Description |
|---|---|
| docs/ARCHITECTURE.md | Module reference, compute_diff() algorithm, _normalise() logic, cache strategy, heatmap threshold, engineering design decisions |
| docs/TESTING.md | Test inventory, per-class breakdown, coverage details |
| docs/DATA_DICTIONARY.md | Input/output schemas, DiffResult field reference, public API |
| docs/ENGINEERING_DECISIONS.md | Six annotated design decisions: set operations, normalisation, key validation, annotation columns, audit log schema, dtype=str |
MIT License — see LICENSE for details.