From da568271cf2b772135746b5ec280a2c99795c51c Mon Sep 17 00:00:00 2001 From: propcgamer20-png Date: Thu, 10 Sep 2026 01:41:00 +0530 Subject: [PATCH] docs: fix counterfactual-fairness.md's COMPAS numbers to the repo-canonical figures The COMPAS "biased model" table cited Black 58.15% / White 28.37% / 29.78pp gap. Those numbers appear nowhere else in the repo. The canonical unfair.py COMPAS result, cited identically by README.md and 8+ other explainers, is Black 87.16% / White 0.40% / 86.77pp. Replaced the table with the canonical figures (and matched the "High-Risk Flag Rate" header wording used elsewhere). The surrounding causal-graph narrative (CustodyStatus proxy, prior arrests downstream of race, why dropping race alone didn't help) is unchanged and holds identically. Regenerated the page, package mirror, and llms-full.txt. Closes #551 --- explainers/counterfactual-fairness.html | 2 +- explainers/counterfactual-fairness.md | 8 ++++---- faircode/_explainers/counterfactual-fairness.md | 8 ++++---- llms-full.txt | 8 ++++---- 4 files changed, 13 insertions(+), 13 deletions(-) diff --git a/explainers/counterfactual-fairness.html b/explainers/counterfactual-fairness.html index 3d2be35..c072794 100644 --- a/explainers/counterfactual-fairness.html +++ b/explainers/counterfactual-fairness.html @@ -217,7 +217,7 @@

The Formal Definition

Concrete Example: COMPAS Recidivism

The COMPAS/ audit in this repo is a direct illustration of counterfactual fairness violation - even without using the causal framework explicitly.

COMPAS assigns recidivism risk scores based on features including prior arrest count, age at first arrest, and custody status. The biased model produces:

-
GroupHigh-Risk Rate
Black defendants58.15%
White defendants28.37%
Fairness Gap29.78pp
+
GroupHigh-Risk Flag Rate
Black defendants87.16%
White defendants0.40%
Fairness Gap86.77pp

Now ask the counterfactual question. Take a Black defendant with 3 prior arrests. Ask: if this person had been white, with the same underlying behaviour and the same 3 prior arrests, would their risk score be the same?

The answer is no - and the reason is CustodyStatus, which the proxy variables explainer identifies as the key proxy. Black communities have been subject to historical over-policing: for the same underlying behaviour, Black individuals are arrested at higher rates than white individuals. Prior arrest count is therefore not causally independent of race - it is partly caused by race, via differential policing. A model trained on prior arrests is therefore using a variable that race, in part, caused.

Race ──→ Policing intensity ──→ Arrest rate ──→ Prior arrests ──→ Risk score
diff --git a/explainers/counterfactual-fairness.md b/explainers/counterfactual-fairness.md index 9d6bb9e..42b76b8 100644 --- a/explainers/counterfactual-fairness.md +++ b/explainers/counterfactual-fairness.md @@ -62,11 +62,11 @@ The [`COMPAS/`](../COMPAS/) audit in this repo is a direct illustration of count COMPAS assigns recidivism risk scores based on features including prior arrest count, age at first arrest, and custody status. The biased model produces: -| Group | High-Risk Rate | +| Group | High-Risk Flag Rate | |-------|---------------| -| Black defendants | 58.15% | -| White defendants | 28.37% | -| **Fairness Gap** | **29.78pp** | +| Black defendants | 87.16% | +| White defendants | 0.40% | +| **Fairness Gap** | **86.77pp** | Now ask the counterfactual question. Take a Black defendant with 3 prior arrests. Ask: *if this person had been white, with the same underlying behaviour and the same 3 prior arrests, would their risk score be the same?* diff --git a/faircode/_explainers/counterfactual-fairness.md b/faircode/_explainers/counterfactual-fairness.md index 9d6bb9e..42b76b8 100644 --- a/faircode/_explainers/counterfactual-fairness.md +++ b/faircode/_explainers/counterfactual-fairness.md @@ -62,11 +62,11 @@ The [`COMPAS/`](../COMPAS/) audit in this repo is a direct illustration of count COMPAS assigns recidivism risk scores based on features including prior arrest count, age at first arrest, and custody status. The biased model produces: -| Group | High-Risk Rate | +| Group | High-Risk Flag Rate | |-------|---------------| -| Black defendants | 58.15% | -| White defendants | 28.37% | -| **Fairness Gap** | **29.78pp** | +| Black defendants | 87.16% | +| White defendants | 0.40% | +| **Fairness Gap** | **86.77pp** | Now ask the counterfactual question. Take a Black defendant with 3 prior arrests. Ask: *if this person had been white, with the same underlying behaviour and the same 3 prior arrests, would their risk score be the same?* diff --git a/llms-full.txt b/llms-full.txt index a99002b..172bcf6 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -3294,11 +3294,11 @@ The [`COMPAS/`](../COMPAS/) audit in this repo is a direct illustration of count COMPAS assigns recidivism risk scores based on features including prior arrest count, age at first arrest, and custody status. The biased model produces: -| Group | High-Risk Rate | +| Group | High-Risk Flag Rate | |-------|---------------| -| Black defendants | 58.15% | -| White defendants | 28.37% | -| **Fairness Gap** | **29.78pp** | +| Black defendants | 87.16% | +| White defendants | 0.40% | +| **Fairness Gap** | **86.77pp** | Now ask the counterfactual question. Take a Black defendant with 3 prior arrests. Ask: *if this person had been white, with the same underlying behaviour and the same 3 prior arrests, would their risk score be the same?*