Skip to content

docs: correct four claims the README makes about the tool - #90

Merged
kmadan merged 1 commit into
mainfrom
docs/readme-accuracy-fixes
Aug 30, 2026
Merged

docs: correct four claims the README makes about the tool#90
kmadan merged 1 commit into
mainfrom
docs/readme-accuracy-fixes

Conversation

@kmadan

@kmadan kmadan commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

🚀 Overview

Corrects four claims README.md and README-pypi.md make about the tool that
do not match the repository or the code. Companion to #89, which carries the
same pass into the 19 files #88 did not touch. Documentation only.

📜 Changes Made

1. Python badge overstated the supported range

pyproject.toml declares requires-python = ">=3.12,<3.13" — 3.12 alone. The
badge read "Python 3.12+", advertising every later release as supported.
Now reads 3.12. The four translated READMEs carry the same fix in #89.

2. --evaluators was documented but has no effect

The options table listed it as a way to restrict the evaluator set.
_run_evaluate accepts the argument and never forwards it:

# aicertify/cli.py
async def _run_evaluate(..., evaluators: Optional[list] = None, ...):
    return await aicertify_app_for_policy(
        contract=contract, policy_folder=policy_folder,
        output_dir=output_dir, report_format=report_format,
        custom_params=custom_params,      # <- evaluators not passed
    )

A value supplied there has no effect on the run. The row is removed.

Left for a separate change: whether the flag should be wired through or
rejected outright is a behaviour question, not a docs one. The parser still
accepts it silently. Worth a follow-up issue.

3. Two of three linked output directories were empty

Link Tracked files
examples/outputs/eu_ai_act/ 0 — dead on GitHub
examples/outputs/medical_diagnosis/ 1, a .gitkeep — renders empty
examples/outputs/loan_evaluation/ 3, including a real PDF

Now links artefacts that are actually committed: the retained fair-lending PDF
and its contract, plus the three expected_report.md files under examples/.
The EU AI Act demo report was already linked directly above and is unchanged.

4. Regal link pointed at the old organisation

Regal moved from StyraInc to open-policy-agent. The link now points at the
current location (open-policy-agent/regal, 401 stars, active) rather than
relying on GitHub's redirect.

✅ Already fixed by #88 — not repeated here

  • Policy count 85/9192
  • The "10-second demo" claim
  • The CLI example now uses aicertify evaluate --policy uk (console script and
    framework name) rather than python -m aicertify.cli with a full policy path

🎯 How to Test

grep -c "3.12%2B\|--evaluators\|StyraInc" README.md README-pypi.md   # -> 0

Verified locally: every relative link in README.md resolves to a tracked
path, and none of 3.12+, --evaluators or StyraInc remains on either
surface.

📚 Documentation

  • README.md, README-pypi.md
  • ✅ No API, CLI or policy behaviour changes

Four statements on the two English acquisition surfaces do not match the
repository or the code they describe.

pyproject declares requires-python = ">=3.12,<3.13", which admits 3.12 alone.
The badge read "Python 3.12+", which advertises every later release as
supported. It now reads 3.12. The four translated READMEs carry the same
correction in #89.

The options table listed --evaluators as a way to restrict the evaluator set.
_run_evaluate accepts the argument and does not forward it to
aicertify_app_for_policy, so a value supplied there has no effect on the run.
The row is removed. Whether the flag should be wired through or rejected is a
behaviour question and is left to a separate change.

Two of the three linked output directories hold no committed report.
examples/outputs/eu_ai_act/ has no tracked file at all, and
examples/outputs/medical_diagnosis/ holds only a .gitkeep, so both rendered as
empty on GitHub. The section now links the artefacts that are committed: the
retained fair-lending PDF and the contract that produced it, and the three
expected_report.md files under examples/. The EU AI Act demo report was already
linked directly above and is unchanged.

Regal moved from StyraInc to the open-policy-agent organisation. The link now
points at the current location rather than relying on the redirect.

Verified: every relative link in README.md resolves to a tracked path, and no
occurrence of "3.12+", "--evaluators" or "StyraInc" remains on either surface.
@github-actions github-actions Bot added 📝 documentation Improvements or additions to documentation 🚀 enhancement New feature or request labels Aug 30, 2026
@kmadan
kmadan merged commit 0cc0d81 into main Aug 30, 2026
7 checks passed
@kmadan
kmadan deleted the docs/readme-accuracy-fixes branch August 30, 2026 05:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

📝 documentation Improvements or additions to documentation 🚀 enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant