diff --git a/README.md b/README.md index e604dc3..1d886c3 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,7 @@
-**GOPAL: Governance Open Policy Agent Library.** Think of it as an open policy pack for AI regulation. +**GOPAL: Governance Open Policy Agent Library.** An open policy pack for AI regulation. 92 policies that translate published regulation into industry-standard policy-as-code, written in Rego for [OPA](https://www.openpolicyagent.org/). Each one takes a named instrument, encodes its obligations as executable rules, cites the article or control it enforces, ships with tests, and appears in a coverage matrix that states what is implemented and what is not. The EU AI Act, NIST AI RMF, aviation safety standards, FERPA and COPPA in education, fair-lending rules in banking, and more. @@ -43,29 +43,27 @@ Run them against two kinds of input: facts you declare about the system and the ## The problem -An AI system changes weekly. A new model, an edited prompt, a rebuilt index. The document describing its compliance was written once and signed. +AI systems in active development change on the order of days through model updates, prompt edits, or index rebuilds. A signed compliance document records an assessment at a point in time; it does not automatically reflect subsequent system changes. Without re-evaluation, the documented state can diverge from the system being operated. A vendor score does not solve the same problem when the policy logic behind it cannot be inspected, contested, or traced to the source provision. -The two drift apart immediately, and nobody finds out until an audit, an incident, or a regulator asks. What runs in between, if anything, is usually a vendor's score: a number you cannot read, cannot argue with, and cannot trace to the provision it claims to cover. +**A signed document records a point-in-time assessment. An executable rule can be re-evaluated whenever the governed system changes.** -**A rule can run every time the system changes. A document cannot.** - -So the obligations are written as rules. Rego, in git, each citing the article it enforces, each with tests beside it, each returning a verdict you can take apart. The same rules run on a laptop, in a browser, and as a required status check on the pull request that changed the model: +GOPAL expresses those obligations as versioned Rego rules held in git. Each rule cites the article or control it enforces, has tests beside it, and returns a verdict that can be decomposed. The same rules run on a laptop, in a browser, or in CI. When GOPAL is configured as a required status check, a failing evaluation blocks the merge.

A terminal session. The production model scores 0.0056 against a 0.1 threshold and passes. After swapping the model, the aggregate is 0.1373, global.v1.toxicity.allow fails, and the output responsible is listed at 0.8106.

- Swap the model, keep the prompts and the classifier. A rule nobody edited stops the merge and names the output responsible. This example runs in CI on every push, and asserts both directions. + The example compares checked-in baseline and candidate toxicity metrics while the prompts, scoring method, and policy remain fixed. The candidate metrics cause the unchanged rule to fail, which blocks the merge and identifies the responsible output. This example runs in CI on every push and verifies both the passing and failing paths.

 

-This does not make compliance automatic, and anything claiming to is selling something. Most of what the EU AI Act obliges is a declaration nobody can measure: whether a conformity assessment was completed, whether a person can halt the system. Of the 185 fields these 29 EU policies read, **170 are declarations and 15 are measurements**. +Policy as code does not make every compliance determination automatic. In GOPAL's current EU AI Act implementation, the 29 policies contain **185 input-field reads: 170 declarations and 15 measurements**. Declarations cover facts such as whether a conformity assessment was completed or whether a person can halt the system. Measurements cover values produced by evaluators. -What changes is that the declaration becomes a versioned artefact re-checked on every commit instead of a PDF re-read once a year, and the 14 that can be measured are measured on every commit too. The [validation preview](https://principledevolution.ai/playground) shows both halves against your own system, in your browser, without sending us anything. +The distinction matters: manually supplying a value is still an assertion unless the value is backed by the evaluator evidence that produced it. GOPAL makes both kinds of input explicit and versionable. When GOPAL is wired into CI, the selected policy set is re-evaluated on every change covered by that workflow, including updated declarations and measurements. The [validation preview](https://principledevolution.ai/playground) shows both input types against your own system, in your browser, without sending us anything.

@@ -76,7 +74,7 @@ What changes is that the declaration becomes a versioned artefact re-checked on

-> **Tell us where this is wrong.** A policy library is only worth trusting if the people using it can argue with it, and the useful arguments are specific: a verdict you think is incorrect, a provision we mapped badly, a regulation we do not cover, an evaluation tool that should have an adapter. We corrected the model-card analysis once already because somebody did exactly that. +> **Tell us where this is wrong.** A policy library is only worth trusting if the people using it can argue with it, and the useful arguments are specific: a verdict you think is incorrect, a provision we mapped badly, a regulation we do not cover, an evaluation tool that should have an adapter. The model-card analysis was corrected once already in response to exactly that kind of report. > > [Request a framework](https://github.com/Principled-Evolution/gopal/issues/new?template=new_framework.yml) · [Request a policy](https://github.com/Principled-Evolution/gopal/issues/new?template=new_policy.yml) · [Report a wrong verdict](https://github.com/Principled-Evolution/gopal/issues/new?template=bug_report.yml) · [Start a discussion](https://github.com/Principled-Evolution/gopal/discussions) > @@ -113,7 +111,7 @@ Use GOPAL when you want AI governance checks that are: The EU AI Act is in force. The NIST AI RMF is the de facto US baseline. The UK, India, Brazil, Singapore, and California are all moving. Aviation regulators are publishing AI/UAS guidance. Financial supervisors are issuing model-risk requirements. -Engineering teams need AI governance checks that run in CI, not PDFs sitting on a shared drive or screenshots pasted into review-board decks. +Engineering teams need AI governance checks that run in CI. Neither a PDF on a shared drive nor a screenshot pasted into a review-board deck can be run against the system as it currently stands. GOPAL ships executable Rego policies for each of those regimes. They are versioned, testable, and reviewable in pull requests. The same tooling your platform team already uses for Kubernetes admission control can now enforce AI-system requirements. @@ -171,16 +169,16 @@ See [AICertify](https://github.com/Principled-Evolution/aicertify) for the full ## Why GOPAL -Most "AI governance" lives in slide decks. The few open implementations are either: - -- **Generic OPA bundles** (great for Kubernetes admission, not for the EU AI Act), or -- **Closed SaaS** that hides the rules you're being judged against. +Generic policy-as-code gives you the execution engine, but not an AI-regulatory policy library. Governance platforms can add inventory, workflow, and evidence management; where their decision logic is proprietary, that logic cannot be independently diffed or reproduced. GOPAL fills a different gap: **open, executable AI-governance policy that runs in standard OPA tooling.** Where GOPAL differs: -1. **AI-specific by construction.** Every policy targets an AI-system concern: bias, transparency, human oversight, model risk, content safety, safety-critical certification. Not generic infrastructure. -2. **Readable.** The rules are Rego. You can `cat` them, diff them in a PR, and reason about them. No black-box scorecards. -3. **Versioned.** Every framework lives under `v1/` (then `v2/`, etc.) with explicit semver guarantees (see [COMPATIBILITY.md](docs/COMPATIBILITY.md)). When the EU AI Act amends, the old version stays put. +1. **AI-specific policy content.** Policies target named AI regulations, standards, and sector requirements covering concerns such as bias, transparency, human oversight, model risk, content safety, and safety-critical certification. +2. **Inspectable by default.** The rules are Rego, source provisions are cited, and verdicts expose their constituent checks rather than only an aggregate score. +3. **Portable.** There is no proprietary policy DSL or hosted runtime requirement. Evaluate with `opa eval`, Conftest, an existing OPA server, or AICertify. +4. **Versioned and testable.** Frameworks live under stable versioned paths (`v1/`, then `v2/`, etc.) with explicit semver guarantees (see [COMPATIBILITY.md](docs/COMPATIBILITY.md)), and policy changes are exercised by tests in CI. + +The policy semantics remain open and independently reproducible even when GOPAL is used inside a larger governance workflow. --- @@ -214,21 +212,21 @@ Build them yourself with [`scripts/build-bundles.sh`](scripts/build-bundles.sh), ### Supplying measured metrics -Policies read two kinds of input: facts a person declares, and metrics a tool measures. The measured half is where an integration has to do real work, and [Plug your evaluator into GOPAL](docs/tutorials/supplying-metrics.md) walks it end to end in plain `opa`: find what a policy reads, use the canonical name from [`helper_functions/metrics.rego`](helper_functions/metrics.rego), write the JSON, gate a build on the result. No Python, no framework. +Policies read two kinds of input: declared facts and measured metrics. A declaration records what an accountable person or source asserts. A measured metric should carry provenance from the evaluator that produced it; typing the same number into JSON does not establish that the measurement occurred. [Plug your evaluator into GOPAL](docs/tutorials/supplying-metrics.md) walks the integration end to end in plain `opa`: find what a policy reads, use the canonical name from [`helper_functions/metrics.rego`](helper_functions/metrics.rego), write the JSON, and gate a build on the result. No Python and no framework are required. -### What automation actually looks like +### Automation in practice -Change the model, keep everything else, and watch a rule stop the merge: +With the prompts, scoring method, and policy held constant, the checked-in candidate metrics cause the existing rule to fail and block the merge:

Two runs of check.sh: the production model passes at 0.0056, the swapped model fails at 0.1373 with the offending output named

-[`examples/model-switch`](examples/model-switch) is the whole thing, and the **Compliance gate demo** badge above runs it on every push. It asserts both directions, because a gate that only ever passes is indistinguishable from a gate that is broken. +[`examples/model-switch`](examples/model-switch) contains the complete deterministic example, and the **Compliance gate demo** badge above runs it on every push. The workflow verifies both the passing baseline and failing candidate so that the CI path exercises both outcomes. -Most of the EU AI Act is declarations a person signs; nothing can measure whether a conformity assessment happened. Five policies run entirely on measured metrics, and those are the ones worth automating first: toxicity, Article 11 technical documentation, fair lending, diagnostic safety, and fairness. The example names each one and what supplies it. +Five policies in the current library can be evaluated entirely from measured metrics: toxicity, Article 11 technical documentation, fair lending, diagnostic safety, and fairness. These are natural starting points for automated CI enforcement. Other EU AI Act policies also depend on declared facts, including facts such as whether a conformity assessment was completed. The example names each metric-based policy and what supplies it. -If you want a Python framework that handles input capture and PDF/Markdown report generation on top, see [AICertify](https://github.com/Principled-Evolution/aicertify). It takes the scaffolding off you; it is not required to use any of this. +If you want a Python framework that handles input capture and PDF/Markdown report generation on top, see [AICertify](https://github.com/Principled-Evolution/aicertify). It supplies that scaffolding; it is not required in order to use these policies. --- @@ -414,7 +412,7 @@ See [CONTRIBUTING.md](.github/CONTRIBUTING.md) for the PR workflow. - **MAS / HKMA banking AI guidance** for APAC financial supervision - **Per-metric test coverage**: every policy is now tested against empty input, but the stronger check is removing one required metric at a time. That is what surfaced the most recent fail-open -Need a framework that isn't here? [Ask for it](https://github.com/Principled-Evolution/gopal/issues/new?template=new_framework.yml). You don't have to write any Rego to make the request. +If a framework you need is not covered, [ask for it](https://github.com/Principled-Evolution/gopal/issues/new?template=new_framework.yml). Making the request does not require writing any Rego. --- @@ -458,25 +456,28 @@ Contributions of any size are welcome; see [CONTRIBUTING.md](.github/CONTRIBUTIN ## How it fits together -Two diagrams, because the commonest misunderstanding is that a policy library -evaluates your model. It does not. It evaluates two kinds of statement that -come from two different places and carry two different levels of proof. +A policy library evaluates supplied evidence about a system rather than the +model itself. GOPAL separates that evidence into declared facts and measured +metrics because they establish different things.

How a GOPAL policy gets its inputs: declared facts asserted by a person, and measured metrics produced by evaluators such as AICertify running DeepEval or LangFair, both feeding a Rego policy that returns satisfied, not satisfied, or no conclusion

-Facts you **declare** are the ones no tool can measure: whether the CE marking -was affixed, whether logs are retained for six months. Metrics an evaluator -**measures** are the ones typing a number would not prove: toxicity, fairness -disparity, content safety. A policy reads both. +Facts you **declare** record assertions such as whether the CE marking was +affixed or whether logs are retained for six months. Metrics an evaluator +**measures** record results such as toxicity, fairness disparity, and content +safety. Typing a number into the input is still a declaration unless it is +backed by the evaluator evidence that produced the measurement. A policy can +read both.

GOPAL as a required status check: a pull request carries committed compliance facts, AICertify runs evaluators to add measured metrics, opa eval runs a pinned GOPAL bundle, and the status check passes with a retained report or fails naming the article and control

-Policy as code only means something once a policy can fail a pull request the -way a unit test does. Sources for both diagrams are in +When GOPAL is configured as a required CI status check, a failing policy +evaluation blocks the pull request in the same review workflow as other required +checks. Sources for both diagrams are in [`docs/diagrams/src/`](docs/diagrams/src), rendered with [`render-all.sh`](docs/diagrams/src/render-all.sh). diff --git a/docs/coverage/README.md b/docs/coverage/README.md index 5e36a31..bdd07b2 100644 --- a/docs/coverage/README.md +++ b/docs/coverage/README.md @@ -2,15 +2,15 @@ This directory documents, per framework, **which obligations are encoded in GOPAL** and **which are not yet**. -The matrices are deliberately honest. A policy is only marked **Implemented** when the Rego rule actually validates input fields against the regulation's requirement. Many directories ship a **Scaffold** that establishes the package path and a `default allow := false` placeholder, useful as a starting point for contributors but not yet enforceable. +The matrices record status conservatively. A policy is only marked **Implemented** when the Rego rule actually validates input fields against the regulation's requirement. Many directories ship a **Scaffold** that establishes the package path and a `default allow := false` placeholder, useful as a starting point for contributors but not yet enforceable. ## Test coverage Every policy has a sibling test file and asserts that its decision denies an empty input, as [CONTRIBUTING.md](../../.github/CONTRIBUTING.md) requires. That is now a CI gate rather than a convention: [`scripts/check-test-coverage.sh`](../../scripts/check-test-coverage.sh) fails the build if a policy has no test, or has one without an empty-input assertion. The seven libraries under `global/v1/common/` and `helper_functions/` are exempt, since they define helpers rather than decisions and have no `allow` to hand an empty input to. -The gate exists because the convention was not enough. The gap had reached 22 of the 96 files then counted as policies with no test at all, including both Article 5 prohibited-practice policies — the ones gating the practices the Act bans outright. An external reviewer changed `default allow := false` to `default allow := true` in the social-scoring policy, turning that gate into a default allow, and the suite still reported 604/604. A green suite and a per-file count were disagreeing, and the uncovered files were the prohibitions rather than the paperwork. +The CI gate exists because the convention alone was insufficient. When it was introduced, 22 of the 96 files counted as policies had no test, including both Article 5 prohibited-practice policies. An external review demonstrated the consequence: changing `default allow := false` to `default allow := true` in the untested social-scoring policy still left the aggregate suite reporting 604/604. The aggregate test count therefore did not establish per-policy coverage. -The exact figures live in [`coverage.json`](coverage.json) under `totals`, and are regenerated from the `.rego` files rather than typed here, so they cannot drift: +The exact figures live in [`coverage.json`](coverage.json) under `totals`. They are regenerated from the `.rego` files and verified in CI, keeping the reported totals synchronized with the policy tree: ```bash jq .totals docs/coverage/coverage.json @@ -19,7 +19,7 @@ jq .totals docs/coverage/coverage.json Writing a test that does not yet exist for a *case* rather than a file is still one of the most useful contributions available, and it needs no new Rego logic. Two patterns are worth copying: - **Empty input.** `not policy.allow with input as {}`. In Rego an undefined value is not `false`, so a missing `default` or an undefined intermediate rule can let a system with no evidence pass. -- **One required metric absent at a time.** This is strictly stronger, and it is what caught the most recent fail-open: a policy whose bias defaults correctly denied an empty input still approved a system whose toxicity had never been measured, because that one metric defaulted to the permissive answer. Empty input alone does not reach that path. +- **One required metric absent at a time.** This is stronger than an empty-input test because it exercises partial-input paths. The most recent fail-open occurred on such a path: a policy whose bias defaults correctly denied an empty input still approved a system whose toxicity had never been measured because that metric defaulted to a permissive answer. A policy without a test is not necessarily wrong. It is unverified, which is a different claim from the ✅ marks below. @@ -31,7 +31,7 @@ A policy without a test is not necessarily wrong. It is unverified, which is a d ## Implemented, matrix not yet written -These frameworks already have real policies in the repo (not scaffolds); nobody has written the per-obligation matrix for them yet. Good first contribution if you want to help without writing Rego. +These frameworks already have real policies in the repo rather than scaffolds; the per-obligation matrix for them has not been written yet. Writing one is a good first contribution and requires no Rego. - India Digital Policy: `international/india/v1/` - Brazil AI Governance Bill: `international/brazil/v1/` @@ -44,7 +44,7 @@ These frameworks already have real policies in the repo (not scaffolds); nobody ## Coming soon -Nothing implemented yet. The UK pro-innovation principles used to sit in this list; they now have six policies and a [matrix](uk.md). +Nothing in this section is implemented yet. The UK pro-innovation principles were previously listed here; they now have six policies and a [matrix](uk.md). - India DPDP Act (distinct from the Digital India Policy above) - MAS / HKMA banking AI guidance @@ -55,9 +55,9 @@ If you want to help expand coverage for a framework, open an issue or send a PR. ## coverage.json -[`coverage.json`](coverage.json) is generated by [`scripts/generate-coverage.sh`](../../scripts/generate-coverage.sh) and holds, for every policy: its package, title, references, decision rules, the `RequiredMetrics` and `RequiredParams` it declares, and whether it has a test and an empty-input test. CI fails if it is out of date, so it stays honest. +[`coverage.json`](coverage.json) is generated by [`scripts/generate-coverage.sh`](../../scripts/generate-coverage.sh) and holds, for every policy: its package, title, references, decision rules, the `RequiredMetrics` and `RequiredParams` it declares, and whether it has a test and an empty-input test. CI fails if it is out of date, providing a mechanical consistency check against the policy tree. -It exists because these matrices were hand-maintained and drifted. Use it to answer questions the prose cannot: +It exists because the coverage matrices were previously hand-maintained and had drifted from the policy tree. The generated file makes that relationship mechanically checkable. Use it to answer questions the prose cannot: ```bash # Which policies need a metric I do not yet collect? @@ -78,4 +78,4 @@ Each row is one obligation, control, or article in the source regulation. Column | **Status** | `Implemented` / `Scaffold` / `Planned` | | **Notes** | What the rule checks, or what's missing | -`Implemented` rules are safe to run in CI and produce structured verdicts. `Scaffold` rules return placeholder denials; they exist so the package path is stable while the logic is fleshed out. `Planned` means there's no file yet. +`Implemented` rules contain executable validation logic and produce structured verdicts suitable for CI evaluation. `Scaffold` rules return placeholder denials; they exist so the package path is stable while the logic is fleshed out. `Planned` means there's no file yet. diff --git a/examples/github-actions/README.md b/examples/github-actions/README.md index bba2d2a..568b15a 100644 --- a/examples/github-actions/README.md +++ b/examples/github-actions/README.md @@ -63,7 +63,7 @@ Recommendations: 1. Improve the completeness of technical documentation, model cards, and explainability information ``` -Exit code `1`. In Actions the `::error::` line becomes an annotation on the pull request, so the reviewer sees the reason and the remediation inline on the **Files changed** tab rather than having to open the log. +The exit code is `1`. In Actions the `::error::` line becomes an annotation on the pull request, so the reviewer sees the reason and the remediation inline on the **Files changed** tab rather than having to open the log. ## Dropping it into your own repo @@ -113,6 +113,6 @@ Call the script once per policy. Each sets its own exit code, and `set -e` in th **The bundle checksum is verified before use.** This bundle decides whether a release can ship, so the workflow downloads the release's `checksums.txt` and compares before evaluating. -**Undefined is not a pass.** The script distinguishes three outcomes: compliant (exit 0), non-compliant (exit 1), and failed-to-evaluate (exit 2). In Rego an undefined value is not `false`, so a policy that reached no verdict has told you nothing. Collapsing that into "not a failure" is how a compliance pipeline starts reporting green while checking nothing. +**Undefined is not a pass.** The script distinguishes three outcomes: compliant (exit 0), non-compliant (exit 1), and failed-to-evaluate (exit 2). In Rego an undefined value is not `false`, so a policy that reached no verdict provides no information. Collapsing that into "not a failure" is how a compliance pipeline starts reporting green while checking nothing. **The report is uploaded even when the build fails**, via `if: always()`. The artifact is the evidence that the check ran and what it decided, which is the thing an auditor asks for. diff --git a/examples/model-switch/README.md b/examples/model-switch/README.md index f55d390..5d02ffd 100644 --- a/examples/model-switch/README.md +++ b/examples/model-switch/README.md @@ -1,56 +1,60 @@ # Example: a model change that fails the build -Somebody swaps the model behind a support assistant. The prompts do not change, -the classifier does not change, and the policy does not change. One rule reaches -a different verdict and the merge stops. +This example compares recorded toxicity metrics for a baseline and candidate +model while holding the prompts, scoring method, and policy fixed. The candidate +metrics cause the existing rule to fail, so CI blocks the merge. -That is the whole example. It runs in this repository's own CI on every push, so -the badge on the README is a live statement rather than a screenshot of one good -afternoon. +The example runs in this repository's own CI on every push, so the badge on the +README reports the current state of the check rather than a recorded result. ## Run it ```bash ./check.sh outputs-baseline.json # the model in production -./check.sh outputs-candidate.json # after the swap +./check.sh outputs-candidate.json # candidate model ``` ![Two runs of check.sh: the production model passes, the swapped model fails with the offending output named](../../docs/demo/model-switch.svg) -Exit code 0 and 1. In CI that is a green check and a blocked merge. +The baseline and candidate runs exit 0 and 1 respectively. In CI, those exit +codes produce a passing check and a blocked merge. -## What is actually being demonstrated +## What the example demonstrates -Not the classifier. Detoxify scoring text is unremarkable, and swapping it for -Perspective or your own model changes nothing here. +The enforcement mechanism is independent of the scoring implementation. +Detoxify supplies the toxicity metrics in this example; Perspective or another +evaluator can be substituted if it produces the input fields expected by the +policy. -What is worth looking at is that **the number is judged by a rule you can read, -cite and version**, and that the same evaluation runs wherever you put it. The -threshold is not a constant in this script. It is -`params.toxicity_threshold` in -[`global/v1/toxicity`](../../global/v1/toxicity/toxicity.rego), defaulting to -0.1, and changing it is a pull request somebody reviews. +The score is evaluated by **a rule that can be read, cited and versioned**, and +the same policy evaluation can run in each environment. The threshold is defined +in the policy rather than in this script: `params.toxicity_threshold` in +[`global/v1/toxicity`](../../global/v1/toxicity/toxicity.rego) defaults to 0.1, +so changing it requires a reviewed pull request. -**One bad answer among many is invisible in an average.** The regression here is -a single output out of six. The aggregate moves from 0.0056 to 0.1373, which is -enough to cross 0.1, but the worst output moves from 0.02 to 0.81. GOPAL keeps -`metrics.toxicity.score` and `metrics.toxicity.max_toxicity` apart and compares -them against 0.1 and 0.7 for exactly this reason. Report only the mean over a -larger suite and the bad answer disappears into it. +**Aggregate metrics can mask individual outliers.** In this six-output example, +the aggregate toxicity score increases from 0.0056 to 0.1373 and exceeds its 0.1 +threshold. The maximum toxicity score increases from 0.02 to 0.81 and exceeds +its 0.7 threshold. GOPAL keeps `metrics.toxicity.score` and +`metrics.toxicity.max_toxicity` separate because, in a larger suite, an +individual high-toxicity output may not cause the aggregate score to exceed its +threshold. -**Nobody had to remember to check.** The gate is a status check, so the argument -happens on the pull request, with the article and the rule attached. +CI runs the policy evaluation automatically as a status check. A failing +evaluation therefore blocks the merge and keeps the applicable article and rule +in the pull-request review context. ## The numbers -`outputs-*.json` carry per-output toxicity scores recorded from Detoxify 0.5.2 +`outputs-*.json` contain per-output toxicity scores recorded from Detoxify 0.5.2 over the same prompt suite. They are checked in so the example is deterministic and needs no model download in CI. -That is a real limitation and worth stating: this example does not run a model -for you. It stages the situation after a model has been run and scored, which is -the situation a policy actually sees. To produce the numbers yourself, AICertify -ships a [Detoxify adapter](https://github.com/Principled-Evolution/aicertify/blob/main/docs/adapters.md) +This example does not execute either model. It starts after model outputs have +been generated and scored; the checked-in files represent the metrics presented +to the policy. This keeps CI deterministic while isolating the policy-enforcement +behavior. To generate equivalent metrics yourself, AICertify ships a +[Detoxify adapter](https://github.com/Principled-Evolution/aicertify/blob/main/docs/adapters.md) that emits exactly this shape, or see [supplying metrics](../../docs/tutorials/supplying-metrics.md) for the mapping written as plain JSON. @@ -71,9 +75,9 @@ model-card equivalent. ## Which policies work like this -Most of the EU AI Act is declarations a person signs, and no tool can measure -whether a conformity assessment happened. Five policies in the library run -entirely on measured metrics, and these are the ones worth automating first: +Five policies in the current library can be evaluated entirely from measured +metrics and are therefore suitable initial candidates for automated CI +enforcement: | Policy | Reads | Supplied by | | --- | --- | --- | diff --git a/industry_specific/README.md b/industry_specific/README.md index 08e31d4..2c2e1de 100644 --- a/industry_specific/README.md +++ b/industry_specific/README.md @@ -36,7 +36,7 @@ When adding new industry-specific policies: 3. Use the package name `industry_specific..v1.` 4. Include a `# METADATA` block referencing the specific standard or regulation it encodes -Adding a brand-new vertical (not one of the six above)? See [`skills/add-framework/SKILL.md`](../docs/skills/add-framework/SKILL.md) for the scaffold, and update the count in this file and the top-level README once it lands. +To add a vertical that is not one of the six above, see [`skills/add-framework/SKILL.md`](../docs/skills/add-framework/SKILL.md) for the scaffold, and update the count in this file and the top-level README once it lands. ## Composition diff --git a/international/README.md b/international/README.md index 975f23e..4b57670 100644 --- a/international/README.md +++ b/international/README.md @@ -42,7 +42,7 @@ When adding new international policies: 3. Use the package name `international..v1.` 4. Include a `# METADATA` block citing the specific article, section, or control the policy encodes -Adding an entirely new framework (a jurisdiction not listed above)? See [`skills/add-framework/SKILL.md`](../docs/skills/add-framework/SKILL.md) for the scaffold, and update the count in this file, `docs/coverage/`, and the top-level README once it lands. +To add an entirely new framework for a jurisdiction not listed above, see [`skills/add-framework/SKILL.md`](../docs/skills/add-framework/SKILL.md) for the scaffold, and update the count in this file, `docs/coverage/`, and the top-level README once it lands. ## Composition diff --git a/international/uk/README.md b/international/uk/README.md index 496f9d8..edf3617 100644 --- a/international/uk/README.md +++ b/international/uk/README.md @@ -18,7 +18,7 @@ One instrument in this directory *is* hard law. The automated decision-making re ## Where the UK diverges from the EU -Worth knowing if you already evaluate against `international/eu_ai_act/`: +These differences matter if you already evaluate against `international/eu_ai_act/`: - **No risk tiering by system class.** There is no UK equivalent of the EU's prohibited / high-risk / limited-risk classification. Obligations arrive through sector regulators instead. - **Automated decisions are permitted by default.** Under Articles 22A-22D a solely automated significant decision on ordinary personal data is lawful provided the Article 22C safeguards are in place. Special category data remains restricted and additionally needs an Article 9(2) condition.