Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,13 @@ only the Contracts needed to govern them before implementation starts.
it was based on, so changing a contract, the code, or the authority behind it
marks the work that depended on it stale and asks for it again. A contract that
drifted from the code does not silently keep passing. Committing only ADF Result
or Evidence Records does not invalidate the product inputs those records verify.
or Evidence Records does not invalidate the product inputs those records verify. After
`adf_add_evidence`, a refreshed `adf_next` Context may include that Evidence and
other Evidence for the same Requirement. They are not retroactive prerequisites
of the verification record. Evidence dependencies captured when the record was
created still have to match. Submit truthful `unsatisfied` or `inconclusive`
outcomes with the corresponding Evidence in `basis_refs` and `output_refs`;
no commit or manual rewriting of Evidence is needed.

**Nothing reviews its own work.** Implementation and challenge are separate
roles, and a post-build challenge runs in a context that did not build the
Expand Down
2 changes: 2 additions & 0 deletions docs/concepts.ja.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,8 @@ Challengerが行う反証とは、依頼・規範・実装が間違っている

ResultとEvidenceの鮮度は、それぞれが確認したコード、Contract、Decision、Changeの内容で判定します。実行時のGit revisionは追跡情報として残しますが、ADFのResultやEvidenceだけをcommitしてHEADが進んでも、確認対象の内容が同じなら再確認は要求しません。

Evidenceを追加してから`adf_next`でContextを取り直しても、Evidence自身や後から同じRequirementに追加したEvidenceのdigestを、過去の検証記録へ書き足す必要はありません。作成時に入力として記録した別のEvidenceへの依存は、引き続き鮮度を確認します。検証が失敗した場合は`unsatisfied`、判定できなかった場合は`inconclusive`として、対応するEvidenceを`basis_refs`と`output_refs`に指定して提出できます。提出前のcommitは不要です。

Contractまたは条項の`evidence_mode`で、検証に掛ける費用を選べます。`direct`は、その条項を何によって確認したかをEvidenceの`claims`へ明示します。`inherited`は、別の条項と共通するテストや実証結果を再利用できます。`review`はEvidenceを要求せず、独立したChallengerによる反証の対象にします。設定のない既存Contractは従来どおり全条項を`direct`として扱います。新しいContractでは、全体を`review`として、事故やデータ破損などにつながる条項だけを`direct`または`inherited`で上書きできます。

## 文脈の再利用と実行コスト
Expand Down
6 changes: 6 additions & 0 deletions skill-src/adf-builder/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,12 @@ an Evidence Record and should not cause extra work.
binding keeps the Evidence current when only ADF Result or Evidence Records are committed,
while a change to the code, Contract, Decision, or Change it verified makes it stale. Keep
`git_revision` as execution provenance; do not rewrite it to the later Record-only commit.
After adding Evidence, you may refresh with `adf_next` and submit with the refreshed
Context and the Evidence IDs in `basis_refs` and `output_refs`, without a commit.
Evidence does not need to capture its own digest or later Evidence for the same
Requirement. Dependencies captured at creation, including other Evidence, still
must match. Report failed or inconclusive verification as `unsatisfied` or
`inconclusive`, backed by its verification record.

Every residual risk needs someone who accepts it and a date by which it is revisited.

Expand Down
15 changes: 15 additions & 0 deletions src/kernel.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1016,8 +1016,23 @@ pub(crate) fn evidence_matches_inputs(
return string_field(&snapshot.repository, "revision")
.is_some_and(|revision| string_field(evidence, "git_revision") == Some(revision));
}
// Evidence produced for the same Requirement can appear only after the
// verification inputs were captured. It is an output, not a prerequisite
// that earlier Evidence must retroactively record. Keep checking every
// recorded dependency below, including Evidence used by this verification.
let instances = string_array_set(&evidence["requirement_instances"]);
let output_evidence = snapshot
.evidence
.iter()
.filter(|candidate| {
string_field(candidate, "change_id") == string_field(evidence, "change_id")
&& !instances.is_disjoint(&string_array_set(&candidate["requirement_instances"]))
})
.filter_map(|candidate| string_field(candidate, "id"))
.collect::<BTreeSet<_>>();
required_inputs
.iter()
.filter(|(reference, _)| !output_evidence.contains(reference.as_str()))
.all(|(reference, digest)| recorded_inputs.get(reference) == Some(digest))
&& recorded_inputs
.iter()
Expand Down
75 changes: 75 additions & 0 deletions src/submission.rs
Original file line number Diff line number Diff line change
Expand Up @@ -712,6 +712,81 @@ mod tests {
}
}

#[test]
fn refreshed_evidence_inputs_preserve_content_and_recorded_dependencies() {
let mut snapshot = snapshot();
for reference in ["code.test", "contract.test", "decision.test"] {
snapshot
.artifact_digests
.insert(reference.to_owned(), format!("sha256:{}", "a".repeat(64)));
}
let inputs = snapshot
.artifact_digests
.iter()
.filter(|(reference, _)| reference.as_str() != "evidence.test")
.map(|(reference, digest)| (reference.clone(), digest.clone()))
.collect::<BTreeMap<_, _>>();
snapshot.evidence[0]["input_refs"] = string_map_value(&inputs);
let evidence = snapshot.evidence[0].clone();
assert!(evidence_matches_inputs(
&evidence,
&snapshot.artifact_digests,
&snapshot
));

let mut sibling = evidence.clone();
sibling["id"] = json!("evidence.sibling");
snapshot.evidence.push(sibling);
snapshot.artifact_digests.insert(
"evidence.sibling".to_owned(),
format!("sha256:{}", "c".repeat(64)),
);
assert!(evidence_matches_inputs(
&evidence,
&snapshot.artifact_digests,
&snapshot
));
for reference in ["code.test", "contract.test", "decision.test", "change.test"] {
let mut changed = snapshot.clone();
changed
.artifact_digests
.insert(reference.to_owned(), format!("sha256:{}", "f".repeat(64)));
assert!(
!evidence_matches_inputs(&evidence, &changed.artifact_digests, &changed),
"changed {reference}"
);
// Even the original Context must not admit changed content.
assert!(!evidence_matches_inputs(
&evidence,
&snapshot.artifact_digests,
&changed
));
}
let mut dependent = evidence.clone();
dependent["input_refs"]["evidence.sibling"] =
json!(snapshot.artifact_digests["evidence.sibling"]);
assert!(evidence_matches_inputs(
&dependent,
&snapshot.artifact_digests,
&snapshot
));
snapshot.artifact_digests.insert(
"evidence.sibling".to_owned(),
format!("sha256:{}", "d".repeat(64)),
);
assert!(!evidence_matches_inputs(
&dependent,
&snapshot.artifact_digests,
&snapshot
));
snapshot.evidence[1]["requirement_instances"] = json!(["other|operation.test"]);
assert!(!evidence_matches_inputs(
&evidence,
&snapshot.artifact_digests,
&snapshot
));
}

#[test]
fn evidence_backed_submission_rejects_a_shallow_attestation() {
let error = prepare_result(
Expand Down
161 changes: 161 additions & 0 deletions tests/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5617,13 +5617,137 @@ fn validate_migration_candidate_cli(root: &Path, candidate: &str) -> Output {
.unwrap()
}

#[test]
fn evidence_submission_after_refresh_without_commit() {
use adf::project_application::ProjectApplicationService;
for (status, verification) in [("unsatisfied", "failed"), ("inconclusive", "inconclusive")] {
for (evidence_ids, refresh_between) in [
(vec!["evidence.first"], false),
(vec!["evidence.first", "evidence.second"], false),
(vec!["evidence.first", "evidence.second"], true),
] {
let project = TestProject::with_evidence_rule(true);
let revision = git_output(&project.root, &["rev-parse", "HEAD"]);
let mut service =
ProjectApplicationService::new(&project.root, Some(project.release_root.clone()))
.unwrap();
let first = service.next("change.place-order", false).unwrap();
let context = &first.next_response["context"]["payload"];
let outcomes = context["requirement_instances"]
.as_array()
.unwrap()
.iter()
.map(|instance| {
json!({
"instance_key": instance["instance_key"],
"definition_digest": instance["definition_digest"],
"status": "satisfied",
"summary": "Reviewed fixture signals",
"basis_refs": ["change.place-order"]
})
})
.collect::<Vec<_>>();
let candidates = context["signal_candidates"]
.as_array()
.unwrap()
.iter()
.map(|candidate| {
json!({
"fingerprint": candidate["fingerprint"],
"status": "confirmed",
"reason": "Fixture operation writes data",
"basis_refs": candidate["evidence_refs"]
})
})
.collect::<Vec<_>>();
service
.submit(
first.issued_action.as_ref().unwrap(),
json!({"outcomes": outcomes, "reviewed_candidates": candidates}),
vec![],
None,
)
.unwrap();
let next = service.next("change.place-order", false).unwrap();
assert_eq!(
next.next_response["next_action"]["action"],
"record-evidence"
);
let mut key = next.issued_action.unwrap();
let instances = next.next_response["context"]["payload"]["requirement_instances"]
.as_array()
.unwrap();
let instance_keys = instances
.iter()
.map(|i| i["instance_key"].clone())
.collect::<Vec<_>>();
for id in &evidence_ids {
service
.add_evidence(
&key,
json!({
"schema_version": "1",
"id": id,
"change_id": "change.place-order",
"requirement_instances": instance_keys,
"method": "Fixture verification",
"outcome": verification,
"summary": "Recorded actual verification result"
}),
)
.unwrap();
if refresh_between {
key = service
.next("change.place-order", false)
.unwrap()
.issued_action
.unwrap();
}
}
let refreshed = service.next("change.place-order", false).unwrap();
let refreshed_instances =
&refreshed.next_response["context"]["payload"]["requirement_instances"];
let outcomes = refreshed_instances
.as_array()
.unwrap()
.iter()
.map(|instance| {
for id in &evidence_ids {
assert!(instance["sources"].get(*id).is_some());
}
json!({
"instance_key": instance["instance_key"],
"definition_digest": instance["definition_digest"],
"status": status,
"summary": "Report actual verification result",
"basis_refs": [evidence_ids.last().unwrap()]
})
})
.collect::<Vec<_>>();
service
.submit(
refreshed.issued_action.as_ref().unwrap(),
json!({"outcomes": outcomes}),
evidence_ids.iter().map(|id| (*id).to_owned()).collect(),
None,
)
.unwrap();
assert_eq!(git_output(&project.root, &["rev-parse", "HEAD"]), revision);
}
}
}

struct TestProject {
root: PathBuf,
release_root: PathBuf,
}

impl TestProject {
fn new() -> Self {
Self::with_evidence_rule(false)
}

fn with_evidence_rule(evidence_only: bool) -> Self {
use std::sync::atomic::{AtomicU64, Ordering};
static SEQUENCE: AtomicU64 = AtomicU64::new(0);
let manifest_root = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
Expand Down Expand Up @@ -5656,6 +5780,31 @@ impl TestProject {
&manifest_root.join("schemas/v1"),
&release_root.join("schemas/v1"),
);
if evidence_only {
let rules_path = release_root.join("rules.yaml");
let mut rules = read_yaml(&rules_path);
rules["requirements"].as_array_mut().unwrap().retain(|r| {
r["id"] == "risk-signals-reviewed" || r["id"] == "data-evidence-recorded"
});
let requirement = &mut rules["requirements"][1];
requirement["phase"] = json!("before-build");
requirement["context"] = json!([
"change",
"matching-contracts",
"matching-decisions",
"affected-code",
"matching-evidence"
]);
rules["rules"].as_array_mut().unwrap().retain(|r| {
r["id"] == "baseline.review-risk-signals"
|| r["id"] == "persistent-data.record-evidence"
});
rules["rules"][1]
.as_object_mut()
.unwrap()
.remove("repository_phase");
write_yaml(&rules_path, &rules);
}
let signing_key = SigningKey::from_bytes(&[7_u8; 32]);
let artifact_digest = write_signed_release(
&release_root,
Expand All @@ -5681,6 +5830,18 @@ impl TestProject {
write_yaml(&root.join(".adf/trusted-release-keys.yaml"), &trust_store);
let framework_lock_path = root.join(".adf/framework.lock");
let mut framework_lock = read_yaml(&framework_lock_path);
if evidence_only {
let rules = read_yaml(&release_root.join("rules.yaml"));
let schemas =
adf::schema::SchemaRegistry::load(release_root.join("schemas/v1")).unwrap();
framework_lock["rule_set"]["source_digest"] = json!(canonical_digest(&rules).unwrap());
framework_lock["rule_set"]["index_digest"] = json!(
adf::rules::compile_rule_index(&rules, &schemas)
.unwrap()
.digest
);
}

framework_lock["schema_version"] = Value::String("2".to_owned());
framework_lock["release_artifact"] = json!({
"artifact_digest": artifact_digest,
Expand Down
Loading