Which failures should disqualify a persuasive answer, and who can justify that judgment? The Catholic Doctrinal Fidelity Index (CDFI) proposes scoring rules and review procedures for evaluating AI responses within the SAICRED project. Research question.
Policy reader: understand in 90 seconds · Reviewer: inspect evidence and limits · Engineer: run the calculator
| Current account | |
|---|---|
| Purpose | Connect domain-specific evaluation rules with sources, scoring and review. Method |
| Status | Evolving. Scoring-contract reconciliation and human theological validation remain open. Research account |
| Evidence type | Source translations, a public calculator and attributed reports from a separate production pipeline. Review guide |
| Version | Archived v1.5, May 31, 2026. The active branch includes later changes. Publication record |
| License | Declared Apache-2.0. Preserved license · attribution |
| Cite | Archived v1.5: 10.5281/zenodo.20475185 · citation metadata |
The method remains provisional. Differences among the formula, configuration and calculator require reconciliation. Qualified human theological review remains open. The public calculator supplies a bounded implementation demonstration. Current status, contract limits.
A fluent response can still mishandle a source or a doctrinal distinction. CDFI asks how an evaluation should represent those failures and who should review its judgment. Its contribution connects selected research ideas to proposed metrics, scoring rules, score caps and reliability checks. A score cap limits a response's numerical score when a declared failure occurs. Research account, translation method.
The work translates selected ideas from seven AI safety publications into a Catholic doctrinal evaluation context. Those translations contain researcher judgments about how findings apply to a new setting. Reviewers can inspect the sources, reasoning and proposed rules separately. Source-to-method map, claim packs.
The current evidence leaves the validity of particular weights, thresholds and institutional uses open. Technical consistency checks have a narrower scope than qualified theological review. The next steps are to reconcile the scoring contract, reproduce results from exact inputs and complete the required human assessment. Research limits, validation status.
flowchart LR
accTitle: CDFI source translation and review path
accDescr: Selected source findings inform a domain interpretation. That interpretation proposes a scoring rule. Technical checks examine behavior, while qualified human review assesses doctrinal judgments. The final human review remains open.
A["Selected source<br/>finding"] -->|informs interpretation| B["Proposed metric<br/>or scoring rule"]
B -->|examines behavior through| C["Technical<br/>checks"]
C -->|retains open questions for| D["Qualified human<br/>review"]
The text sequence is: identify a source claim, explain the domain interpretation, propose a scoring rule, check its behavior and obtain qualified human review. The diagram represents a method under development. It leaves empirical effectiveness and theological validity open. Translation method, research status.
| Material | What is available | What remains open |
|---|---|---|
| Research translations | Seven source publications and eight machine-readable claim packs. Traceability · claim index | Source findings, researcher inferences and proposed design choices require separate assessment. Translation method |
| Public calculator | Four built-in demonstrations of score calculation and caps. Calculator | Input scales, weights and gate handling differ across artifacts. Current review |
| Reported SAICRED results | Attributed summaries of the external production pipeline. Results summary | Public files alone cannot reproduce the private pipeline or settle conflicting reported values. Review guide |
| Human review | Named responsibilities and proposed publication gates. Validation status | Authority classification and human theological validation remain pending. Current research status |
SAICRED means Standard for Assessing AI for Catholic Reliability and Doctrinal Fidelity. Its project leadership, production engineering and CDFI methodology have distinct contributors. The following account preserves the existing project contribution record.
| Role | Person | Recorded contribution |
|---|---|---|
| Project Lead | Filip Ponulak, PhD | Originated SAICRED, designed its overall framework and evaluation criteria, and holds theological authority and publication ownership. |
| Lead Engineer | Naveen Kumar Puppala | Built the production pipeline, including automated judging, metric computation and the results dashboard. |
| Evals Expert | Mark Julius Banasihan | Developed the CDFI scoring architecture and evaluation governance methodology. |
| Team member for future work | William J. Benjamin, PhD | Joined for future work; this credit attributes no contribution to the existing release. |
The framework citation identifies Banasihan's CDFI work. It preserves the wider project's separate leadership and contributor credits. Named participation alone leaves independent validation unestablished. Contribution and citation boundary, publication provenance.
The verification guide gives exact commands, the four current outputs and their limits. The calculator uses Python's standard library and makes no model calls. The private production data and judge-reliability suite require separate access and evidence. Calculator, validation-status register.
The project records Claude assistance with methodology, drafting, scoring architecture and repository construction during March to May 2026. Gemini 2.5 Flash serves a separate role as the reported automated judge. Mark Julius Banasihan retains responsibility for adopting material within his CDFI work. AI-use disclosure.
The inspected record lacks a complete task-level model history and a uniform record of human checks for every generated contribution. Reported technical checks, author acceptance and qualified theological review retain separate scopes. Disclosure and verification limits, validation status.
This block renders the existing archived-release identity in CITATION.cff. For later development, also record the exact Git commit. The active branch and archived v1.5 snapshot remain separately identifiable. Publication record.
@software{banasihan2026cdfi,
author = {Banasihan, Mark Julius},
title = {CDFI Framework: Evaluation Governance Infrastructure for Domain-Specific AI Doctrinal Benchmarking},
year = {2026},
month = may,
version = {1.5},
doi = {10.5281/zenodo.20475185},
url = {https://doi.org/10.5281/zenodo.20475185},
note = {Archived release, May 31, 2026}
}Technical catalog, prior entry and publication history
The complete prior README preserves the entry at audit commit b2d635c.
Evaluation governance research by Mark Julius Banasihan. Node & Norm · Research integrity.