Skip to content

feat: calibrated honesty — trust signals (confidence / provenance / review-state) across the pipeline #82

Description

@rajnavakoti

Principle

CB cannot own extraction accuracy — an LLM reasoning from partial, per-document context will over-fit (e.g. "Quarkus underlies Argo for Cleveron", or mis-typing Confluence/GitHub as service). But CB must own honesty: surfacing calibrated uncertainty so a human curator AND a consuming agent know what to trust.

A KB that's 80% right and honest about which 80% beats one that's 80% right and confidently wrong about the rest.

The real bug (observed)

On a real 3-doc extraction, every entity came out at confidence 0.9+ (zero below 0.6). The LLM is systematically overconfident, and CB passes that straight through to the consuming agent via MCP with no signal to doubt it. An agent reads "pickup-point-manager is a domain," believes it, and acts on it — because CB never told it to be unsure.

Tracking — four concrete pieces

  1. Calibrated confidence — replace uniform 0.9 self-ratings with signals that actually discriminate.
  2. Provenance — track source corroboration per entity AND per relationship (single-doc = weak).
  3. Review-state — distinguish raw-LLM output from human-verified.
  4. Propagate trust signals into retrieval + MCP — so the consuming agent receives confidence/provenance/review-state and can hedge.

(Sub-issues linked below.)

Pitch implication

Position CB as "honest structured knowledge that tells you and your agents exactly what to trust" — not "perfect KB", not "just a mechanism." It's the only position that survives the pickup-point-manager example, and more defensible than promising accuracy nobody can deliver.

Out of scope

  • Guaranteeing extraction accuracy (impossible; user owns ontology/prompts/source quality + unavoidable error).

Metadata

Metadata

Assignees

No one assigned

    Labels

    frameworkScientific frameworks

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions