Skip to content

feat: review-state — distinguish raw-LLM entities from human-verified #85

Description

@rajnavakoti

Part of #82 (calibrated honesty).

Problem

There's no way to tell a raw LLM-extracted entity (unverified, possibly wrong like "pickup-point-manager is a domain") from one a human has actually reviewed and confirmed. Consumers treat both identically. The human-in-the-loop curation loop needs a state to track.

Goal

A review_state on every entity (and ideally relationship) so both curators and consuming agents know what a human has vetted.

Proposed states

unreviewed (raw LLM output, default) → verified (human confirmed) → corrected (human edited) → rejected/deprecated.

Acceptance Criteria

  • review_state field on entities, defaulting to unreviewed on extraction
  • Curation actions (edit/confirm via viewer or API) transition the state
  • Re-extraction (source re-publish) does not silently reset a human-verified/corrected entity to unreviewed without signalling
  • Queryable: "show all unreviewed entities" (the curation worklist) and "% of KB human-verified" (a health metric)
  • Tests: state defaults to unreviewed; a confirm action sets verified

Out of scope

  • The full curation UI/queue (separate) — this is the state model + transitions
  • Propagating the state to retrieval/MCP (that issue)

Metadata

Metadata

Assignees

No one assigned

    Labels

    frameworkScientific frameworks

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions