Skip to content

Let business rules join entities and conclude relations - #861

Merged
WaylandYang merged 3 commits into
deeplethe:devfrom
iuiu-py:feat/rule-entity-join
Sep 24, 2026
Merged

WaylandYang merged 3 commits into
deeplethe:devfrom
iuiu-py:feat/rule-entity-join

Conversation

@iuiu-py

@iuiu-py iuiu-py commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

What

This adds the missing one-hop relation join to business rules and lets a rule conclude a relation rather than only an entity or attribute. Conditions can now select the relevant side (x/y) of a declared relation, rule evaluation can traverse that relation when deriving matches, and accepted rules may emit the joined relation as a conclusion with evidence and reasoning provenance.

The backend changes cover rule storage/validation, evaluation, reasoning, API, and MCP responses. The rules UI adds relation-aware condition sides and relation conclusion rendering, with English and Chinese strings. Database state is extended by migration 0071_a_rule_joins_two_entities.sql.

Why

Closes #818. A user could previously express that two entities have attributes, but could not make one rule join them across a meaningful relation. That prevents several ordinary business assertions from being modeled, such as "if an organization is the supplier of a contract, the contract belongs to the organization." This change keeps those derivations explicit, validated against ontology relation ends, and visible in reasoning results.

The design decision is recorded in ADR 0047-a-rule-may-conclude-a-relation.md, and the decisions index is updated.

Testing

  • cargo fmt --all --check
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo test --workspace with UTOPIA_TEST_REQUIRE_DB=1 and UTOPIA_DATABASE_URL: 354 tests passed, 1 ignored, before an unrelated chat-persistence test encountered a transient PostgreSQL deadlock; that test passed when rerun, for 355 passed total in the server suite.
  • pnpm build
  • pnpm test — 129 tests in 19 files passed.
  • Confirmed 0071 remains available after checking the latest upstream migration (0070).

@WaylandYang

Copy link
Copy Markdown
Contributor

I wrote 0047, so I checked this against what the record actually decided rather than against what it sounds like. Decisions 1 through 3 are implemented faithfully, including the hard one.

Decision 3 is the one I expected to be fudged, and it is not. Inside the round loop, timed is the asserted pool extended with relation_edges, derive_with_blocked runs on that, and contradictions runs on the candidates — so a rule-concluded edge really does re-enter the axiom pass, per round, and the two reasoners meet where the record said they should. The comment "Rule edges have provisional ids in spans. They enter the axiom pool as ordinary timed edges" is the right sentence to have written.

Decision 2 is exact. premises.push(edge.id) before validity(&premises, spans) means the join edge is a premise, so it is both in the interval intersection and in the proof tree — the record required both and it would have been easy to do only the first.

Also noticed and appreciated: CURRENT_SCHEMA_VERSION is bumped in the same commit as migration 0071, which is the trap #832 fell into last week, and 0047's status line and the index row are both updated.

Three things.

1. Blocked rule-concluded edges have no row in the Review queue. This is the one I would fix before merging.

run() is untouched — the earliest hunk in reasoning.rs is at line 1001, and run() is at 218. It still derives from timed_edges() alone, so it never sees a rule-concluded edge. materialize() now blocks candidates that run() cannot reproduce, and the comment sitting in run() says exactly why that matters:

第六类(0017):推出来却落不了地的派生。与 materialize 用同一个函数算,所以这里报的正是那边拦下的——两边各算一套的话,队列会跟图对不上

That invariant — every blocked derivation has a row in Review — is what 0017 rests on, and after this change a relation conclusion that loses to an assertion is dropped with nothing to show a person. The graph will have a hole the queue cannot explain.

2. Rule::Transitive as a placeholder is contained, but only just. I traced it: the fake label reaches candidate_derivation, which feeds contradictions and the clash grouping, and it does not reach the axiom persistence loop where rules.get(&(d.via, d.rule.as_str())) would miss and silently increment unruled. So it is safe today. It is safe because of where the candidates are appended, though, not because anything prevents the other path — and d.rule.as_str() is written into a user-visible clash payload elsewhere. A variant that can name a business rule, or a separate candidate list that never borrows the axiom enum, would make that safety structural rather than positional.

3. Decision 4 is not satisfied. The record says the three caps are set by a measurement published in the PR that changes them, and gives the reason: a number chosen in a record is a guess wearing a decision's clothes. MAX_COMBOS is now applied per (rule, X, Y) rather than per subject, so the population it bounds became pairs times the combinations on each side — and no number in this PR says what that costs on a real base. The SEC and contracts corpora are what 0047 names. This does not have to block the code, but it should not land claiming 0047 is implemented while the decision that was deliberately deferred to implementation is still deferred.

CI has not reported yet beyond DCO; I have approved the run.

@WaylandYang WaylandYang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The migrations job failure reproduces locally against Postgres 16, and it is a
regression this branch introduces rather than a flake. Control run on dev with
the same database: cargo test -p utopia-store → 315 passed, 0 failed (116 test binaries), including
test a_contradiction_points_upstream ... ok. On this
branch, same command, same database:

panicked at crates/utopia-store/tests/a_contradiction_points_upstream.rs:189:9:
assertion `left == right` failed
  left: 0
 right: 1

Line 189 is assert_eq!(m.derived, 1);.

The cause looks like an unintended change to what DeriveReport.derived counts.
On dev:

derived: derivation.facts.len(),   // reasoning.rs:1722

On this branch it starts at 0 and accumulates merge_axiom_derivation's first
return value, which is wanted.len() - before — the number of new distinct rows
queued for write
. That loop skips blocked derivations, derivations with no valid
span, unruled ones, and anything whose or_insert key already exists.

So derived has moved from "derivations the engine produced" to "distinct new
derived rows this pass wants to write". In this test the one derivation is blocked
on purpose — the next assertion is assert_eq!(m.blocked, 1, "the derivation that hits an assertion stays out") — so the old count was 1 and the new one is 0.

Two things suggest this wasn't intended:

  • the field's doc comment is unchanged and still describes the old meaning
    (/// 这一轮算出来的派生总数)
  • derived is part of an HTTP response body
    (api/review_routes.rs:1196 and :1207), so the change is externally visible

If the new meaning is what you want, it probably deserves its own commit plus a
doc-comment and test update, rather than riding along inside the join feature.

The migration itself is fine — 0071 applies cleanly on a fresh database, the
number is free on dev, and the composite FK resolves against
relation_types_kb_id_key UNIQUE (kb_id, id) added by 0070.

One caveat: cargo stops after the first failing test binary, so there may be more
failures behind this one. --no-fail-fast would show the full picture.

wangzifei and others added 2 commits September 25, 2026 00:12
Signed-off-by: wangzifei <wangzifei@cit.group.hk>
… count, and bucket the join

Review fixes on deeplethe#861, applied as maintainer edits.

run() and materialize() now share one resolve() step: asserted edges
plus the surviving rule-concluded relation edges form the pool, derive()
runs over it once per round, and contradictions() sees every relation
candidate. A candidate that loses leaves the pool but stays in the
candidate list, so the queue row exists for it and says which business
rule produced it (`rule: business_rule`, `attribute_rule_id`); the
conclusions that stood on it retire with it, and a refused key is not
retried, so the fixed point still ends. The queue's key and foreign key
fall back to the last asserted premise when the chain runs through a
provisional edge.

`Rule::Business` names such a candidate instead of borrowing
`Transitive`, so nothing positional keeps a rule-concluded edge out of
the axiom persistence loop. `derive_with_blocked` goes: the pool is
rebuilt each round, so a refused edge is simply not in it.

`DeriveReport.derived` is again what the engine produced (the last
round's axiom derivations and relation candidates, plus the rules'
distinct conclusions), which `a_contradiction_points_upstream` pins.

`joined_evaluate` buckets the join edges by subject: scanning all edges
per X was quadratic in pairs, 5.9 s for 100,000 pairs against 82 ms for
10,000; bucketed it is 105 ms. Decision 4's numbers are in the record
and the PR; the caps stay.

Migration 0071 keeps its number, CURRENT_SCHEMA_VERSION is the file
count (75), and the two tests dev gained since the branch use the new
condition side and join argument.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Wayland Yang <wayland0916@gmail.com>
@WaylandYang

Copy link
Copy Markdown
Contributor

Rebased onto current dev and applied the review points as maintainer edits in a second commit; your commit and sign-off are kept.

Review queue (my point 1). run() and materialize() now share one resolve() step. A rule-concluded relation that loses to an assertion or another derivation leaves the pool but stays in the candidate list, so the queue gets a derived_contradiction row for it with rule: business_rule and the attribute_rule_id, and the queue matches the graph on the second run. Conclusions that stood on a refused edge retire with it; a refused key is not retried, so the fixed point ends. New store test a_relation_the_graph_refuses_still_reaches_the_review_queue pins all of that.

Rule::Business (my point 2). The candidate carries its own label now, so nothing positional keeps it out of the axiom persistence loop, and the label is what the queue and the rules-disagree defects show. derive_with_blocked is gone: the pool is rebuilt every round, so a refused edge is simply not in it.

derived (the CI regression). Back to what the engine produced; a_contradiction_points_upstream passes again with derived == 1, blocked == 1.

Decision 4. Measured in release on a synthetic base and written into the record. One derive() pass over 100,000 asserted edges is 29 ms, 52 ms with 100,000 concluded edges in the pool, so a round costs the rule evaluation, not the axiom pass. Joined evaluation was quadratic in pairs (5.9 s at 100,000 pairs, 82 ms at 10,000) because every X rescanned all join edges; bucketed by subject it is 105 ms at 100,000 pairs and 1.5 s at 64 combinations a pair. The caps stay; the record says the corpora it names should repeat this when they are to hand.

Also: migration 0071 keeps its number, CURRENT_SCHEMA_VERSION is 75 (file count), and the two tests dev gained since your branch (a_plan_step_follows_its_premise, sources_cleanup_tests) use the new condition side and join argument.

Locally on a fresh database: utopia-reason 86/86, utopia-store all suites, utopia-server 409/409, utopia-cli 13/13, workspace clippy clean, web 135/135 and build. Merging once CI agrees. Thanks for building the hard decision faithfully.

@WaylandYang WaylandYang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review points applied in the second commit (shared resolve step so the queue sees a refused relation, Rule::Business label, derived count restored, join bucketed, decision 4 measured and recorded); full local suites and CI green. Approving to clear my earlier request-changes.

@WaylandYang
WaylandYang merged commit 535aac0 into deeplethe:dev Sep 24, 2026
7 checks passed
@WaylandYang WaylandYang mentioned this pull request Sep 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A rule cannot join two entities

2 participants