Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions docs/design/iter-slm251-lot1-02-not-authorized-20260725.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
{
"contract_hash": "f358dcc954a6a96c6e494acbabc8635b79184ca15b9d80f3f59ff7d3e7416ace",
"contract_id": "lot1-02-launch-gate-v1",
"gate_1_transfer_authorization": {
"actual_verdict": "needs_target_trace_contract",
"gate_name": "fidelity_contract_authorization",
"met": false,
"rationale": "Differentiation from every existing repo mechanism (masked recursive denoiser, SemanticPlanV1 predictors/consumers, causal FTPO adapters, explicit compiler traces, valid-state tree-edit/lattice search) is clean and survives line-by-line comparison -- see non_duplication_map, all rows no_duplication. The blocker is narrower: LOTUS's explicit-to-latent curriculum (mechanism row 5) and its causal_latent_use claim class both presuppose a step-decomposable 'explicit trace' analogous to GSM8K CoT steps that this repo does not yet define for OpenUI program synthesis. Defining that target-trace contract honestly -- without assuming math-payload equivalence to GSM8K, and without silently reusing causal_trace.py's visible-token decode capture or SemanticPlanV1's plan IR as if either already were that trace -- is out of scope for this docs/spec-only issue (no corpus generation beyond bounded target-contract probes). Authorizing bounded implementation now would force either an undefined curriculum or a quietly borrowed non-LOTUS trace source; neither is honest. This is not a 'differentiation is weak' close and not a semantic-floor block -- it is a scoped prerequisite.",
"required_verdict": "authorize_bounded_implementation",
"source_contract_hash": "801ce267b64f52b88e6e80fa091084c5f1a6628de60658d2d161db82f5117af2",
"source_contract_id": "lotus-openui-fidelity-contract-v1",
"source_linear_issue": "SLM-248"
},
"gate_2_trace_oracle_ceiling": {
"actual_verdict": "inconclusive",
"gate_name": "trace_oracle_ceiling",
"met": false,
"rationale": "The trace contract, typed step schema, deterministic extractor, and lossless visible serialization are implemented and tested against the repository's existing SemanticPlanV1/production_codec owners (no new compiler/parser/evaluator). A bounded n=16 fixture probe (not a production corpus) confirms every probe record decomposes into all six declared stages with zero truncation, giving a provisional K=6/c=479-chars budget. The oracle ceiling experiment itself -- the only evidence that could support an oracle_ceiling_positive, no_downstream_ceiling, or explicit_trace_equivalent_to_existing_plan verdict -- is specified as a matched multi-arm, multi-seed plan but deliberately not run: the LOT0-01 authorization scopes this issue to a fixture-scale plan only, with GPU training and corpus generation explicitly blocked. Declaring any positive or negative ceiling verdict without running that plan would be exactly the 'no claim that explicit trace benefit implies latent benefit' failure this issue's acceptance criteria forbid. leakage_or_ambiguity_blocked and target_support_insufficient are also not honest: no leakage was found and the bounded probe supports the stage set within its declared scope. inconclusive -- pending the oracle-ceiling campaign -- is the only accurate verdict.",
"required_verdict": "oracle_ceiling_positive",
"source_contract_hash": "6cdd695bf5b32036519f1ee2e787a44e4afbb045fc059f75a92a2d069f0ee7c5",
"source_contract_id": "compiler-reasoning-trace-v1",
"source_linear_issue": "SLM-249"
},
"gate_3_explicit_kc_stage_authorization": {
"actual_verdict": "none: the K x c latent-workspace model implementation (SLM-250 / LOT1-01) remains gated on a positive oracle-ceiling result, which requires running the plan defined here -- not authorized by this issue.",
"gate_name": "explicit_kc_stage_authorization",
"met": false,
"rationale": "SLM-249's own allowed_lot1_implementation field authorizes no LOT1 implementation; the oracle ceiling is inconclusive and K/c is a bounded probe recommendation only, not a data-derived production budget.",
"required_verdict": "explicit allowed_lot1_implementation entry with K/c/stage targets",
"source_contract_hash": "6cdd695bf5b32036519f1ee2e787a44e4afbb045fc059f75a92a2d069f0ee7c5",
"source_contract_id": "compiler-reasoning-trace-v1",
"source_linear_issue": "SLM-249"
},
"linear_issue": "SLM-251",
"lot1_01_disposition": "not_authorized",
"schema_version": "lot1_02_launch_gate/v1",
"verdict": "not_authorized",
"verdict_rationale": "LOT1-02 launch prerequisite(s) unmet: transfer_authorization (SLM-248), trace_oracle_ceiling (SLM-249), explicit_kc_stage_authorization (SLM-249 allowed_lot1_implementation). LOT1-01's disposition is 'not_authorized', so no faithful K x c model path or curriculum hooks exist to train; SLM-249's oracle ceiling is not positive and authorizes no LOT1 implementation. There is no treatment arm to run and no matched continued-explicit control to attribute against. Closing not_authorized in plan-only mode; no training, curriculum, or model code is added by this disposition.",
"version_stamp": {
"code_commit": "941730cbbe0ac6a956b87085b54ff874089e54e1",
"code_dirty": true,
"components": {
"harness.experiments": "v110"
},
"stamp_schema": "version_stamp/v1",
"stamped_at": "2026-07-25T15:59:29.204608+00:00"
}
}
32 changes: 32 additions & 0 deletions docs/design/iter-slm251-lot1-02-not-authorized-20260725.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# SLM-251 LOT1-02 — Launch-gate disposition (lot1-02-launch-gate-v1)

Verdict: **not_authorized**

LOT1-02 launch prerequisite(s) unmet: transfer_authorization (SLM-248), trace_oracle_ceiling (SLM-249), explicit_kc_stage_authorization (SLM-249 allowed_lot1_implementation). LOT1-01's disposition is 'not_authorized', so no faithful K x c model path or curriculum hooks exist to train; SLM-249's oracle ceiling is not positive and authorizes no LOT1 implementation. There is no treatment arm to run and no matched continued-explicit control to attribute against. Closing not_authorized in plan-only mode; no training, curriculum, or model code is added by this disposition.

LOT1-01 disposition: `not_authorized`

## Gate 1 — transfer authorization (SLM-248)

- Source contract: `lotus-openui-fidelity-contract-v1` (`801ce267b64f52b88e6e80fa091084c5f1a6628de60658d2d161db82f5117af2`)
- Required: authorize_bounded_implementation
- Actual verdict: `needs_target_trace_contract`
- Met: **False**

## Gate 2 — trace oracle ceiling (SLM-249)

- Source contract: `compiler-reasoning-trace-v1` (`6cdd695bf5b32036519f1ee2e787a44e4afbb045fc059f75a92a2d069f0ee7c5`)
- Required: oracle_ceiling_positive
- Actual verdict: `inconclusive`
- Met: **False**

## Gate 3 — explicit LOT1 implementation allowance (SLM-249)

- Source contract: `compiler-reasoning-trace-v1` (`6cdd695bf5b32036519f1ee2e787a44e4afbb045fc059f75a92a2d069f0ee7c5`)
- Required: explicit allowed_lot1_implementation entry with K/c/stage targets
- Actual verdict: `none: the K x c latent-workspace model implementation (SLM-250 / LOT1-01) remains gated on a positive oracle-ceiling result, which requires running the plan defined here -- not authorized by this issue.`
- Met: **False**

## Non-goals honored

No training campaign, no curriculum code, no Stage 0 checkpoint, no GPU dispatch, and no production default change. This disposition is itself the LOT1-02 deliverable while the launch prerequisites are unmet.
70 changes: 70 additions & 0 deletions scripts/evaluate_lot1_02_activation_gate.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
#!/usr/bin/env python3
"""Evaluate SLM-251 (LOT1-02)'s launch prerequisites in plan-only mode.

No training, curriculum, or model code is loaded or executed. This reads
the real committed upstream contract artifacts and emits the
``Lot102LaunchGateV1`` disposition.

Example:
python -m scripts.evaluate_lot1_02_activation_gate \
--fidelity-contract docs/design/lotus-openui-fidelity-contract-v1.json \
--trace-gate-contract docs/design/compiler-reasoning-trace-v1.json \
--out outputs/runs/slm251_launch_gate
"""

from __future__ import annotations

import argparse
import json
from pathlib import Path

from slm_training.harnesses.experiments.lot1_01_activation_gate import (
load_upstream_contract,
)
from slm_training.harnesses.experiments.lot1_02_activation_gate import (
evaluate_lot1_02_launch_gate,
render_markdown,
)


def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(
description="SLM-251 LOT1-02 launch-gate evaluator (plan-only, no training code)"
)
parser.add_argument(
"--fidelity-contract",
type=Path,
default=Path("docs/design/lotus-openui-fidelity-contract-v1.json"),
help="SLM-248 LotusOpenUIFidelityContractV1 JSON artifact",
)
parser.add_argument(
"--trace-gate-contract",
type=Path,
default=Path("docs/design/compiler-reasoning-trace-v1.json"),
help="SLM-249 CompilerReasoningTraceGateV1 JSON artifact",
)
parser.add_argument(
"--out",
type=Path,
default=Path("outputs/runs/slm251_launch_gate"),
)
args = parser.parse_args(argv)

fidelity_contract = load_upstream_contract(args.fidelity_contract)
trace_gate_contract = load_upstream_contract(args.trace_gate_contract)

contract = evaluate_lot1_02_launch_gate(fidelity_contract, trace_gate_contract)

args.out.mkdir(parents=True, exist_ok=True)
(args.out / "lot1_02_launch_gate.json").write_text(
json.dumps(contract.to_dict(), indent=2, sort_keys=True, default=str) + "\n",
encoding="utf-8",
)
markdown = render_markdown(contract)
(args.out / "lot1_02_launch_gate.md").write_text(markdown, encoding="utf-8")
print(markdown)
return 0


if __name__ == "__main__":
raise SystemExit(main())
216 changes: 216 additions & 0 deletions src/slm_training/harnesses/experiments/lot1_02_activation_gate.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,216 @@
"""SLM-251 LOT1-02 launch-gate evaluator (plan-only closeout).

LOT1-02 ("Run the explicit-to-latent curriculum against an update-matched
continued explicit-trace control") declares three upstream dependencies:

1. SLM-248 (LOT0-01) transfer authorization -- preregistered margins and a
``authorize_bounded_implementation`` verdict.
2. SLM-249 (LOT0-02) ``CompilerReasoningTraceGateV1`` -- K/c/stage support
and a positive oracle ceiling.
3. SLM-250 (LOT1-01) -- the faithful K x c looped-latent model path and
curriculum hooks.

The campaign cannot launch honestly unless every dependency is authorized:
there is no treatment arm without the LOT1-01 model path, and no causal
attribution without the SLM-249 oracle ceiling. This module derives the
LOT1-02 launch disposition from the same real, committed upstream contract
artifacts as the LOT1-01 gate (reusing
:mod:`slm_training.harnesses.experiments.lot1_01_activation_gate`) plus the
trace contract's own ``allowed_lot1_implementation`` field. It contains no
training, curriculum, or model code -- evaluating this gate honestly *is*
the LOT1-02 deliverable while the prerequisites are unmet.
"""

from __future__ import annotations

import hashlib
import json
from dataclasses import asdict, dataclass, field
from typing import Any

from slm_training.harness_core.versioning import build_version_stamp
from slm_training.harnesses.experiments.lot1_01_activation_gate import (
AUTHORIZED_WIRING_ONLY,
NOT_AUTHORIZED,
GateEvaluation,
evaluate_activation_gates,
)

__all__ = [
"GATE_SCHEMA_VERSION",
"LAUNCH_GATE_CONTRACT_ID",
"Lot102LaunchGateV1",
"evaluate_lot1_02_launch_gate",
"render_markdown",
]

GATE_SCHEMA_VERSION = "lot1_02_launch_gate/v1"
LAUNCH_GATE_CONTRACT_ID = "lot1-02-launch-gate-v1"


@dataclass(frozen=True)
class Lot102LaunchGateV1:
"""LOT1-02 launch disposition derived from real upstream contracts."""

schema_version: str = GATE_SCHEMA_VERSION
contract_id: str = LAUNCH_GATE_CONTRACT_ID
linear_issue: str = "SLM-251"
lot1_01_disposition: str = NOT_AUTHORIZED
gate_1_transfer_authorization: GateEvaluation | None = None
gate_2_trace_oracle_ceiling: GateEvaluation | None = None
gate_3_explicit_kc_stage_authorization: GateEvaluation | None = None
verdict: str = NOT_AUTHORIZED
verdict_rationale: str = ""
version_stamp: dict[str, Any] = field(default_factory=dict)

def to_dict(self) -> dict[str, Any]:
data = dict(asdict(self))
for key in (
"gate_1_transfer_authorization",
"gate_2_trace_oracle_ceiling",
"gate_3_explicit_kc_stage_authorization",
):
gate = getattr(self, key)
data[key] = gate.to_dict() if gate is not None else None
data["contract_hash"] = self.contract_hash()
return data

def contract_hash(self) -> str:
payload = {
"schema_version": self.schema_version,
"contract_id": self.contract_id,
"linear_issue": self.linear_issue,
"lot1_01_disposition": self.lot1_01_disposition,
"verdict": self.verdict,
}
return hashlib.sha256(
json.dumps(payload, sort_keys=True, default=str).encode("utf-8")
).hexdigest()


def _evaluate_kc_stage_authorization(trace_gate_contract: dict[str, Any]) -> GateEvaluation:
"""Check SLM-249's own explicit LOT1 implementation allowance.

``allowed_lot1_implementation`` is a list whose only honest negative
entry starts with ``none``. Any other entry is an explicit authorization
carrying K/c/stage targets.
"""
gate = trace_gate_contract.get("gate") or {}
allowed = gate.get("allowed_lot1_implementation") or []
explicit = [entry for entry in allowed if not str(entry).strip().lower().startswith("none")]
met = bool(explicit)
return GateEvaluation(
gate_name="explicit_kc_stage_authorization",
source_contract_id=trace_gate_contract.get("contract_id", ""),
source_contract_hash=trace_gate_contract.get("contract_hash", ""),
source_linear_issue=gate.get("linear_issue", "SLM-249"),
required_verdict="explicit allowed_lot1_implementation entry with K/c/stage targets",
actual_verdict="; ".join(str(e) for e in allowed) or "<missing>",
met=met,
rationale=(
"SLM-249 explicitly authorizes LOT1 implementation: " + "; ".join(explicit)
if met
else "SLM-249's own allowed_lot1_implementation field authorizes no LOT1 "
"implementation; the oracle ceiling is inconclusive and K/c is a bounded "
"probe recommendation only, not a data-derived production budget."
),
)


def evaluate_lot1_02_launch_gate(
fidelity_contract: dict[str, Any],
trace_gate_contract: dict[str, Any],
) -> Lot102LaunchGateV1:
"""Evaluate LOT1-02's launch prerequisites against real upstream contracts.

Reuses the LOT1-01 evaluator for the shared transfer-authorization and
oracle-ceiling gates, then adds LOT1-02's own requirement that SLM-249
explicitly authorize LOT1 implementation. The verdict is
``not_authorized`` unless every prerequisite is met; even a met gate
authorizes only wiring/fixture evidence, never a trained-quality claim.
"""
lot1_01 = evaluate_activation_gates(fidelity_contract, trace_gate_contract)
gate_3 = _evaluate_kc_stage_authorization(trace_gate_contract)

all_met = (
lot1_01.gate_1_fidelity_authorization.met
and lot1_01.gate_2_trace_oracle_ceiling.met
and gate_3.met
)
if all_met:
verdict = AUTHORIZED_WIRING_ONLY
rationale = (
"All LOT1-02 launch prerequisites are met. The curriculum "
"campaign may be wired in fixture/plan-only mode; this gate "
"authorizes no GPU run or semantic-quality claim by itself."
)
else:
verdict = NOT_AUTHORIZED
unmet = []
if not lot1_01.gate_1_fidelity_authorization.met:
unmet.append("transfer_authorization (SLM-248)")
if not lot1_01.gate_2_trace_oracle_ceiling.met:
unmet.append("trace_oracle_ceiling (SLM-249)")
if not gate_3.met:
unmet.append("explicit_kc_stage_authorization (SLM-249 allowed_lot1_implementation)")
rationale = (
"LOT1-02 launch prerequisite(s) unmet: " + ", ".join(unmet) + ". "
f"LOT1-01's disposition is '{lot1_01.verdict}', so no faithful "
"K x c model path or curriculum hooks exist to train; SLM-249's "
"oracle ceiling is not positive and authorizes no LOT1 "
"implementation. There is no treatment arm to run and no matched "
"continued-explicit control to attribute against. Closing "
"not_authorized in plan-only mode; no training, curriculum, or "
"model code is added by this disposition."
)

return Lot102LaunchGateV1(
lot1_01_disposition=lot1_01.verdict,
gate_1_transfer_authorization=lot1_01.gate_1_fidelity_authorization,
gate_2_trace_oracle_ceiling=lot1_01.gate_2_trace_oracle_ceiling,
gate_3_explicit_kc_stage_authorization=gate_3,
verdict=verdict,
verdict_rationale=rationale,
version_stamp=build_version_stamp("harness.experiments"),
)


def render_markdown(contract: Lot102LaunchGateV1) -> str:
lines = [
f"# SLM-251 LOT1-02 — Launch-gate disposition ({contract.contract_id})",
"",
f"Verdict: **{contract.verdict}**",
"",
contract.verdict_rationale,
"",
f"LOT1-01 disposition: `{contract.lot1_01_disposition}`",
"",
]
for title, gate in (
("Gate 1 — transfer authorization (SLM-248)", contract.gate_1_transfer_authorization),
("Gate 2 — trace oracle ceiling (SLM-249)", contract.gate_2_trace_oracle_ceiling),
(
"Gate 3 — explicit LOT1 implementation allowance (SLM-249)",
contract.gate_3_explicit_kc_stage_authorization,
),
):
if gate is None:
continue
lines += [
f"## {title}",
"",
f"- Source contract: `{gate.source_contract_id}` (`{gate.source_contract_hash}`)",
f"- Required: {gate.required_verdict}",
f"- Actual verdict: `{gate.actual_verdict}`",
f"- Met: **{gate.met}**",
"",
]
lines += [
"## Non-goals honored",
"",
"No training campaign, no curriculum code, no Stage 0 checkpoint, no "
"GPU dispatch, and no production default change. This disposition is "
"itself the LOT1-02 deliverable while the launch prerequisites are unmet.",
"",
]
return "\n".join(lines)
Loading
Loading