tla-export: a trace spec for trace validation (tlc_conform) - #54
Open
kiranandcode wants to merge 8 commits into
Open
kiranandcode wants to merge 8 commits into
kiranandcode wants to merge 8 commits into
Conversation
<State>_tla_trace.tla EXTENDS the export and follows one logged
implementation behaviour: a newline-delimited JSON log (a header naming
the module, then {step, params, state} per step) read with the Json
module. TraceNext takes the logged step, conjoins Next (so it only
narrows the model) and compares the observed fields in the successor;
codec operators generated from the export's type map decode params and
compare observations partially (a field left out is free). TraceEnabled
and TraceDiagnosis explain a divergence. Tested on the counter with a
conforming and a wrong-step log, and on collections and parameters.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
7 of 8 tasks
…s and header; a VerusSync param ranges over its Step hole - A last segment two steps share names neither: those steps are logged by their full paths (the report marks them short_name_shared), and the short name stops TLC with an Assert naming the candidates. - A logged parameter the step does not declare stops TLC (TraceParamsDeclared), as an unknown state field already did. - TraceInit asserts the header's module is the generated one. - A step parameter left out of the log ranges over the hole constant that bounds it in Next (VerusSync's Dom_Step_<t>_v<i>) before its type's domain or a Dom_<Type> hole, so it is enumerated in TraceEnabled. - README and the generated header: records inside Set elements, Map keys and parameters are decoded whole; the verus-tla shape has one step. - Tests: a malformed log (header, parameter, step, field), a shared short name, a VerusSync log with a parameter left out, a record in a Set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e step; a trace's header names its export A step that branches into helpers was not loggable by its own name (only its helpers were), since the steps came from the unassigned-variable report's leaves. The trace spec now takes every plain operator Next reaches through branches, intermediate ones and dispatchers included. TraceInit asserts the header's "export" is the exported module path, as exports whose states share a name share a module name. The collections test observes a Map's values partially and an enum whose variants share a field label. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sitions are steps, a conjoined one when it dispatches or is Next's only one
TraceNext, TraceEnabled and TraceDiagnosis conjoin Next first, so a helper
step reading a primed variable its caller assigned filters Next's successors
instead of stopping TLC. A loggable step is given the post state, so a guard
reached in a branch is never one; a transition Next conjoins is one when it
branches into steps or is the only transition Next conjoins (next = t_step).
Tests for both, for the verus-tla shape (mutex_tla.rs with an Option), and
for `{}` observing nothing of a Seq while `[]` is the empty Seq.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…an unobserved tag is free; depth 0 is a bad header state
- Each bound variable carries its domain (a closed quantifier bound, an
operator parameter, or a field of a matched value), and each call records
its arguments'. A step parameter the log leaves out ranges over the union
of what Next's calls pass it, resolved once the export is written: a
Step variant's field whatever the parameter's position (the hole of the
field's own type when the Step's domain split it off), or the bound Next
gives a quantifier (exists|n: u64| n < 3 gives 0..2). A hole is taken
only for a parameter of its type. The by-name Step-hole lookup is gone.
- The report's trace steps say whether TraceEnabled enumerates them
("enumerated"); a parameter of no domain says why when left out.
- An enum observed without its tag compares the named fields under the
model's variant; decoded whole (a parameter), an enum value without a tag
stops TLC with an Assert saying so.
- Depth 0 (no initial state) is documented as the header's observed state
being no initial state, in the trace spec, its .cfg and the README.
- Tests: variant fields passed reordered, a quantifier-bounded parameter and
an unenumerable one, a bad header state, an enum observed without a tag.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…bel no variant declares stops TLC; a unit struct decodes as the export prints it; the Dom_ holes must cover the log Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… a logged parameter is narrowed to its domain; a pass is a state behaviour, the step label counts through its effect Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… Seq observed by a key that is no index stops TLC Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the exporter half of
plans/tlc_conform.md:-V tla-exportnow also writes a trace spec beside the export, which TLC uses to check that a logged run of the implementation is a behaviour of the exported model.What was built
examples/tla/README.mdand in the generated module's header). The log is newline-delimited JSON: a header line naming the module and the export ({"module": "State_tla", "export": "counter"}, optionally with"state", the observed initial state), then one line per step,{"step": "t_inc", "params": {...}, "state": {...}}.stepnames a transition (a spec fn given the post state) thatNextreaches through its branches (anIF,match, disjunction orexists): a step, a helper a step branches into, or a dispatcher on aStepenum (next_step, VerusSync'snext_by). A transitionNextconjoins is a step when it branches into steps itself or is the only transitionNextconjoins (next = t_step). A guard on the pre state or on a value is never a step. It is given by its last segment or its full path.Nextitself is a step only when it reaches none of these (the verus-tla shape). When two steps share a last segment, only the full path is accepted: the report marks themshort_name_shared, and the short name stops TLC with anAssertnaming the candidates.paramsholds its parameters other than pre/post, by Rust name. A name the step does not declare stops TLC (TraceParamsDeclared), so a misspelling is not silently ignored.modulemust be the generated module and itsexportthe module path given to-V tla-export;TraceInitasserts both. Exports whose state types share a name share a module name (State_tla), so the module name alone does not identify the export.stateholds the observed fields after the step.tagged enums/Options, arrays forSeqand tuples, element arrays forSet,[k, v]pairs forMap.{}(the export's[tag |-> "unit"]). ASeqcan be observed by index ({"1": {...}}),{}observes nothing of it, and[]is the emptySeq. A record inside aSetelement, aMapkey or a parameter is decoded whole and must name every field (documented in the README and the generated header).<State>_tla_trace.tla(vir/src/tla.rs,Exporter::trace_spec), plus a.cfgskeleton. ItEXTENDSthe export and reads the log with theJsonmodule (ndJsonDeserialize).TraceNextconjoinsNextand then the logged step (aCASEover the transitionsNextreaches), so it only ever narrows the model, and a helper step reads the successorNextassigned.TraceEnabledandTraceDiagnosisuse the same order. It then compares the observed fields in the successor.TraceDec_T,TraceObs_T, allRECURSIVE). A parameter left out of the log ranges over whatNext's calls to the step pass it, the union over every call: the bound Next gives the quantifier binding the argument (exists|n: u64| n < 3gives0..2), or the field of a value matched against a constructor pattern, whatever the parameter's position (a VerusSync step'sDom_Step_<t>_v<i>, a hole of the field's own type). A hole is accepted only for a parameter of its type. Failing that, it ranges over its type's finite domain, never the export'sDom_<Type>hole, which holds only what a quantifier binds and not a value a call computes. A logged parameter is narrowed to its domain, so a value outside it is a step the model cannot take. The verus-tla shape has a single step,next.tagleaves the tag free: the fields it names are compared under whichever variant the model has. Decoded whole (a parameter, aSetelement or aMapkey), an enum value without a tag stops TLC with anAssertsaying so.TraceEnabledgives the model's enabled steps with their parameters.TraceDiagnosissays whether the logged step is enabled at all, and which observed fields no successor matches.tracesection lists the trace module, the index variable, the observable fields, and every loggable step with its parameters and domains, and whetherTraceEnabledenumerates it (enumerated).Deviations from the plan
The trace module is named
<Module>_trace(State_tla_trace.tla), not<State>Trace.tla. It follows the export's own<State>_tlanaming, and tlc_conform finds it by that rule.The verdict is not an invariant or postcondition. TLC ends without error either way. The log conforms when the search depth is its step count plus one (
TraceAccepted); otherwise the deepest index is the divergence. Depth 0 (TLC generates no initial state) means the header's observed state is not an initial state (documented in the trace spec, its.cfgand the README). This keeps nondeterminism in what the log leaves out from producing false divergences: a deadlock in one branch is not a verdict.Enabled steps come from a generated
TraceEnabledevaluated in the diverging state, rather than fromtlc_neighbours. Neighbours would list the trace spec's singleTraceNextaction, not the model's steps.TraceNextconjoinsNext, so the.cfg'sDom_constants must cover every value the log carries: a logged value outside a hole is a divergence (the README, the generated header and the.cfgcomment say so).A step parameter with no domain (no call passes it a bounded variable, and its type has no finite domain) must be logged. If it is not, the step
Asserts with that reason, and it is left out ofTraceEnabled; the report marks the step"enumerated": false.A pass means the observed state sequence is a behaviour of the model; the logged step's name and parameters count only through their effect on the observed state.
TraceNextrequires a successor bothNextand the logged step allow, not thatNexttook it by that step, so a step another ofNext's steps explains is accepted (the README gives an example; the generated header says so too). The plan's "constrainsNextto the logged step and parameters" is met for states, not for the step's identity. RestrictingNextto the logged step's call sites is left as a follow-up.Test plan
tla_export_trace_spec_follows_a_counter_log: the counter's trace spec parses (SANY).counter_trace_ok.ndjson(5 steps) is followed to depth 6.counter_trace_bad.ndjsonlogst_dblwhere the counter tookt_inc; TLC stops at step 4 witht_dblenabled andxunmatched, andTraceEnabled= {t_dbl, t_inc}.tla_export_trace_spec_decodes_collections_and_parameters:Seqof records (whole and partial by index),Set,Map,Option, and a tagged enum with a payload, plus au8parameter logged and aboolone left out (it ranges overBOOLEAN). A parameter the guard forbids stops the log at step 1.tla_export_trace_spec_stops_at_a_malformed_log: a header for another module, an undeclared parameter, an unknown step and an unknown state field each stop TLC with theirAssert.tla_export_trace_spec_names_a_shared_step_by_its_path:a::stepandb::stepare both reportedshort_name_shared. A log by full paths is followed, and the short namestepstops TLC.tla_export_trace_spec_follows_a_verussync_log:adder_sync'sadd(v: int)reports domainDom_Step_add_v0. A log leavingvout is followed. A step novexplains stops at depth 2, whereTraceEnabledenumeratesvover the hole.tla_export_trace_spec_decodes_a_record_in_a_set_whole: a partial record in the state and a whole one in aSetare followed, and a partialSetelement stops TLC.tla_export_trace_spec_logs_a_step_that_branches_into_helpers:t_recv(b)isif b { bump } else { reset }, andbumpis also a branch oft_other. The steps arebump,reset,t_otherandt_recv. A log namingt_recv(withblogged, then left out),t_otherandbumpis followed to its end, andt_recvwithbfalse where the state showsbumpstops at step 1.Mapwith record values partially (keys whole), and an enum whose variantsLo(u8)andHi(u8, bool)share the labelv0. A wrong value field, a wrong key, the wrong tag or a wrong shared-label value each stops the log at that step."stat","param", a header's"State") stops TLC (TraceKeys), as does a step line without"step", so a misspelling never observes nothing. ASeqobserved by a key that is no index ({"x1": ...}) stops TLC; an index past the end only diverges.tla_export_trace_spec_takes_next_before_the_logged_step: a helperchkreadingy' = x', which its caller assigns, is followed when logged.next = t_stepmakest_steploggable, and the conjoinedframeis not a step.tla_export_trace_spec_logs_no_guard: guards in a branch (ready(pre),small(v)) are not steps. Withnext = ready(pre) && t_only(pre, post), the only step ist_only.tla_export_trace_spec_follows_a_verus_tla_log:mutex_tla.rs, whose only step isnext, observed throughOptionholder. A wrong thread or count stops the log."s": {}is followed and"s": []stops on a non-emptySeq.tla_export_trace_spec_takes_a_parameter_domain_from_the_match:Step::t_set(f, n)passes its fields tot_set(n, f)swapped.n: intgetsDom_Step_t_set_v1andf: boolthev0field (the booleans). Before this,fgot the int hole. A log leaving each out is followed, and annoutside the hole stops the log.tla_export_trace_spec_takes_a_parameter_domain_from_a_quantifier:exists|n: u64| n < 3 && t_go(n)givesnthe domain0..2, andTraceEnabledlists t_go's three.t_jump(pre.x + 5)has no domain, is"enumerated": false, and leavingtoout stops TLC with the reason.tla_export_trace_spec_diverges_at_a_bad_header_state: a header state that is not an Init state ends at depth 0 with 0 states generated.tla_export_trace_spec_leaves_an_unobserved_tag_free:{"v0": 2}without a tag is followed. A field the model's variant lacks, or a wrong value, stops the log. A tagless enum parameter stops TLC with the tagAssert.tla_export_trace_spec_binds_no_name_of_the_export: a state with fieldsv,k,e,j,p,r(aSeq, aMap, anOption). Every name the trace module binds is fresh against the export's (TraceParam(e_2, k_2, Dec(_), D)), SANY accepts it, a log is followed and a wrong value diverges.tla_export_trace_spec_decodes_a_unit_struct: a struct without fields is observed from{}or{"tag": "unit"}and decoded whole as a parameter. A field named in it stops TLC.Rec has no field aa,Mode has no field v2), as a misspelled state field does. A label another variant declares still only fails to match.tla_export_trace_spec_takes_no_hole_for_a_computed_argument:t_jump(pre, post, pre.x + 5)besideexists|n: int| t_set(pre, post, n)givestono domain (notDom_int) and"enumerated": false.to5 logged, outsideDom_int = {0, 1}, is followed; left out, it stops TLC with the reason. A loggedn7 outsideDom_intstops at step 1.t_jumpwithto0, whicht_set(0)explains, is followed (the label limitation).tla_exportsuite withTLA2TOOLS_JARset: 79 passed.cargo fmt -- --check,cargo clippy --all-targets -- -D warnings(insource/).Companion PRs: verus-tools-mcp
tlc_conform(runs this trace spec) and toyDB (trace_step!emission, and all 58 node goldenscripts conforming to the hand-writtentla/Raft.tla).🤖 Generated with Claude Code