AxoMEME 2.0: browser inference, verified against the reference implementation - #153
Merged
Conversation
DM3's own tree inference emits negative branch lengths and at least one consumer crashes on them rather than degrading. NJ.bf:214-220 computes the three-taxon closed form (d01 + d02 - d12) / 2 with no Max(0, ...), so any triplet violating the triangle inequality yields a negative. NJ.bf:99 returns a saturation sentinel of 1000 for a saturated pair, which propagates through that same subtraction to roughly -499.9. Measured against AxoMEME 2.0's inference path, which computes log((node_count + 1.0) / (dist + 0.1)): any patristic distance <= -0.1 is a log of a negative and an uncaught ValueError. Running the ML team's own calculate_patristic_distances over 283 real DM3 trees raised that error on 13 of them (4.6%), worst case -114.9. This module reports and does not rewrite. Clamping changes branch lengths, branch lengths are the input a model's distances are built from, and quietly altering them would be a fabrication. The caller decides whether to refuse the run or to clamp with the user told. hasCrashingBranchLength is documented and tested as necessary but NOT sufficient: the threshold is on patristic path sums, which can be more negative than any single branch on the path.
These guards asserted "DM3 ships no ML runtime, anywhere". For as long as the MEME hit-likelihood gate was the only model in the repository, that was the same statement as the one actually worth defending. It is not any more: AxoMEME 2.0 is a 3.78 MB transformer whose graph is a real neural network, and no 40-line walker is going to execute it, so onnxruntime-web is now a legitimate dependency and the old assertion would fail for a legitimate reason. The fix someone reaches for when a guard fails legitimately is deleting the guard. What was being defended was never "no runtime exists". It was that the gate costs almost nothing and is reachable from every method, so it must not drag a runtime behind it -- the first version of that feature downloaded 13.5 MB of ONNX Runtime WASM for all fifteen methods and rendered nothing for fourteen of them. So the vitest guard now walks the gate's own import closure and proves no ML runtime is in it, which is strictly stronger than the per-file grep it replaces (that version listed four files by hand and would not have noticed a fifth). package.json is checked as an allowlist rather than a ban. Writing it surfaced a real hole: the first version did not fire when `import 'onnxruntime-web';` was injected into a reachable module, because a side-effect import has no `from` clause -- and that is exactly the shape a WASM runtime arrives in, since you import it for registration rather than for a binding. All three specifier forms are now matched, and both halves are proven to fail. The e2e byte guards do not weaken at all. They are scoped to two flows, select-FEL and select-MEME, and neither is AxoMEME; only their stated rationale was false. Their invariant was always per-flow: a method must not pay for a model it does not render. That matters more now, not less, because a stray static import resolves instead of erroring.
AxoMEME 2.0 predicts MEME's per-site statistics in seconds instead of the hours a full run takes. The exported graph is the MODEL ONLY: it takes five already-computed tensors and returns five, and the preprocessing that produces those tensors is not in the graph. torch.linalg.eigh has no ONNX lowering -- verified directly, the identical module exports with eigh removed and fails with it present -- so mds_coords is a graph input and the whole pipeline has to be rebuilt in JS. Parity is proven against the ML team's own functions, extracted verbatim from predict_regression_nexus.py and exec'd rather than reimplemented, over 270 real DataMonkey trees: patristic distances 10,405,350 cells, worst |delta| 0.000e+0 (exact) MDS coordinates 270/270 within 1e-5 relative, worst 3.35e-7 The MDS result was not expected to be reachable. Two things made it work. Their code already canonicalises eigenvector signs, which removes half the ambiguity for free. And the reference squares FLOAT32 distances, because dist_tensor is a torch.float32 tensor -- feeding it full float64 changes components 2 and 3 by 40-99% on real trees, since squared distances reach ~1e6 while the fourth eigenvalue can be ~1e-1. One Math.fround took the failures from 5 to 1 and the worst error from 2.03e-1 to 1.665e-4. The tokenizer deliberately implements the TRAINING vocabulary, not the one in the handoff's inference driver. Those disagree on 63 of 64 codons: the driver defines its own 60-codon alphabetical CODON_LIST, then redefines get_codon_token without redefining CODON_TO_IDX, and imports only three non-tokenizer names from the training module. TTA plus the three stops are absent from its list entirely and collapse to "unknown". A model trained on TCAG-64 must be served TCAG-64. Pinned from both sides so we fail if we drift from training and if someone aligns us to the driver. Cost discipline, because AxoMEME is one method of fifteen and the runtime is ~13 MB: onnxruntime-web is loaded by dynamic import inside loadSession and never at module scope, and the model is fetched from static/ rather than bundled. Verified absent from the client bundle. The artifact's sha256 is pinned and checked, because the contract's conclusions -- eval-mode export, lrt already decoded, rate heads in log1p space -- were read off that specific graph and a swapped model would make all of them wrong while everything kept running. Every site batches into one graph run, since dist_matrix, mds_coords and padding_mask are per-alignment; the reference driver loops one forward pass per codon. Other behaviours reproduced deliberately rather than improved: Max-PD seeds at alignment index 0 and can select duplicates, duplicate tip names resolve to the last leaf in preorder, MDS runs on the padded matrix so coordinates depend on max_species, and invariant sites are zeroed before the model is consulted. Findings that need an ML-side decision are written up separately.
This joins the separately-verified stages -- newick parse, patristic distances, Max-PD selection, tokenisation and MDS -- into the bundle the ONNX graph accepts, which makes the whole path runnable end to end. Two decisions in here are not obvious. MDS runs on the PADDED matrix and is then sliced, while the GRAPH is fed only the real species. Those are different questions with different answers. The padded zeros take part in MDS's double-centring, so coordinates genuinely depend on max_species and computing them on the real N gives different numbers -- that has to be reproduced. But feeding 512 slots to the graph is measurably equivalent to feeding N real species with the remainder masked: worst delta 1.192e-07, one float32 ulp, which is what masking is for. The difference is 462 MB versus 2.3 MB for dist_matrix on a 441-site alignment, because ONNX needs materialised data where torch used an expand view, and attention is quadratic in N on top of that. Species order comes from the TREE, not the alignment, with the reference sequence moved to index 0. Order matters twice: it fixes the distance matrix rows and therefore MDS, and index 0 seeds the Max-PD traversal, so it decides which taxa survive the cap. The reference sequence is chosen by a heuristic that looks for 'hg' / 'hg38' / 'human' before falling back to the first sequence -- a TOGA-mammal artifact, so on viral traffic the fallback is the real behaviour. Sites are batched rather than assembled whole, sized against the dist_matrix budget, because DataMonkey accepts uploads up to 12,000 codons. Verified end to end on the bundled CD2 demo (10 taxa, 17 codons): 726 ms to prepare, 88 ms of inference, 13 of 17 sites variable, and the top site reports dN+ 4.47 against dS 0.025 with p_pos 0.9999 while calling Neutral -- the LRT of 1.86 does not clear the 3.12 gate, which is the right answer at that alignment size.
Registers AxoMEME in the method registry and dispatches it to a third runner. The project's checklist for adding an analysis assumes a HyPhy method at every step -- a hyphy command, an outputSuffix naming the HyPhy JSON, CLI argument mapping, a backend socket event, a hyphy-eye visualiser keyed to that JSON -- and AxoMEME has none of them. So methodConfig carries `command: null` and `outputSuffix: null` rather than invented values, and anything dispatching on them fails loudly instead of running the wrong binary. AxomemeAnalysisRunner implements the BaseAnalysisRunner lifecycle and nothing else, and is dispatched in AnalyzeTab BEFORE the backend/WASM split because it belongs to neither. The executionMode toggle is ignored rather than routing a browser-only method through a socket with no handler for it. Two runtime-loading bugs, both found by running it in a browser and neither visible to a unit test: onnxruntime-web does not bundle its WASM binary. With no wasmPaths it resolves to a jsDelivr CDN, which breaks the project's core constraint that the site be servable with no other domains involved, and fails outright offline. The runtime reports "no available backend found", which reads like a broken model rather than a missing asset. scripts/copy-ort-wasm.mjs now vendors the binary into static/ort/ at build time -- copied rather than committed, because it is 12.9 MB of third-party output already pinned by package.json, and a committed copy would go stale on upgrade with nothing noticing. The DEFAULT package entry then asks for the JSEP (WebGPU) binary, a different 26.8 MB file, and fails identically even with the CPU binary present and served correctly. Importing onnxruntime-web/wasm fixes it and drops WebGPU/WebNN backends this feature never uses. e2e/19-axomeme.spec.js asserts WHICH URLS THE PAGE REQUESTS, not just that a number came out, since both bugs were in the fetch rather than in any computation. It also re-asserts the cost invariant from the other direction: a non-AxoMEME method must download none of the 17 MB. Advanced options are deliberately near-empty. There are no branch sets to select (the model consumes the whole tree as a distance matrix), no rate variation switch, and no genetic code choice -- the code table is baked into the model's tokenizer. Calling mode is the one real control, because it is a threshold applied after inference. Verified end to end in Chrome on the CD2 demo: method appears, runs, and reports completion.
Two UI affordances implied capabilities AxoMEME does not have, and both are now
driven off a `browserOnly` flag on the method rather than a name check:
- EXECUTION MODE offered "Backend Server". There is no server-side AxoMEME;
the dispatch ignores the toggle and always runs in-browser, so choosing the
server silently did the local thing. Replaced with a sentence saying it runs
in your browser and there is nothing to choose.
- GENETIC CODE was selectable but unreachable. The model's tokenizer bakes in
the universal table, so a user's choice could not affect the result. It
defaulted to Universal, which is right, which is what made it dangerous --
changing it would have done nothing and said nothing.
The results view leads with the framing rather than burying it. AxoMEME
estimates what MEME would report; a researcher reading "Tier 1 (High)" next to a
site is one step from writing it up, so the heading says "predictions" and the
first sentence says MEME was not run.
Invariant sites render "not scored — no amino-acid variation" instead of 0.000.
The reference zeroes those before consulting the model, so printing zeros beside
real zeros would conflate "the model said nothing" with "we never asked".
A warnings block reports what the model was actually given -- sequences dropped
for not being in the tree, taxa cut by the 512 cap, repeated taxon slots, and
negative branch lengths. None of that is visible in the numbers themselves and
all of it changes them.
Rows are capped at 500 with an explicit control to show more, because a
12,000-codon alignment is ~100k DOM nodes.
e2e covers the render and the suppressed controls, and checks that the controls
come BACK for a method that has them, so the flag cannot silently disable them
everywhere.
…eline AxoMEME failed on the bundled large.nex demo. Its NJ-inferred tree produces a patristic sum of -1.04e-5 -- zero with rounding error on it -- and the input check rejected the entire run for it. Neither obvious response is right. Refusing the tree rejects an ordinary alignment for a rounding artifact, and DM3's own NJ emits these routinely: NJ.bf:214-220 computes (d01 + d02 - d12)/2 with no Max(0, ...), so any triangle-inequality violation yields one. Passing it through unclamped feeds the model something training never showed it. The handoff README settles it: "This build's training pipeline clamps distances >= 0." So clamping at inference matches what the weights were fitted against. It is the reference's INFERENCE path that omits the clamp, and that omission is why it throws on 13 of 283 real DM3 trees. This is the one place in the port that deliberately alters a value rather than passing it through, so it is not done silently: the count and the most negative magnitude are recorded, and the results page warns when the worst distance is below -0.001, which separates a genuinely broken tree from NJ float noise. e2e now runs AxoMEME on every bundled demo, so a tightened validator cannot quietly break the datasets a user is most likely to click.
Compares the whole JS pipeline -- parse, patristic distances, Max-PD,
tokenisation, MDS, inference, postprocessing -- against
predict_regression_nexus.py on real DataMonkey MEME submissions, per site.
Result on 12 alignments and 2,195 sites: worst |delta| 5.77e-6, Pearson
r = 1.000000, Spearman rho = 1.000000, zero variable-site classification
mismatches.
Two patches must be applied to the reference before it can serve as ground
truth, and both are its own bugs rather than concessions to make the numbers
agree:
- The codon tokenizer disagrees with the model's training vocabulary on 63 of
64 codons.
- model.train() at line 1364 leaves DROPOUT ON, so the script is
non-deterministic: two runs of the same input differ by up to 0.41 in
predicted_lrt, which is large next to the 3.12 and 4.45 calling gates.
Without disabling it the same comparison reports r = 0.925 -- the reference
is sampling a different network on every run.
Both sides are fed FASTA dumped by the REFERENCE's own parser, so a
disagreement is a pipeline disagreement rather than two parsers reading a file
differently. That is also necessary here: the corpus alignments are
HyPhy-generated NEXUS using NOLABELS, which DM3's own parseNexus cannot read.
This measures PORT CORRECTNESS, not model quality. Most of the corpus is
AxoMEME fine-tuning data, so accuracy numbers from it would be in-sample.
…ntile
Measured across 12 real DataMonkey MEME submissions and 662 variable sites, the
model's predicted LRT does not reach the thresholds the reference driver
compares it against:
highest predicted LRT anywhere 3.902
sites clearing the Tier 2 gate 3.12 1 of 662
sites clearing the Tier 1 gate 4.45 0 of 662
On an alignment where MEME itself reports 17 sites at p <= 0.05, the model's
maximum was 2.484. Under the reference's `pvalue` default this feature ships
reporting nothing on real data.
That is not a threshold to tune. It reflects what the model is: its authors
report SPEARMAN RANK CORRELATION, not calibration, and rank correlation can be
good while the absolute scale is off by a factor of two. So the default is now
`percentile`, which asks the question the model can answer -- which sites in
THIS alignment look most interesting -- rather than one it cannot. The gates
themselves are unchanged, so selecting `pvalue` still reproduces the reference
exactly.
The presentation follows the same decision:
- Tier labels state the rule that produced them ("Top 2%", "Z >= 2.5",
"LRT >= 4.45") instead of "High" and "Medium", which imply a calibrated
confidence the model does not have.
- The results table leads with RANK, and the model's output is labelled Score
rather than LRT. Calling it an LRT invites a comparison against chi-square
thresholds that it clears once in 662 sites.
- The header says the score is not a p-value and orders sites within this
alignment only.
Also: stop warning about float-noise negative branch lengths. inspectBranchLengths
reports ANY negative, and DM3's NJ routinely emits values around -1e-5 -- the
bundled large.nex has two. Those are clamped to match the model's training
pipeline, and warning about them trains users to ignore the warning box. A
meaningfully negative tree still surfaces, via the filter and via the
most-negative-distance line.
Moves the plots and the per-site table into hyphy-scope, where every consumer of AxoMEME output can reach them, and leaves behind only what is specific to THIS application: what DataMonkey did to the user's data before the model saw it. That split is not bookkeeping. DM3's own neighbour-joining inference emits negative branch lengths, sequences absent from the tree are dropped, and taxa past the 512 cap are subsampled -- none of it visible in a per-site score, all of it changing that score. hyphy-scope has no business knowing about NJ.bf, and a reader has every business knowing their tree was altered. The library component brings four plots (ranked sites, score by site, score distribution, dS vs dN+), a sortable paginated table, and deliberately no significance threshold line -- see its commit for why. vite.config.ts gains a note rather than a change: hyphy-scope must NOT be added to optimizeDeps.exclude even when linked. It pulls d3, phylotree, circos and @observablehq/plot, and excluding it makes the dev server transform all of that per request -- enough to stall the page, which presents as an analysis that never finishes rather than as a slow import. Pick up library changes by rebuilding hyphy-scope and restarting dev.
The MODEL is still under active development, not the integration around it, and that is the distinction the badge is for: the code is settled, the numbers may move between releases. It appears in both places a user meets the method, because they are reached independently — someone opening a shared results view never sees the selector, and someone browsing methods never reaches the results. Follows the existing METHOD_INFO pattern (a flag beside `supported`, rendered next to the Coming Soon badge) rather than a special case, so a second beta method needs one line. Coloured in the app's accent rather than the amber used for Coming Soon, so "beta" does not read as a warning. The library component takes `beta` as a prop rather than hard-coding it; DataMonkey knows this model's status, hyphy-scope should not assume it. e2e asserts the badge in both places AND its absence on FEL, so it cannot start decorating every method.
The AxoMEME visualization lives in hyphy-scope, which was consumed via `npm link` while both sides were developed together. Published 1.9.1 predates the component, so until now this branch only built on a machine with the symlink in place. 1.10.0 is published with AxomemeVisualization; this pins it and drops the link. Verified with the symlink removed and the Vite cache cleared, so nothing stale could mask a failure: clean build, 514 unit tests, 9 AxoMEME e2e including every bundled demo, and the MEME byte guards still passing. The note in vite.config.ts stays. It explains why hyphy-scope must not be added to optimizeDeps.exclude — it pulls d3, phylotree and circos, and excluding it stalls the dev server badly enough to look like a hung analysis — and that applies to anyone who links it again for local development.
The team runs e2e locally before pushing, which made the CI copy redundant on every PR and expensive: 13 minutes, of which 84% was the tests themselves. That cost was not fixable the easy ways. The setup — install, browsers, build — is only 2 minutes, so caching or a faster runner buys nothing. Playwright runs with `workers: 1` in CI deliberately, because parallel workers contend for the single dev server and reintroduce the flake that setting was chosen to stop, so raising it trades minutes for intermittent red. Sharding across jobs would work and is more machinery than a locally-run suite justifies. The workflow is kept rather than deleted, and gains a suite selector, so it can be started from the Actions tab or `gh workflow run e2e.yml`. The header says when that is worth doing: before a release, and on any change to what the app DOWNLOADS. The byte-budget guards exist because a previous version shipped 13.5 MB of ML runtime to every method and rendered nothing while an "is the element absent?" assertion passed throughout — the class of regression least likely to be checked from memory. `wasm-e2e` was previously gated on `github.ref == 'refs/heads/main'`. With no push trigger left that condition could never fire, so the selector replaces it rather than leaving a job that silently never runs. main is not a protected branch, so no required status check is affected.
Nine findings, each verified before being fixed.
STATE
Execution mode was clobbered permanently. `browserOnly` assigned
`executionMode = 'local'` with no restore, so choosing Backend Server for FEL,
browsing to AxoMEME, and returning to FEL silently ran the next analysis through
WASM — for a large dataset, the exact case the server exists for. The previous
choice is now saved and given back.
The runner reported `executionMode: 'browser'`, a third value no consumer knows.
It is persisted to IndexedDB, and `cleanupInterruptedAnalyses` reaps stale runs
by testing `=== 'wasm'`, so an AxoMEME run interrupted during the ~600 ms-2 s
synchronous MDS was stuck at `running` forever, and the viewer showed
"Execution: Unknown". It now reports 'wasm', which is what it is.
CORRECTNESS
The calling mode was resolved with OPPOSITE precedence for scoring and for
reporting, so a config carrying both keys could score with percentile gates while
the footer told the reader the calls meant "Z >= 2.5". One source of truth now.
`push(...out[key])` overflowed V8's argument limit. `batchSizeFor` scales as
1/N^2, so a ten-taxon alignment batches 167,772 sites — measured to throw
"Maximum call stack size exceeded" at ~90% progress. Copied element-wise instead.
Sequences were re-resolved with `names.indexOf(name)`, which returns the FIRST
match, while `orderSpecies` keeps the LAST. With duplicate FASTA headers the
variability flags came from a different sequence than the model was tokenised
from. `prepareAlignment` now exposes indices.
`loadSession` memoised on a hand-listed set of options that omitted
`verifyHash`, so one `loadSession({verifyHash: false})` could cache a session
never checked against the pinned hash — which the runner then stamped as
verified. The guard is inverted: memoise only when no options are passed.
HONESTY
`matchedFromTree` was plumbed into the summary and read by nothing. When no
sequence name matches a tree label — `seq1` vs `seq1_2009` — every distance and
every embedding coordinate is zero, yet every other counter looks healthy, so the
caveat panel stayed hidden and the user saw a full ranked table built without a
phylogeny. It is now the first caveat listed.
The tree guard checked only that the string was non-empty, while the comment
directly above it said branch lengths were mandatory. A topology-only tree passed
and produced confident numbers from an all-zero distance matrix. It now uses
`treeHasBranchLengths`, and says why in the error.
TESTS
The test named "memoises, so a second alignment does not re-download 17 MB"
asserted the opposite — it passed options, which bypass the memo, so the
production path had no coverage. Rewritten to drive the real path, plus a second
test pinning that options bypass it.
CANCELLATION. The runner never registered in `activeAnalyses`, which is how the base class cancels, and its scoring loop read no abort flag. Cancelling an AxoMEME run left it scoring, calling updateProgress against a row the user had cancelled, and finally writing `completed` over the cancelled record with a success toast. It now records cancellations itself and checks between batches — the only point the loop yields — returning without touching the record. DUPLICATE TOASTS. BaseAnalysisRunner.completeAnalysis already reports both outcomes, so the branch's own success toast stacked a second one, and letting the runner's rethrow reach the outer catch stacked a second error toast. Neither other branch rethrows on the normal path, which is why only AxoMEME double-reported. STALE ESTIMATE. MemeHitLikelihood kept rendering the previous alignment's band, and its "about 93%" sentence, while a new estimate was in flight. `seq` stops a superseded RESPONSE overwriting a newer one but does not clear what is on screen. `computing` was assigned in three places and read in none; gating `shown` and `idle` on it lets the pending branch fire. REVALIDATION. dist_matrix, mds_coords and padding_mask are per-alignment, emitted as b memcpy'd copies of one array, and every batch rescanned up to the whole 64 MB budget three times to re-check the same block. Validated once instead. RUNOUTLOOK. The panel lived inside the execution-mode branch, so making AxoMEME browser-only removed it as a side effect of nesting. It now has its own condition, browser-only methods are excluded deliberately rather than accidentally — RunOutlook's runtime model has nothing to say about a fixed neural model — and their real cost is stated in the note instead. `outlookTree` is no longer computed for a panel that is not mounted. LAYOUT RESERVE. RunOutlook's placeholder height was two hand-measured pixel constants duplicated from MemeHitLikelihood.css, with a comment telling the next person to re-measure whenever the copy changed, and recording that they were already 5.5px stale. The reserve is now declared once in app.css — which is always loaded, unlike the lazily-loaded chunk the placeholder is drawn before — and the placeholder derives its height with calc(). The derived values are identical to the measured ones (121px / 103.5px), and an e2e now asserts the property those numbers existed to protect: the Run button does not move when the estimate resolves. Measured shift is 0.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds AxoMEME 2.0 as a browser-only analysis method: a 3.78 MB neural model that ranks the sites of an
alignment by how MEME-like their selection signal looks, in seconds rather than the hours a real MEME
run takes.
The shape of the problem
The ML team's ONNX export is the model only — five already-computed tensors in, five out. Every
preprocessing stage that produces those tensors is outside the graph, because
torch.linalg.eighhasno ONNX lowering (verified directly: the identical module exports with eigh removed and fails with it
present). So
mds_coordsis a graph input, and newick parsing, patristic distances, Max-PD taxonselection, tokenisation and MDS all had to be rebuilt in JS.
Verification
Parity is measured against the ML team's own functions — extracted verbatim from their driver and
exec'd, not reimplemented — over real DataMonkey submissions:0.000e+03.35e-7(float32's own resolution is 1.19e-7)Reproducible via
npm run verify:axomeme-reference+npm run verify:axomeme-preprocessing.The MDS result was not expected to be reachable. Two things made it work: their code already
canonicalises eigenvector signs, and the reference squares float32 distances (its
dist_tensorisa float32 tensor). Feeding float64 changes components 2–3 by 40–99% on real trees, because squared
distances reach ~1e6 while the fourth eigenvalue can be ~1e-1. One
Math.froundtook the failuresfrom 5 to 1.
Decisions worth review
It is treated as a ranker, and defaults to
percentilecalling. Across 12 real submissions and662 variable sites, the model's predicted LRT reached the p ≤ 0.10 gate once and the p ≤ 0.05 gate
never — on an alignment where MEME reports 17 significant sites, its maximum was 2.484. Under the
reference's
pvaluedefault the feature reports nothing on real data. Its authors report Spearmanrank correlation, not calibration. Gates are unchanged, so
pvaluestill reproduces the reference.MDS runs on the padded matrix; the graph is fed only real species. Different questions, different
answers. Padding participates in the double-centring, so coordinates depend on
max_speciesand thatmust be reproduced. But feeding N real species is measurably equivalent to feeding 512 masked — one
float32 ulp — and worth 462 MB → 2.3 MB for
dist_matrixon a 441-site alignment.We implement the training tokenizer, not the shipped driver's. They disagree on 63 of 64 codons
(details below). Pinned from both sides so we fail if we drift either way.
Cost discipline.
onnxruntime-webis dynamically imported and the model served fromstatic/, sothe other fourteen methods download none of it. Asserted by e2e, not assumed. The WASM runtime is
copied from
node_modulesat build time rather than committed, and served locally — with nowasmPathsset, onnxruntime resolves to a jsDelivr CDN, which breaks the project's self-containedrequirement and fails offline.
Findings for the ML team (in the corpus notes, not blocking)
alphabetical vocabulary, then redefines
get_codon_tokenwithout redefiningCODON_TO_IDX.TTAand all three stops collapse to "unknown".
model.train()leaves dropout on, so two runs of the sameinput differ by up to 0.41 in
predicted_lrt, against calling gates of 3.12 and 4.45. Withoutpatching it the parity comparison above reports r = 0.925 rather than 1.000000.
trees. Their training pipeline clamps, which is why we do too.
Dependency (resolved)
The visualization lives in hyphy-scope. It was consumed via
npm linkwhile both sides weredeveloped together; stevenweaver/hyphy-scope#14 has since merged and 1.10.0 is published, so this
branch now pins
hyphy-scope@^1.10.0from the registry and the link is gone.Verified with the symlink removed and the Vite cache cleared, so nothing stale could mask a failure.
Testing
514 unit tests, 9 AxoMEME e2e including every bundled demo dataset and the Beta badges, and the MEME
byte guards still pass. All re-run against the published package rather than the link. Verified end
to end in Chrome.