Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions packages/libs/SubstrateML/docs/AGENT_MAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ sources:
- path: Sources/SubstrateML/InformationTheory.swift
blob: f7ba2a27904ceb19eef0fb5b06d1c5aad8b3894b
- path: Sources/SubstrateML/JacobiSVD.swift
blob: fcd924ff0a8409f224b56e7229f59659a6b5f51e
blob: ca87cf2e57a5469b950ab46f4951ca3a05c1c864
- path: Sources/SubstrateML/LatticeDistance.swift
blob: a7b96a376dfba0c815e4dc91962da1ccaeb2ac1d
- path: Sources/SubstrateML/LLMCalibrationCurve.swift
Expand Down Expand Up @@ -133,7 +133,7 @@ ENTRY POINTS (no single facade; per-family):
- NMFAlternatingLeastSquares.swift:45 `struct NMFFactorization` (W m×k, H k×n, rank, iterations, finalError RMS); :81 `factorize(V:rank:maxIterations:100:tolerance:1e-4:seed:0xDEADBEEFCAFEBABE:estate:ts:)`: Lee-Seung multiplicative updates, eps 1e-9, init = SplitMix64 16-bit draws /0xFFFF, preconditions rectangular/finite/non-negative; flat-buffer unsafe-pointer hot loops (loop nest + reduction order preserved EXACTLY; conformance CRC 0x300bf633; ~100× perf vs bounds-checked); emits `nmf.factor` = finalError; :220 `reconstructionError(V:W:H:)` public RMS; internal flat/nested matMul helpers :247/:288/:309/:329/:353/:370/:388
- NMFDoubleFrobeniusSquared.swift:54 `struct NMFDoubleFrobeniusSquaredFactorization` (:93 `loadings(forRow:)`); :113 `enum NMFDoubleFrobeniusSquared`: :116 `defaultMaxIterations = 100`, :122 `defaultTolerance = 1e-6`, eps 1e-9; :153 `factorize(o:rows:cols:rank:seed:0xC0FFEE_BABE_BEEF:...)`: f64 scalar, RAW Frobenius² convergence (not RMS), init floored at 1e-3; :302 `struct NMFDoubleFrobeniusSquaredRNG` (SplitMix64 core, 53-bit output, floor 1e-3). PRODUCTION GATE: no consumer may wire to this pending substrate_math_performance benchmark (ruling 2026-06-13); still conformance-gated vs Rust (nmf_double_frobenius_squared.json)
- AnomalyDetection.swift:26 `enum AnomalyDetection`; :31 `zScore(value:mean:stddev:)` (stddev ≤ 0 → 0); :51 `rollingZScore(window:current:estate:ts:)`: emits `anomaly.flag`; :90 `modifiedZScore(value:median:mad:)`: 0.6745 consistency constant, mad ≤ 0 → 0; :110 `rollingModifiedZScore(...)`: IN-PLACE sort (5× perf, ~400KB saved at 100k; CRC 0x6c6fda4d; must match Rust sort_by partial_cmp for non-NaN); :152 `isAnomalous(zScore:threshold: 3.0)`
- JacobiSVD.swift:74 `struct SVDResult` (U, singularValues non-increasing, Vt, rank); :129 `decompose(A:rank:sweeps:) -> SVDResult`: one-sided cyclic Jacobi; sweeps default 30 PINNED (fixed count, not convergence-tested; MUST match across ports); eps 1e-9 skip/zero cutoff; scalar Float32 only, NO SIMD/FMA/Accelerate; sign convention: largest-|entry| of each left vector forced positive; preconditions m ≥ n, rectangular, non-empty; unsafe-buffer sweep loops (perf, indices loop-derived); :368 `jacobiCS(alpha:beta:gamma:)`: pinned expression tree (ζ,t,c,s), DO NOT refactor
- JacobiSVD.swift:74 `struct SVDResult` (U, singularValues non-increasing, Vt, rank); :129 `decompose(A:rank:sweeps:) -> SVDResult`: one-sided Jacobi over a TOURNAMENT sweep schedule (round-robin circle method, replaces the old lexicographic (p,q) nest); sweeps default 30 PINNED (fixed count, not convergence-tested; MUST match across ports); eps 1e-9 skip/zero cutoff; scalar Float32 only, NO SIMD/FMA/Accelerate; sign convention: largest-|entry| of each left vector forced positive; preconditions m ≥ n, rectangular, non-empty; :331 `tournamentRounds(_:)`: pure integer function of n, returns column-DISJOINT (p,q) pairs per round (unit-asserted disjointness), twinned in Rust `JacobiSvd::tournament_rounds` and pinned by a shared schedule hash; each round fans out over `DispatchQueue.concurrentPerform` when `activeProcessorCount` > 1 and the round has 2+ pairs, else runs serially; disjointness makes a round's rotations commute, so output is bit-identical regardless of thread count; :376 `rotatePair(...)`: unsafe-buffer-pointer rotation body, same arithmetic and skip threshold as the pre-tournament serial code; :437 `jacobiCS(alpha:beta:gamma:)`: pinned expression tree (ζ,t,c,s), DO NOT refactor
- FFT.swift:36 `struct Complex` (:51 `magnitude`, :58 `magnitudeSquared`, :63/:68/:73 +,-,*); :83 `enum FFT`: :99 `forward(real:) -> [Complex]` Cooley-Tukey radix-2 DIT, length MUST be power of two (constitutional; callers zero-pad), :162 `magnitudeSpectrum(real:)`; :169/:180 `bitReverse`/`log2Floor` (@usableFromInline); :201 `struct RhythmResult` (dominantPeriodSeconds nil = "no rhythm", honest); :218 `enum RhythmAnalysis`: :229 `analyze(series:bucketDurationSeconds:)` scans bins 1...N/2, energy excludes DC; :281 `analyze(fingerprints:block:bitPosition:bucketDurationSeconds:)`: bit-series extraction, block 0..3 / bit 0..63 preconditions. Production vDSP path must be BIT-IDENTICAL to this scalar reference
- InformationTheory.swift:38 `entropy(_:)`; :48 `mutualInformation(joint:)`; :75 `klDivergence(_:_:)`: length precondition; skips p>0,q=0 terms ⇒ result is a LOWER BOUND; :88 `crossEntropy(_:_:)`; :99 `jensenShannon(_:_:)` bounded [0,1]; :112 `normalizedMutualInformation(joint:)`: ragged matrix → 0 sentinel. All log2 (bits); 0·log0 = 0 by term-skipping; distribution validity is caller's contract

Expand Down
45 changes: 28 additions & 17 deletions packages/libs/SubstrateML/docs/DETAILS.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ sources:
- path: Sources/SubstrateML/InformationTheory.swift
blob: f7ba2a27904ceb19eef0fb5b06d1c5aad8b3894b
- path: Sources/SubstrateML/JacobiSVD.swift
blob: fcd924ff0a8409f224b56e7229f59659a6b5f51e
blob: ca87cf2e57a5469b950ab46f4951ca3a05c1c864
- path: Sources/SubstrateML/LatticeDistance.swift
blob: a7b96a376dfba0c815e4dc91962da1ccaeb2ac1d
- path: Sources/SubstrateML/LLMCalibrationCurve.swift
Expand Down Expand Up @@ -724,26 +724,37 @@ term-document matrix into low-dimensional semantic embeddings, for
latent semantic analysis in CorpusKit.

SVD factors a matrix into rotation, scaling, and rotation:
`A = U S Vᵀ`. The implementation is one-sided cyclic Jacobi. It
repeatedly applies two-by-two planar rotations to each pair of
columns, in a fixed cyclic order. It accumulates the rotations into
V. After the sweeps, the column norms become the singular values.
The normalized columns become U. A sign convention forces the
largest-magnitude entry of each left vector positive. This removes
the inherent plus-minus ambiguity.
`A = U S Vᵀ`. The implementation is one-sided Jacobi. Each sweep
walks a tournament schedule of rounds, not the old fixed
lexicographic order. Within a round every column pair is disjoint,
so the round's rotations touch no shared column. The implementation
applies two-by-two planar rotations to each pair and accumulates the
rotations into V. After the sweeps, the column norms become the
singular values. The normalized columns become U. A sign convention
forces the largest-magnitude entry of each left vector positive.
This removes the inherent plus-minus ambiguity.

Each convergence-affecting choice is pinned, for cross-port
bit-identity. That is why this exists instead of a call to
Accelerate or LAPACK. The sweep count is fixed, at a default of 30,
and is not convergence-tested. The rotation order is fixed. All
arithmetic is scalar Float32, with no SIMD and no fused
multiply-add. The rotation formula in `jacobiCS(alpha:beta:gamma:)`
keeps a pinned expression tree that must not be refactored.
`decompose(A:rank:sweeps:)` requires at least as many rows as
columns. It returns an `SVDResult`: U, non-increasing singular
values, Vt, and rank. The sweep loops use unsafe buffer pointers,
for the same documented vectorization reason as the NMF engine.
Each index is derived from loop bounds, never from data.
and is not convergence-tested. The tournament schedule is a pure
integer function of n. The Rust port twins it exactly in
`JacobiSvd::tournament_rounds`. A shared schedule hash pins both
ports to the identical round order. Cross-port bit-identity holds
exactly as it did under the old lexicographic order. Because a
round's pairs are disjoint, its rotations commute. They may run on
any number of threads with bit-identical output. Output never
depends on thread count. A dense factorization over many columns
once pinned one core for minutes. The tournament order now runs many
rotations at once. All arithmetic is scalar Float32, with no SIMD
and no fused multiply-add. The rotation formula in
`jacobiCS(alpha:beta:gamma:)` keeps a pinned expression tree that
must not be refactored. `decompose(A:rank:sweeps:)` requires at
least as many rows as columns. It returns an `SVDResult`: U,
non-increasing singular values, Vt, and rank. The sweep loops use
unsafe buffer pointers, for the same documented vectorization reason
as the NMF engine. Each index is derived from loop bounds, never
from data.

## FFT.swift

Expand Down
2 changes: 1 addition & 1 deletion packages/libs/SubstrateML/docs/OVERVIEW.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ sources:
- path: Sources/SubstrateML/InformationTheory.swift
blob: f7ba2a27904ceb19eef0fb5b06d1c5aad8b3894b
- path: Sources/SubstrateML/JacobiSVD.swift
blob: fcd924ff0a8409f224b56e7229f59659a6b5f51e
blob: ca87cf2e57a5469b950ab46f4951ca3a05c1c864
- path: Sources/SubstrateML/LatticeDistance.swift
blob: a7b96a376dfba0c815e4dc91962da1ccaeb2ac1d
- path: Sources/SubstrateML/LLMCalibrationCurve.swift
Expand Down
2 changes: 1 addition & 1 deletion packages/libs/SubstrateTypes/docs/AGENT_MAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ sources:
- path: Sources/SubstrateTypes/FNV.swift
blob: f3d8874705ad66f513e49ccbf0c714980a1172e8
- path: Sources/SubstrateTypes/GSetAuditLog.swift
blob: 1a4452e4c8a147d4fac299a3319b0cb4e70cb795
blob: 46508e5af2a855b29ec45b4e02635477e3460544
- path: Sources/SubstrateTypes/Hamming.swift
blob: dc899742f77434910d51bb3cd830cc9603f56f0a
- path: Sources/SubstrateTypes/HLC.swift
Expand Down
2 changes: 1 addition & 1 deletion packages/libs/SubstrateTypes/docs/DETAILS.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ sources:
- path: Sources/SubstrateTypes/FNV.swift
blob: f3d8874705ad66f513e49ccbf0c714980a1172e8
- path: Sources/SubstrateTypes/GSetAuditLog.swift
blob: 1a4452e4c8a147d4fac299a3319b0cb4e70cb795
blob: 46508e5af2a855b29ec45b4e02635477e3460544
- path: Sources/SubstrateTypes/Hamming.swift
blob: dc899742f77434910d51bb3cd830cc9603f56f0a
- path: Sources/SubstrateTypes/HLC.swift
Expand Down
2 changes: 1 addition & 1 deletion packages/libs/SubstrateTypes/docs/OVERVIEW.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ sources:
- path: Sources/SubstrateTypes/FNV.swift
blob: f3d8874705ad66f513e49ccbf0c714980a1172e8
- path: Sources/SubstrateTypes/GSetAuditLog.swift
blob: 1a4452e4c8a147d4fac299a3319b0cb4e70cb795
blob: 46508e5af2a855b29ec45b4e02635477e3460544
- path: Sources/SubstrateTypes/Hamming.swift
blob: dc899742f77434910d51bb3cd830cc9603f56f0a
- path: Sources/SubstrateTypes/HLC.swift
Expand Down
2 changes: 1 addition & 1 deletion packages/libs/SubstrateTypes/docs/topology.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading